# SPECFEM3D\_GLOBE: Attempting to use multiple nodes on Cray System

**URL:** <https://community.geodynamics.org/t/specfem3d-globe-attempting-to-use-multiple-nodes-on-cray-system/1949>\
**Category:** SPECFEM\
**Tags:** specfem3d\
**Created:** [August 12, 2021, 4:53pm UTC](https://community.geodynamics.org/t/specfem3d-globe-attempting-to-use-multiple-nodes-on-cray-system/1949 "2021-08-12T16:53:22Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![djherm](https://yyz2.discourse-cdn.com/flex036/user_avatar/community.geodynamics.org/djherm/32/546_2.png) [@djherm](https://community.geodynamics.org/u/djherm)\
**Post date:** [August 12, 2021, 4:53pm UTC](https://community.geodynamics.org/t/specfem3d-globe-attempting-to-use-multiple-nodes-on-cray-system/1949/1 "2021-08-12T16:53:22Z")

</div>

Since there was another query posted on Specfem on a Cray Machine, I thought I would add another. Recently, I installed SPECFEM3D\_GLOBE on a Cray machine. I believe I have it successfully compiled and installed as I have been able to generate .sac and .ascii files when I am using 1 node with multiple processors/cores with it. However, there seems to be an issue when requesting runs with multiple nodes. When attempting to run a job with multiple nodes, I usually receive an error message when xmeshfem begins along the lines of:  
Rank 337 [Thu Aug 5 11:16:43 2021] [c8-1c1s2n3] application called MPI\_Abort(MPI\_COMM\_WORLD, 30) - process 337  
forrtl: error (76): Abort trap signal  
Image PC Routine Line Source  
xmeshfem3D 000000000059DEEB for\_\_signal\_handl Unknown Unknown  
libpthread-2.26.s 00002AAAABD6E2D0 Unknown Unknown Unknown  
libc-2.26.so 00002AAAAC2EB520 gsignal Unknown Unknown  
libc-2.26.so 00002AAAAC2ECB01 abort Unknown Unknown  
libmpich\_intel.so 00002AAAAB319218 Unknown Unknown Unknown  
libmpich\_intel.so 00002AAAAB226566 MPI\_Abort Unknown Unknown  
libmpich\_intel.so 00002AAAAB26DA05 mpi\_abort Unknown Unknown  
xmeshfem3D 000000000042F8A7 Unknown Unknown Unknown  
xmeshfem3D 00000000004188DA Unknown Unknown Unknown  
xmeshfem3D 0000000000455183 Unknown Unknown Unknown  
xmeshfem3D 000000000045EF77 Unknown Unknown Unknown  
xmeshfem3D 0000000000470B05 Unknown Unknown Unknown  
xmeshfem3D 00000000004ABC79 Unknown Unknown Unknown  
xmeshfem3D 000000000048C92F Unknown Unknown Unknown  
xmeshfem3D 0000000000483C3F Unknown Unknown Unknown  
xmeshfem3D 000000000044A21D Unknown Unknown Unknown  
xmeshfem3D 000000000040BB52 Unknown Unknown Unknown  
libc-2.26.so 00002AAAAC2D634A \_\_libc\_start\_main Unknown Unknown  
xmeshfem3D 000000000040BA6A Unknown Unknown Unknown  
If it is not this, I have also noticed an error message like this: " PBS: job killed: walltime 947 exceeded limit 900" is quite common. However, when I increase the available walltime it still runs into issues and give an output such as “PBS: job killed: walltime 2014 exceeded by limit 2000”. This walltime message does not occur at the end of my script but rather when I try to start xmeshfem. I am curious if these two messages could be related or are different issues? I have tried multiple combinations of processors and dimensions (NEX\_XI & NPROC\_XI, etc.) following the chart on pg. 23 in the 3D\_GLOBE manual and runtimes, yet this does not appear to resolve the issue. By any chance, is anyone familiar with what might be causing this or are familiar with the error messages? I am not sure if this is specifically related to it being on a Cray machine or just some of my inexperience with working on programs like this. Any advice or suggestions would be appreciated.