Apologies for the back to back posts, but it recently became apparent that I needed to install SPECFEM3D on another machine than what I am currently using. The new machine does not appear to have mpirun, however it does have mpiexec installed. I thought mpiexec could work as a substitute for mpirun since it can act as a replacement, but i have found that this is apparently not the case.
When running:
./configure FC=gfortran CC=gcc MPIFC=mpif90
can we substitute mpif90 with mpiexec? I have tried simply running ./configure MPIFC=mpiexec, letting it find the other compilers itself, but this causes failure when the code attempts to use mpi during the examples. Any advice is appreciated. I would like to install mpirun, but have been told by the IT that it would take months to do so on their end.
mpiexec is not a fortran compiler so that won’t work. and most likely it’s just a wrapper on your system. make sure you have a fortran compiler installed together with an MPI library like OpenMPI, MPICH etc., then you can check if mpif90 is available. ask your system admin in case you need help for that. linux distributions all provide different standard mpi libraries.
once installed, you likely will find mpirun on your system now as well. but if preferred, you could replace mpirun with mpiexec (with a slight modification of the command flags) when launching the xmeshfem3D and xspecfem3D binaries in the provided example bash scripts. again, the configuration ./configure needs a fortran compiler with MPI libraries, so mpif90 is your friend (or mpifort with intel compilers, or ftn on cray systems etc.).
Thank you for the response. It looks like I am working with a cray system. I tried to run configure with:
./configure --prefix=$HOME/specfem3d FC=ftn CC=cc MPIFC=ftn CXX=CC
It looks like this doesn’t quite work when I try to make xmesh and xspec in the next steps. Will this require altering the scripts for xmesh3D and xspec3D?
don’t use --prefix … since there is no make install, but you could probably do something manually to set it up like what you have in mind.
for the other parts:
what “doesn’t quite work”?
can you be more specific or attach an error output?
what do you mean by the “scripts” for xmeshfem3D and xspecfem3D?
there are example bash scripts run_this_example.sh in all the EXAMPLES/ folders. they are setup such that you would run the simulation directly in that example folder as working directory (and thus find and link to the binaries in a place like ../../bin/). they are mostly meant to show how to run an example on a workstation or laptop, not on an HPC system. so, these scripts are really just what they are, i.e., examples.
please modify them according to how you want your simulation setup.
so, when you’re on a Cray system (and funny, “it looks like … a cray system” - you would notice when sitting in a ferrari…), you probably want to run the simulations from a /scratch directory and not from within your /home directory because of different filesystems. one way to set this up is to have the SPECFEM3D/ folder somewhere in /home/ for compilation only, and then create and setup a temporary simulation directory on scratch/ where you could copy the binaries and create folders like DATA/ and OUTPUT_FILES/ in your new work directory. well, there are many possibilities, you’ll figure that one out…
Hi @djherm,
I will not be able to answer any SPECFEM specific questions (since I am not one of the developers), but here are some general suggestions that may help you proceed:
As @dpeter mentioned in order to find the problem we need much more detailed information than this doesnt quite work. E.g. did the call to configure succeed? Did running make succeed? What is the step you are currently in? In particular including the screen output, the error message, and all prior steps you took is crucial. Getting an HPC code like SPECFEM to run on a new cluster is a process with a lot of steps, so taking one at a time and realizing which one you are in at the moment is important.
As mentioned Cray systems are highly individualized for high-performance computing, so it is often impossible to give exact directions and you will have to figure out a lot based on the available documentation of your system, or in correspondence with the sysadmins. Most of this will only be relevant once you are actually running models (like what folder to store model output in), but some things are important to consider now: E.g. is it even allowed/possible to run a small test model on a login node (I assume you are working on a cluster with separated login/compute nodes)? If not, how do you get interactive access to a compute node? Maybe compiling works, and you just need to figure out how to start the model and submit it to the batch system?