I’m trying to compile ASPECT on a Centos 7 cluster. The support staff built the libraries already and so all that is left for me to do is to compile ASPECT itself, but I’ve run into some issues near the very end. The output gets to 100% and reads:
[ 96%] Building CXX object CMakeFiles/aspect.dir/source/volume_of_fluid/reconstruct.cc.o
[ 96%] Building CXX object CMakeFiles/aspect.dir/source/volume_of_fluid/setup_initial_conditions.cc.o
[ 96%] Building CXX object CMakeFiles/aspect.dir/source/volume_of_fluid/solver.cc.o
[ 97%] Building CXX object CMakeFiles/aspect.dir/source/volume_of_fluid/utilities.cc.o
[ 97%] Building CXX object CMakeFiles/aspect.dir/unit_tests/additional_outputs.cc.o
[ 97%] Building CXX object CMakeFiles/aspect.dir/unit_tests/first.cc.o
[ 98%] Building CXX object CMakeFiles/aspect.dir/unit_tests/parse_map_to_double_array.cc.o
[ 98%] Building CXX object CMakeFiles/aspect.dir/unit_tests/termination_criteria.cc.o
[ 98%] Building CXX object CMakeFiles/aspect.dir/unit_tests/utilities.cc.o
[100%] Linking CXX executable aspect
While the error file gives me the following:
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘klu_free_symbolic’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘klu_free_numeric’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘klu_solve’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘klu_tsolve’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘klu_factor’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘klu_analyze_given’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘klu_analyze’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘klu_defaults’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_BoomerAMGSetup’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_AMSSetup’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_ADSSetup’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_IJMatrixGetObject’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_IJVectorGetObject’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_ADSSetCoordinateVectors’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_ADSSetDiscreteGradient’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_ADSSetDiscreteCurl’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_ADSSetInterpolations’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_AMSSetDimension’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_AMSSetEdgeConstantVectors’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_AMSSetCoordinateVectors’
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/libdeal_II.so.9.1.1: error: undefined reference to ‘HYPRE_AMSSetDiscreteGradient’
And there is probably more than 100 error: undefined references relating to HYPRE/hypre that I decided not to show to keep this message from being too long. I appreciate any help that can be given to help me overcome these errors! Also I’m not sure if this is at all related, but before running the make command to compile ASPECT, the cmake command that constructs the build directory gave me the following output:
– Found deal.II version 9.1.1 at ‘/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/lib/cmake/deal.II’
– Using ASPECT_USE_PETSC = ‘OFF’
– The CXX compiler identification is Intel 18.0.5.20180823
– Check for working CXX compiler: /opt/apps/software/compiler/icc/2018.5.274-GCC-6.3.0-2.26/compilers_and_libraries_2018.5.274/linux/bin/intel64/icpc
– Check for working CXX compiler: /opt/apps/software/compiler/icc/2018.5.274-GCC-6.3.0-2.26/compilers_and_libraries_2018.5.274/linux/bin/intel64/icpc – works
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Detecting CXX compile features
– Detecting CXX compile features - done
– Query git repository information.
– Found Git: /usr/bin/git (found version “1.8.3.1”)
– Setting up test project, see tests/setup_tests.log for details.
– Performing Test HAVE_DLOPEN
– Performing Test HAVE_DLOPEN - Success
– Enabling dynamic loading of plugins from the input file
– Looking for C++ include link.h
– Looking for C++ include link.h - found
– Enabling checking of compatible deal.II library when loading plugins
– World Builder not found.
– Performing Test HAVE_FP_EXCEPTIONS
– Performing Test HAVE_FP_EXCEPTIONS - Failed
– No support for feenableexcept(), disabling runtime floating point exception checks.
– Disabling cotire.
– Parameter GUI not found: install and provide a hint using -D PARAMETER_GUI_DIR or set -D PARAMETER_GUI_EXECUTABLE directly.
– Writing config into detailed.log…
– Configuring done
– Generating done
I have not seen this specific error before when building ASPECT, but perhaps one the key deal.II libraries (or supporting libraries) was not included in the build?. Another thought that this could be an issue related to using the intel compilers, but on closer inspection it looks like gcc is being used?:
Check for working CXX compiler: /opt/apps/software/compiler/icc/2018.5.274-GCC-6.3.0-2.26/compilers_and_libraries_2018.5.274/linux/bin/intel64/icpc
Can you provide a bit more information about the compilers?
The support staff built the libraries already and so all that is left for me to do is to compile ASPECT itself,
After building deal.II, I have compile (if not done by default) and run deal.II step-32, as that is the closest deal.II example to ASPECT and contains a good deal of common functionality.
I appreciate the response! I contacted the support team and they told me that the way they built deal.II is as outlined in:
and they used intel 2018 compilers to do the building. I also tried using the intel 2018 compilers for compiling ASPECT and that gave the error messages that I sent previously. One other thing that I have noticed is that in the output there are lines every so often that say:
/opt/apps/software/devel/Boost/1.69.0-intel-2018.5.274/include/boost/pending/integer_log2.hpp(7): note: This header is deprecated. Use <boost/integer/integer_log2.hpp> instead.
Regarding your other point, I have just tried to run step-32 but that gives me the errors:
– Configuring incomplete, errors occurred!
CMake Error: Cannot open file for write: /opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/examples/step-32/CMakeCache.txt.tmp
CMake Error: : System Error: Permission denied
CMake Error: Unable to open cache file for save. /opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/examples/step-32/CMakeCache.txt
CMake Error: : System Error: Permission denied
[daniel92@node-0001 step-32]$ cmake . && make mpirun -n 2 ./step-32
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineSystem.cmake:174 (file):
file failed to open for writing (No such file or directory):
CMake Error: Could not open file for write in copy operation /opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/examples/step-32/CMakeFiles/3.13.3/CMakeSystem.cmake.tmp
CMake Error: : System Error: No such file or directory
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineSystem.cmake:186 (configure_file):
configure_file Problem configuring file
Call Stack (most recent call first):
CMakeLists.txt:58 (PROJECT)
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineCompilerId.cmake:737 (file):
file failed to open for writing (No such file or directory):
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineCompilerId.cmake:737 (file):
file failed to open for writing (No such file or directory):
– The C compiler identification is unknown
CMake Error: Could not open file for write in copy operation /opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/examples/step-32/CMakeFiles/3.13.3/CMakeCCompiler.cmake.tmp
CMake Error: : System Error: No such file or directory
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineCCompiler.cmake:192 (configure_file):
configure_file Problem configuring file
Call Stack (most recent call first):
CMakeLists.txt:58 (PROJECT)
CMake Error: Could not find cmake module file: /opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/examples/step-32/CMakeFiles/3.13.3/CMakeCCompiler.cmake
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineCompilerId.cmake:161 (file):
file problem creating directory:
/opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/examples/step-32/CMakeFiles/3.13.3/CompilerIdCXX
Call Stack (most recent call first):
/opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineCompilerId.cmake:31 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineCXXCompiler.cmake:107 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:58 (PROJECT)
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineCompilerId.cmake:737 (file):
file failed to open for writing (No such file or directory):
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineCompilerId.cmake:737 (file):
file failed to open for writing (No such file or directory):
– The CXX compiler identification is unknown
CMake Error: Could not open file for write in copy operation /opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/examples/step-32/CMakeFiles/3.13.3/CMakeCXXCompiler.cmake.tmp
CMake Error: : System Error: No such file or directory
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeDetermineCXXCompiler.cmake:190 (configure_file):
configure_file Problem configuring file
Call Stack (most recent call first):
CMakeLists.txt:58 (PROJECT)
CMake Error: Could not find cmake module file: /opt/apps/software/phys/deal.II/9.1.1-intel-2018.5.274/examples/step-32/CMakeFiles/3.13.3/CMakeCXXCompiler.cmake
– Check for working C compiler: /opt/apps/software/compiler/GCCcore/6.3.0/bin/cc
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeTestCCompiler.cmake:25 (file):
file failed to open for writing (No such file or directory):
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeTestCCompiler.cmake:37 (try_compile):
Unknown extension “.c” for file
try_compile() works only for enabled languages. Currently these are:
C CXX
See project() command to enable other languages.
Call Stack (most recent call first):
CMakeLists.txt:58 (PROJECT)
– Check for working C compiler: /opt/apps/software/compiler/GCCcore/6.3.0/bin/cc – broken
CMake Error at /opt/apps/software/devel/CMake/3.13.3/share/cmake-3.13/Modules/CMakeTestCCompiler.cmake:48 (file):
file failed to open for writing (No such file or directory):
You need to copy the step-32/ folder somewhere were you have write permission to test it.
As the others have pointed out, this is likely a problem with the installation of deal.II and will appear if you run step-32.
When looking at the install script for deal.II you posted, I see no mention of trilinos. This will be problematic as well.
Can you post the detailed.log files from the deal.II installation and from your ASPECT build directory?
In addition to posting the files Timo mentioned, I would check to see if the cluster has GNU (or clang) and associated MPI compilers available. If so, you can likely install deal.II in your own work directory following a workflow similar to those for ASPECT on Stampede2. In the past all of us have struggled to build deal.II and ASPECT successfully with the intel compilers, but perhaps that has changed recently?
Intel compilers have more bugs and are less tested in the deal.II community. But I have been using intel 2018 and 2019 compilers on stampede2 and frontera and things work okay.
Intel compilers have more bugs and are less tested in the deal.II community. But I have been using intel 2018 and 2019 compilers on stampede2 and frontera and things work okay.
Thanks for the feedback! I’ll have to post the detailed.log from the deal.II directory in a seperate reply since it is too big. However, the detailed.log from my ASPECT directory is:
As for the step-32 example, I ran it in a directory where I have write permissions and it did indeed give me the same sort of errors as in my first post (the undefined reference errors). There are GNU compilers on the cluster, but when I tried to use these I got errors about licencing:
It fails with the following output:
Change Dir: /home/daniel92/step-32/CMakeFiles/CMakeTmp
Run Build Command:"/usr/bin/gmake" "cmTC_b2bc2/fast"
/usr/bin/gmake -f CMakeFiles/cmTC_b2bc2.dir/build.make CMakeFiles/cmTC_b2bc2.dir/build
gmake[1]: Entering directory `/home/daniel92/step-32/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_b2bc2.dir/testCXXCompiler.cxx.o
/opt/apps/software/compiler/icc/2018.5.274-GCC-6.3.0-2.26/compilers_and_libraries_2018.5.274/linux/bin/intel64/icpc -o CMakeFiles/cmTC_b2bc2.dir/testCXXCompiler.cxx.o -c /home/daniel92/step-32/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
Error: A license for Comp-CL could not be obtained (-1,359,2).
Is your license file in the right location and readable?
The location of your license file should be specified via
the $INTEL_LICENSE_FILE environment variable.
License file(s) used were (in this order):
** 1. /home/daniel92/Licenses
** 2. /opt/apps/software/compiler/icc/2018.5.274-GCC-6.3.0-2.26/compilers_and_libraries_2018.5.274/linux/bin/intel64/../../Licenses
** 3. /opt/intel/licenses
** 4. /Users/Shared/Library/Application Support/Intel/Licenses
** 5. /opt/apps/software/compiler/icc/2018.5.274-GCC-6.3.0-2.26/compilers_and_libraries_2018.5.274/linux/bin/intel64/*.lic
Apologies for the triple post but the detailed.log was too big to even fit in one post by itself! Here is the last little bit of the file that I trimmed off to meet the character requirements:
DEAL_II_WITH_UMFPACK set up with external dependencies
I’m not certain about how they included trilinos when they built deal.II, but when I first asked them to build deal.II they built deal.II, p4est, petsc and trilinos seperately. I told them that that was preventing me from compiling ASPECT and they rebuilt deal.II with the existing installions of trilinos the other libraries. I am running on a compute node yes, you mentioned that sometimes intel compilers can be buggy, would using GNU compilers potentially fix the issues that I’m having?
I’m not certain about how they included trilinos when they built deal.II
They did point to a pre-installed Trilinos library, but we don’t what options they used to build it.
I am running on a compute node yes, you mentioned that sometimes intel compilers can be buggy, would using GNU compilers potentially fix the issues that I’m having?
I think the simplest thing for the cluster admins to do next is to use candi to build Trilinos, p4est and deal.II with the intel libraries. Using candi requires minimal user time and Timo has successfully built deal.II/ASPECT on Stampede2 with the intel compilers (2018/2019). From the link I posted for Stampede2 above, all one would need to do is replace the gcc compilers there with the appropriate intel compilers, mkl library, etc. Also, the most recent deal.II version is now 9.1.1 (not 9.0.0).
If that produces lots of bugs, then switching to GNU compilers would be the right path forward. In any case, building with candi makes the whole process relatively painless.
I’ll certainly try my hand at installing the libraries using the link you sent. Here’s hoping it works, I’ll post in here once it works or I run into another suite of problems, once again I appreciate all the help!