Failing to compile trilinos with candi on Ubuntu 22.04

ASPECT
,
1

hi,
I am trying to install ASPECT + dependencies using candi. My workstation is a Dell tower (Intel® Xeon(R) Gold 6226R CPU @ 2.90GHz × 64 , 192 Gb) running Ubuntu 22.04.
It is a brand new computer, so in candi.cfg I have removed petsc and added openblas and scalapack (I also previoulsy installed git, cmake, etc … as recommended). Candi version is 9.4.0-r2, it’s trying to install trilinos 12-18-1. Here is the error message:

10%] Built target trilinosss
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/create_proc_list.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/phg/phg_Vcycle.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/box_assign.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/inertial1d.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/inertial2d.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/inertial3d.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/point_assign.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/rcb_box.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/rcb_partition_tree.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/rcb.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/rcb_util.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/rib.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/rib_util.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/rcb/shared.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/reftree/reftree_coarse_path.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/reftree/reftree_build.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/reftree/reftree_hash.c.o
[ 11%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/reftree/reftree_part.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/simple/block.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/simple/cyclic.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/simple/random.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/timer/timer_params.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Communication/comm_exchange_sizes.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Communication/comm_invert_map.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Communication/comm_do.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Communication/comm_do_reverse.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Communication/comm_info.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Communication/comm_create.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Communication/comm_resize.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Communication/comm_sort_ints.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Communication/comm_destroy.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Communication/comm_invert_plan.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Timer/zoltan_timer.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Timer/timer.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Find.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Set_Neighbor_Hash_Fn3.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Destroy.c.o
[ 12%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Create.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Remove.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Stats.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Hash2.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Update.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Memory.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Print.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Set_Neighbor_Hash_Fn2.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Set_Neighbor_Hash_Fn1.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/Memory/mem.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/DDirectory/DD_Set_Hash_Fn.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/shared/zoltan_align.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/Utilities/shared/zoltan_id.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_coord.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_gen_files.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_hash.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_map.c.o
[ 13%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_heap.c.o
[ 14%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_init.c.o
[ 14%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_obj_list.c.o
[ 14%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_rand.c.o
[ 14%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_set_fn.c.o
[ 14%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_sort.c.o
[ 14%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_struct.c.o
[ 14%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_back_trace.c.o
[ 14%] Building C object packages/zoltan/src/CMakeFiles/zoltan.dir/zz/zz_util.c.o
[ 14%] Linking C shared library libzoltan.so
[ 14%] Built target zoltan
[ 14%] Linking CXX shared library libpamgen.so
[ 14%] Built target pamgen
[ 14%] Linking CXX shared library libgtest.so
[ 14%] Built target gtest
make: *** [Makefile:166: all] Error 2
Failure with exit status: 2
Exit message: There was a problem building trilinos 12-18-1.

When re-running ./candi.sh I now get:

885 | static_assert(std::__is_complete_or_unbounded(__type_identity<_Tp>{}),
| ~~~~~~~~~^~
/usr/include/c++/11/type_traits:885:52: note: ‘std::__is_complete_or_unbounded<std::__type_identity<std::function<void()> > >((std::__type_identity<std::function<void()> >{}, std::__type_identity<std::function<void()> >()))’ evaluates to false
make[2]: *** [packages/kokkos/core/src/CMakeFiles/kokkoscore.dir/build.make:90: packages/kokkos/core/src/CMakeFiles/kokkoscore.dir/impl/Kokkos_Core.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:4156: packages/kokkos/core/src/CMakeFiles/kokkoscore.dir/all] Error 2
make: *** [Makefile:166: all] Error 2
Failure with exit status: 2
Exit message: There was a problem building trilinos 12-18-1.

Any help would be appreciated,
Neil

2

Neil:
I’d need to see more of the error message above where it starts in line 885. But in any case, since you’re on Ubuntu, you could also just install the deal.II package via apt-get, which will then pull in a complete version of Trilinos and all other dependencies. Then you only need to deal with installing ASPECT.
Best
Wolfgang

3

It worked! ASPECT now runs on the machine. Thanks you!

4

Great, glad to hear!

5

Dear Wolfgang (if I may),

Many thanks for your remarkably quick and helpful response,
which solved the problem! I’m delighted that I can now join
the ASPECT user community.

Best wishes,

Neil (Ribe)

6

Hi all,

Myself and a few others also ran into this error with the standard compilers on Ubuntu 22.04 for deal.II 9.4.0. We are getting the same errors with Trilinos, and the full candi build log is attached here. There are a large number of error messages, with nearly (or all) occurring during the Kokkos build.

Neil, Cedric - I Just checked and the version of deal.II available through apt-get is 9.3.2. This is still compatible with ASPECT, but some recent features (particle optimizations, GMG + free surface & elasticity) will not work. Hopefully we can get deal.II 9.4.0 working on ubuntu 22.04 relatively soon, but if you need 9.4.0 before then an option would be to run it in a docker container on your machine.

Wolgang - it was on my list for this week to open an issue on the candi repo, and if needed also one on the Trilinos repo. It was not apparent to me what the correct fix is when looking at the error messages. Are there any other steps I can take to help with the debugging. FYI, I tried an earlier version of Trilinos (12.4.1 versus 12.18), but that build also failed.

Cheers,
John
candi_build.log (3.6 MB)

7

We are getting the same errors with Trilinos

Somehow I forgot about this problem. The fix is the open PR update Trilinos to 13.2 by tjhei · Pull Request #297 · dealii/candi · GitHub that updates Trilinos to 13.2.

I Just checked and the version of deal.II available through apt-get is 9.3.2

9.4 is available as a backport. See Getting deal.II · dealii/dealii Wiki · GitHub

8

Timo:
Do you happen to know whether the issue with Trilinos 12.8 is one of the compiler used on Ubuntu 22.04, or one that rests in how candi sets things up? Since this happens before we even compile deal.II, it isn’t a problem in the latter.

The PR you link to states that Trilinos 13.2 can be compiled, but does not actually have all of the functionality we need (at least not if Tpetra is enabled). What is our longer term strategy? Or should we discuss this on the Trilinos call next week?

Best
W.

9

candi does not set up compilers. The system compiler of 22.04 is “gcc version 11.2.0 (Ubuntu 11.2.0-19ubuntu1)”.

Not sure. We could disable Tpetra (I don’t know of anyone using it through deal.II). Let’s discuss with the other deal.II developers.

10

Dear John (if I may),

Many thanks for your reactivity on this - it’s very helpful indeed!

Best wishes,

Neil Ribe

11

Dear Timo (if I may),

Many thanks for your reactivity on this - it’s very helpful indeed!

Best wishes,

Neil Ribe

12

I merged the pull request into candi (master and 9.4 branch), so you should get Trilinos 13.2 by default now.

13

Dear Timo,

Many thanks for taking care of this!

Best wishes,

Neil

14

Indeed, thanks Timo! As a side note, the deal.II 9.4.0 install via the backport worked without issue and was a very quick install.

Cheers,
John