How to use GMG solver with global coarsening?

ASPECT
1

Hi everyone,

I’m trying the block GMG solver o run a global model using prescribed velocity. The ASPECT version is 3.1-pre (last week on GitHub) and Dealii is 9.6.2
slurm-30493200.out.txt (20.0 KB)
. When I use local smoothing method, the model can run normally as before. However, when I turn to global coarsening method, it reports such errors:

0  /usr/lib64/libucs.so.0(+0x17970) [0x2b6d8b861970]
1  /usr/lib64/libucs.so.0(+0x17b22) [0x2b6d8b861b22]
2  /public3/home/t6s010829/para/install/dealii/lib/libdeal_II.so.9.6.2(_ZN6dealii8internal49evaluate_tensor_product_value_and_gradient_shapesILi3ENS_5PointILi3EdEENS_15VectorizedArrayIdLm2EEELi1ELb1ELi1EEESt5arrayINS0_18ProductTypeNoPointIT0_T1_E4typeEXplT_T2_EEPKNS0_7ndarray11HelperArrayIS9_JLm2EXT_EEE4typeEiPKS8_RKSt6vectorIjSaIjEE+0x1fb5) [0x2b693f60b5b5]
3  /public3/home/t6s010829/para/install/dealii/lib/libdeal_II.so.9.6.2(_ZN6dealii8internal42evaluate_tensor_product_value_and_gradientILi3ENS_5PointILi3EdEENS_15VectorizedArrayIdLm2EEEEESt4pairINS0_18ProductTypeNoPointIT0_T1_E4typeENS_6TensorILi1EXT_ESB_EEERKSt6vectorINS_11Polynomials10PolynomialIdEESaISI_EERKNS_9ArrayViewIKS8_NS_11MemorySpace4HostEEERKNS2_IXT_ES9_EEbRKSF_IjSaIjEE+0x96d) [0x2b693f60d50d]
4  /public3/home/t6s010829/para/install/dealii/lib/libdeal_II.so.9.6.2(_ZN6dealii8internal22MappingQImplementation39maybe_update_q_points_Jacobians_genericILi3ELi3EEEvNS_14CellSimilarity10SimilarityERKNS_8MappingQIXT_EXT0_EE12InternalDataERKNS_9ArrayViewIKNS_5PointIXT_EdEENS_11MemorySpace4HostEEERKSt6vectorINS_11Polynomials10PolynomialIdEESaISM_EERKSJ_IjSaIjEERSJ_INSB_IXT0_EdEESaISV_EERSJ_INS_14DerivativeFormILi1EXT_EXT0_EdEESaIS10_EERSJ_INSZ_ILi1EXT0_EXT_EdEESaIS14_EE+0x56f) [0x2b693f613b9f]
5  /public3/home/t6s010829/para/install/dealii/lib/libdeal_II.so.9.6.2(_ZN6dealii8internal22MappingQImplementation22do_fill_fe_face_valuesILi3ELi3EEEvRKNS_8MappingQIXT_EXT0_EEERKNS_13TriangulationIXT_EXT0_EE13cell_iteratorEjjNS_10QProjectorIXT_EE17DataSetDescriptorERKNS_10QuadratureIXmiT_Li1EEEERKNS4_12InternalDataERKSt6vectorINS_11Polynomials10PolynomialIdEESaISP_EERKSM_IjSaIjEERNS0_22FEValuesImplementation18MappingRelatedDataIXT_EXT0_EEE+0x194) [0x2b693f617254]
6  /public3/home/t6s010829/para/install/dealii/lib/libdeal_II.so.9.6.2(_ZNK6dealii8MappingQILi3ELi3EE19fill_fe_face_valuesERKNS_12TriaIteratorINS_12CellAccessorILi3ELi3EEEEEjRKNS_2hp11QCollectionILi2EEERKNS_7MappingILi3ELi3EE16InternalDataBaseERNS_8internal22FEValuesImplementation18MappingRelatedDataILi3ELi3EEE+0x175) [0x2b693f617855]
7  /public3/home/t6s010829/para/install/dealii/lib/libdeal_II.so.9.6.2(_ZN6dealii12FEFaceValuesILi3ELi3EE9do_reinitEj+0x197) [0x2b693f709337]
8  /public3/home/t6s010829/para/install/dealii/lib/libdeal_II.so.9.6.2(_ZN6dealii11VectorTools8internal26map_face_dof_to_vector_dofILi3ELi3EEEvRKNS_10DoFHandlerIXT_EXT0_EE13cell_iteratorEjRKSt3setIjSt4lessIjESaIjEERNS_2hp12FEFaceValuesIXT_EXT0_EEERKNS_8IndexSetEjRSt3mapINS1_11FaceDoFInfoESt5arrayIjXT_EES9_ISN_ESaISt4pairIKSN_SP_EEE+0x286) [0x2b693e05abd6]
9  /public3/home/t6s010829/para/install/dealii/lib/libdeal_II.so.9.6.2(+0x2a71209) [0x2b693e05e209]
10  /public3/home/t6s010829/para/install/dealii/lib/libdeal_II.so.9.6.2(_ZN6dealii11VectorTools8internal55compute_nonzero_normal_flux_constraints_active_or_levelILi3ELi3EdEEvRKNS_10DoFHandlerIXT_EXT0_EEEjRKSt3setIjSt4lessIjESaIjEERKSt3mapIjPKNS_8FunctionIXT0_ET1_EES9_SaISt4pairIKjSJ_EEERNS_17AffineConstraintsISG_EERKNS_7MappingIXT_EXT0_EEEbRKNS_8IndexSetEj+0x1518) [0x2b693e0602b8]
11  /public3/home/t6s010829/para/install/dealii/lib/libdeal_II.so.9.6.2(_ZN6dealii11VectorTools34compute_no_normal_flux_constraintsILi3ELi3EdEEvRKNS_10DoFHandlerIXT_EXT0_EEEjRKSt3setIjSt4lessIjESaIjEERNS_17AffineConstraintsIT1_EERKNS_7MappingIXT_EXT0_EEEb+0x217) [0x2b693e060a77]
12  aspect-release(_ZN6aspect53StokesMatrixFreeHandlerGlobalCoarseningImplementationILi3ELi2EE10setup_dofsEv+0x1cfd) [0x103541d]
13  aspect-release(_ZN6aspect9SimulatorILi3EE10setup_dofsEv+0x377) [0x1348637]
14  aspect-release(_ZN6aspect9SimulatorILi3EE11refine_meshEj+0xdd1) [0x134bff1]
15  aspect-release(_ZN6aspect9SimulatorILi3EE27maybe_do_initial_refinementEj+0x3e) [0x134c7ae]
16  aspect-release(_ZN6aspect9SimulatorILi3EE3runEv+0x2f3) [0x134ccc3]
17  aspect-release() [0x1aac20c]
18  aspect-release(main+0x3140) [0xb41f50]
19  /lib64/libc.so.6(__libc_start_main+0xf5) [0x2b695fab9555]
20  aspect-release() [0xb42a50]

I would like to know how to solve this problem and whether new options should be added when compiling dealii.
The log file is also attached.

Thanks a lot!
Xinyu Li

2

The global coarsening GMG is a very early and experimental feature and is missing many features. In fact, you might be one of the first people to try it on something complicated. :slight_smile:

I am keeping track of things to do here: Global Coarsening GMG · Issue #6831 · geodynamics/aspect · GitHub

If you want to help, try to make a small example that you can run on your computer in debug mode. Then you can help me by posting the error message and call stack that is only available in debug mode.

Thanks!

3

@lixinyv When encountering errors, it’s always useful to run in debug mode first. Can you switch to debug mode and see whether you get a more useful error message?

Best

W.

4

Hi @tjhei

Thanks for your suggestion. I have updated the dealii version to deal.II-9.7.0. I added some outputs to determine the error location, using 1920 cpu cores to run the program. It seems that the error occurs in the Line 1672 (original code) of the file source/simulator/solver/stokes_matrix_free_global_coarsening.cc, in the function StokesMatrixFreeHandlerGlobalCoarseningImplementation<dim, velocity_degree>::setup_dofs(). This line is

dof_handler.distribute_dofs(fe_p);

The following is the program output and commented code file:

log
log1680×1216 79.5 KB

comm_code
comm_code1387×883 163 KB

The two files are attached here:
stokes_matrix_free_global_coarsening.cc.txt (90.3 KB)
slurm-30620697.out.txt (4.8 KB)

For the debug mode, I met some difficulties these days and would try it in the following. @tjhei @bangerth

Thanks a lot!
Xinyu Li

5

Hi @bangerth @tjhei

Here is the first log file using debug mode. The reported problem is about velocity boundary condition.
slurm-30621415.out.txt (327.0 KB)

The following is my velocity boundary condition setting:

subsection Boundary temperature model
  set Fixed temperature boundary indicators = 0,1
end

subsection Boundary velocity model
  set Tangential velocity boundary indicators = 0
end

subsection Boundary velocity model
  set Prescribed velocity boundary indicators = 1:gplates
end

subsection Boundary velocity model
  subsection GPlates model
    set Data directory = ../look_up_tables/Liu23/lat_lon_velocity_domain_1200_2400/

    set Velocity file name = velocity.%d.txt
    set Data file time step = 1e6
    set First data file number = 240
    set Decreasing file order = true
    set Lithosphere thickness = 1e3
  end
end

I modified the GPlates plugin to reduce memory consumption and it has been verified using local smoothing GMG.

Thanks a lot!
Xinyu Li

6

Hi,

After commented the Assertion, the new report with debug mode occured after the return of the Function Simulator::setup_dofs ().
slurm-30621492.out.txt (31.5 KB)

Thanks a lot!
Xinyu Li

7

Xinyu,

I left the Assertion in the code, because other velocity boundary conditions are not yet supported/implemented for global coarsening GMG, sorry.