Aspect hangs at same timesteps when using isosurfaces stratagey

ASPECT
1

Hello all,
I am using isosurfaces strategy but it stuck when using More than or equal to 8 refinement. At 7 refinement it works but my slab breaks. I am using 32 cores.
I am using this code
set Isosurfaces = 8, 8, weak_layer : 0.25 | 2;8, 8, lith : 0.25 | 2; 8.8, oceanic_crust : 0.25 | 2;

The problem is that in the log file we are not getting any error messages. The run simple stops at about the same time step (between 149 and 179). It looks like ASPECT is hanging.

We believe this is not related to the degrees of freedom, because without isosurfaces it successful runs had up to 7 million degrees of freedom at the final time step, while the problematic runs reach only between ~410,000 and ~780,000 degress of freedom, which is off course significantly less than a full run.
I appreciate your help.

2

@sharmamd,

The run simple stops at about the same time step (between 149 and 179). It looks like ASPECT is hanging.

A few questions to try and help diagnose the issue:

  1. Can you post the an example of the place in the log.txt where ASPECT hangs?
  2. Is the location (i.e., point in the log.txt) where the model hangs always consistent?
  3. Have you tried running in debug mode to see if more if this gives more information about why ASPECT is hanging?
  4. What version of ASPECT and deal.II are you currently using, and what type of system (laptop, desktop, cluster; OS) are you running on?
  5. How many cores are you using?

Cheers,
John

3

log.txt (200.7 KB)
I have attached log.txt file here.
Yes location where it hangs is same, with 8 refinement it stuck around 170 timestep and with refinement 9 it stuck around 148 timestep every time. I also restart but it stuck at same timestep.
No I not yet tried in debug mode.
I am using my local machine with 32 cores and I am using ASPECT v2.6.0-pre.
32 cores I used.

Apart from this I also find some weird errors. I fixed the range of top and bottom temperature which is 293K and 1673K but If I used isosurfaces it goes beyond these values after sometime step.

Thanks

4

@sharmamd The very first thing you should do when encountering a bug, hanging jobs, or anything else, is to run the program in debug mode. deal.II and ASPECT together have 17,000 places where we check the correctness of the internal state of the program, or of program inputs. Make use of this resource! Trying to find the cause of a problem in release mode is like trying to find a needle in a haystack while being blindfolded.
Best
W.

5

@sharmamd - In parallel to running the models in debug mode, you may also want to try building the current development version of ASPECT (now on 3.1.0-pre) and checking if the issue still persists. However, it is still worth taking the time to determine what the current issue is via debug mode, regardless of whether it a newer version of ASPECT (or deal.II) fixes it!

Cheers,
John

6

Thanks a lot @bangerth for helping me in this, I will try with debug mode.

7

Thanks for helping me, yes first I checked with debug mode than also update aspect.