Specfem3d_Cartesian AMD HIP Port Merge

[ChinchoreTejas]

Dear all,

We have ported the Specfem3d_Cartesian application to AMD platform. The application tested on AMD GPU’s (Single node Multi GPU Setup). Specfem3d ported code work on both NVIDIA and AMD platforms. We would like to contribute this ported code to Specfem3d_Cartesian original repository.

Kindly let us know your viewpoint for the same.

Please find the code here: https://github.com/ROCmSoftwarePlatform/SPECFEM3D

[dpeter]

great, that’s fun work!

this could also become very helpful for the upcoming Frontier supercomputer. unfortunately, SPECFEM is not part of the Frontier CAAR projects (https://www.olcf.ornl.gov/caar/frontier-caar/), but providing AMD GPU support is excellent to have.

let me see how we could add this to the official version. in principle, it would be preferable to add HIP support rather than substitute CUDA:

• maybe a wrapper similar to what ESPResSo is doing (https://github.com/espressomd/espresso/blob/8a8856a6f6cf86e706ed2734636c00ebb760975d/src/core/cuda_wrapper.hpp) could be a good starting point.
• your HIP/CUDA kernels look pretty much identical. the kernel launches and cuda-function names converted to HIP look mostly like a renaming exercise.
• is there any major design change required compared to how the CUDA kernels were setup? what about preferred kernel block sizes and performance tuning for Radeon GPUs?

many thanks!

[ChinchoreTejas]

[AMD Official Use Only - Internal Distribution Only]

Hi Peter,

Thanks for your response, we will check and update to you on open points.

Regards,
Tejas