SIParCS 2016 - Pranay Reddy Kommera

Pranay Reddy Kommera, University of Wyoming

Optimizing a 2D Discontinuous Galerkin dynamical core for both CPU and GPU execution

(Slides)  (Recorded Talk)

The non-hydrostatic (NH) primitive equations are used in atmospheric models when running at high-resolution, i.e. on the order of a few kilometers, where the vertical motion of atmosphere becomes important. These models require development of spatial and temporal discretization schemes. The Discontinuous Galerkin (DG) method is a popular spatial discretization scheme due to its high-order accuracy, high parallel efficiency and lower interprocessor communication overhead. A 2D prototype NH DG model has been implemented with one vertical plus one horizontal dimension, i.e. in the x-z plane. This method promises to enable high levels of parallelism in a fully 3D model, and is therefore a prime candidate for acceleration using NVIDIA Graphics Processing Units (GPUs). In this summer project, the 2D NH code was modified to support parallelism under both the OpenMP and OpenACC directive-based programming models.

Optimal performance and readability are two important, but often conflicting design goals for any scientific code. The original prototype NH model, written in Fortran, uses arrays of derived data types containing multiple variables. By interleaving variables in memory, this approach creates memory access problems and constrains vector length. We have compared this approach with one using “simple arrays” to describe each variable, which we have shown to be more efficient. We have also compared serial and parallel execution performance. Our results on NVIDIA GPUs using OpenACC show a significant performance improvement relative to the PGI compiler’s serial performance on the CPU. For these tests we have used the Intel 16.0.2 and PGI 16.5 compilers on Yellowstone’s Caldera platform, which contains dual 8-core, 2.6 GHz Intel Sandy Bridge processors and two NVIDIA K20X GPUs per node.

Mentors: Ram Nair, Rich Loft, Raghu Prasanna Kumar, CISL