Optimizing CESM components from I/O and Load-Balance perspective

07/31/2013 - 10:00am to 11:00am
NCAR Mesa Lab Main Seminar Room
Babak Behzad



Babak Behzad, University of Illinois at Urbana-Champaign

Optimizing CESM components from I/O and Load-Balance perspective 


Although CESM components have been scaled to thousands of processors, there is still room for improvement in their scalability. Two of the major roadblocks to the scalability of current parallel scientific applications are I/O and Load-Balance. We have tried to address both of these issues in this project.

PIO library developed at NCAR is an application-level parallel I/O library for reading/writing distributed arrays from/to different kind of scientific file formats. Currently, the best performing interface in PIO is pNetCDF, outperforming NetCDF-4 interface by a large factor. Since NetCDF-4 offers a wide variety of desirable features, we have tried to characterize the I/O performance of the NetCDF-4 interface and come up with several optimization techniques to increase its performance.

Space-Filling Curves have been implemented and used in different CESM components and have shown much higher performance than other partitioning schemes (such as METIS). However, these implementations are more specific to the grids and climate components. We will present an implementation of a general Space-Filling Curve, which can be used for different grids with different properties.