A PGAS Implementation of the ECMWF Integrated Forecasting System (IFS) NWP Model
George Mozdzynski
European Centre for Medium-Range Weather Forecasts
Reading, England
Abstract
ECMWF is a partner in the Collaborative Research into Exascale Systemware, Tools and Applications (CRESTA) project, funded by a recent EU call for projects in Exa-scale computing, software and simulation (ICT-2011.9.13). The significance of the research carried out within the CRESTA project is that it will demonstrate techniques required to scale the current generation of petascale simulation codes towards the performance levels required for running on future exascale systems.
Within CRESTA ECMWF is exploring the use of Fortran2008 coarrays; in particular it is possibly the first time that coarrays have been used in a world leading production application within the context of OpenMP parallel regions. The purpose of these optimizations is primarily to allow the overlap of computation and communication, and further, in the case of the semi-Lagrangian optimization, to reduce the volume of the data communicated by removing the need for a constant width halo for computing the trajectory of particles of air backwards in time. The importance of this research is such that if these developments are successful then the IFS model can continue to use the spectral method to 2025 and beyond for the currently planned model resolutions on an exascale sized system. This research is further significant as the techniques used should be applicable to other hybrid MPI/OpenMP codes with the potential to overlap computation and communication.
Today ECMWF uses a 16 km global grid for its operational deterministic model, and plans to scale up to a 10 km grid in 2014-15, followed by a 5 km grid in 2023-24, and a 2.5 km grid in 2031-32. These future resolution increases will require IFS to run efficiently on about 4 million cores in the 2030's on an exascale sized system that is expected to be both available and affordable by then.
The current status of the coarray scalability developments to IFS will be presented in this talk, including an outline of other planned scalability developments.