HPC and Big Data: COLA's Experience in the Advanced Scientific Discovery Program
Dr. James L. Kinter
Center for Ocean-Land-Atmosphere Studies (COLA)
George Mason University
Fairfax, Virginia
Abstract
At the installation of the Yellowstone high-performance computing (HPC) system at NCAR – an IBM iDataPlex with over 72,000 cores – the system was turned over to a small set of high-end projects that could make substantial use of the system before it went into full-scale production for the entire NCAR computing community. This program, called the Advanced Scientific Discovery (ASD) program, allocated millions of core-hours to each project. The Center for Ocean-Land-Atmosphere Studies (COLA), in partnership with the European Centre for Medium-range Weather Forecasts (ECMWF), received an allocation of 21 million core-hours, and conducted an unprecedented seasonal climate prediction experiment that consumed well over the allocated amount and generated over a petabyte of output data. In this presentation, the general nature of the numerical experiments and output data sets will be described, along with some preliminary scientific results and lessons learned from both the ASD experience and the earlier experience in Project Ath.