Inline Data Compression for High-Performance Computing

10/08/2014 - 10:00am to 11:00am
Mesa Lab Main Seminar Room
Peter Lindstrom, Ph.D.



Peter Lindstrom, Ph.D., Lawrence Livermore National Laboratory



With exascale computing on the horizon, the high-performance computing community faces a staggering challenge as to how to solve the data movement bottleneck. Data movement is needed to bring data to and from disk, exchange results between distributed processing nodes, and migrate data between memory banks on tomorrow's massively multicore computers. Not only is the performance cost of such data movement orders-of-magnitude higher than the subsequent cost of computations on the data, but data movement will also overwhelmingly dictate total power consumption at exascale. This imbalance invites the opportunity to spend otherwise wasted compute cycles to reduce data movement.

In this talk, I will discuss one potential strategy toward mitigating the cost of data movement based on removing redundancy in the data representation. I will particularly focus on recent work by our research group on both lossless and lossy compression of floating-point data generated in simulation codes. We are investigating inline, on-demand data compression of the simulation state itself to reduce memory traffic and communication bandwidth, whereas prior data compression work has focused on reducing offline storage and accelerating I/O. I will present a new lossy compression scheme that enables many-fold reduction in memory bandwidth that supports read and write random access that has successfully been used in several Department of Energy simulation codes without significantly affecting the outcome of simulations.

CISL Seminar Recording