IBM BlueGene/L

Clock Speed: 
Dates Used: 
Friday, March 25, 2005 to Thursday, May 31, 2012
Microprocessor Peak Teraflops: 
Memory (terabytes): 
Number of Processors: 
8 192.00
Electrical Power Consumption: 
83.10 kW
Experimental and Production

NCAR's IBM BlueGene/L system was delivered on March 15, 2005. It went online on March 25 after passing five days of acceptance testing on the first try. Named "Frost" because it ran cooler than most microprocessor-based high-end systems, originally planned to be at NCAR for three years, the system ultimately served NCAR, CU-Boulder, and TeraGrid users for more than seven years.

Initially comprising just one Blue Gene/L rack, Frost had 1,024 dual-processor compute nodes and 32 I/O nodes. Each processor ran at 0.7 GHz, which is relatively slow — for example, Intel Pentium processors of that time ran at 3.2 GHz. However, because Frost's processors were slow, they produced less heat and could be more tightly packed. Thus, the Frost system:

  • Took up 3% of the floor space of NCAR's flagship computer at the time, an IBM p690 named Bluesky
  • Used 6% of the electricity required by Bluesky
  • Delivered 69% of Bluesky's peak computational power

Frost, with one cabinet containing 2,048 processors, was far more compact than Bluesky, which had 50 cabinets that each contained 32 processors. By one key measure, Frost was also more powerful. Although Bluesky had a top performance of 8.3 teraflops, it achieved just 4.2 teraflops on the Linpack benchmark. Frost, in contrast, ran the Linpack benchmark at 4.6 teraflops.

Frost seemed to be tilted because a triangular-shaped air duct, or plenum, was attached to it on either side. The plenum on the left directed cold air from below the floor through the machine. The plenum on the right directed hot exhaust air upward.

Because Frost originally was an experimental system and lacked a complete computational environment for users, it was not used for production computing initially. CISL (still called the Scientific Computing Division, or SCD, at the time of Frost's installation) first used it for computer science research: running performance I/O tests, repartitioning the architecture into smaller blocks to accommodate a variety of users, setting up job queues, and writing tools to support the machine. SCD also collaborated with researchers at the University of Colorado, IBM, Argonne National Laboratory, Lawrence Livermore National Laboratory, and the San Diego Supercomputer Center to see how the Blue Gene/L architecture could best be used for atmospheric science. Joint projects includeed testing applications, debugging software, developing new system configurations, and evaluating job schedulers.

In addition, SCD joined the Blue Gene/L Consortium, an association of laboratories, universities, and industrial partners working to develop scientific and technical applications for Blue Gene/L. SCD made the machine available to select users in May 2005 for experimental purposes.

In July 2007, Frost entered service as a production computer as a TeraGrid resource, while it continued to serve as a production resource for NCAR-CU collaborations. In 2009, Frost was quadrupled in size with three additional Blue Gene/L racks from the San Diego Supercomputer Center. In this final configuration, with 8,192 processors, Frost sustained 22 teraflops on the Linpack benchmark. Frost ended four years of TeraGrid service when the TeraGrid program ended on July 31, 2011. However, CISL was able to keep the system running to support a few NCAR and CU-Boulder collaborations, as well as to support the Asteroseismic Modeling Portal (AMP) gateway. Notably, after its TeraGrid retirement, usage of the six-year-old Frost increased, delivering more than 3 million core-hours per month on average to its small set of devoted users.

In total, Frost delivered more than 126 million core-hours to TeraGrid and non-TeraGrid users over the 58 months between July 2007 and May 2012.