IBM p575

Clock Speed: 
Dates Used: 
Saturday, August 27, 2005
Microprocessor Peak Teraflops: 
Memory (terabytes): 
Number of Processors: 
Electrical Power Consumption: 
210.60 kW

The Scientific Computing Division, part of NCAR's Computational Science and Information Systems Laboratory, took delivery of a new IBM p5-575 supercomputer named "bluevista" on August 27, 2005.

The new system, based on IBM's POWER5 processor and High-Performance Switch technology, has approximately the same sustained computing capacity as bluesky, NCAR's IBM POWER4 system, and nearly doubles the number of computational cycles available to users.

At 1.9 gigahertz (GHz), the POWER5 processor runs at a faster clock speed than the POWER4 processor (1.3 GHz). In addition, the POWER5 processor also can sustain a higher memory bandwidth. Thus, NCAR applications on bluevista show enhanced performance through the faster clock and a higher percent of peak computational capacity.

The bluevista system consists of 78 IBM POWER5 p5-575 Symmetric Multi-Processor (SMP) nodes. Each node has eight POWER5 simultaneous multithreading (SMT) processors and 16 gigabytes of memory.

Nodes are allocated as follows:

  • 72 nodes for computational workload
  • 2 nodes for interactive login sessions
  • 4 nodes for filesystem input/output, management of the 55-terabyte disk subsystem, and connectivity to NCAR's Mass Storage System

All nodes of the system are interconnected with IBM's pSeries High-Performance Switch (HPS), previously known as the Federation switch. The HPS provides a single-link, unidirectional, point-to-point communication bandwidth of 2 gigabytes per second and latency of seven microseconds. Each node of the system will have two bidirectional link interfaces to the HPS.

While bluevista provides roughly the same sustained computational capacity as bluesky on NCAR models, it is a “denser, hotter” system. Bluevista need 276 kilowatts of power to operate (the average U.S. home consumes 10.5 kilowatts). It occupies only a third of the floor space as bluesky, but requires two-thirds of bluesky's power and cooling.

SCD initially allocated bluevista to experimental research for the Nested Regional Climate Model (NRCM) project. NRCM is a joint collaboration between NCAR's Mesoscale and Microscale Meteorology Division and the Climate and Global Dynamics Division. Researchers are using bluevista to study the effects of tropical convection on hurricanes and other tropical weather systems, using a regional climate model embedded in a global model. This work is relevant to the question of whether global climate change is helping produce more Category 4 and 5 hurricanes (the most powerful). The problem is so computationally demanding that it occupied all of bluevista's available time from October through December 2005. SCD made bluevista available for general use in January 2006.

SCD acquired bluevista as part of an option in NCAR's Advanced Research Computing System (ARCS) contract with IBM. The decision was made after consulting with NCAR scientists and management and with the SCD Advisory Panel. Bluevista is the last in a sequence of supercomputer systems and system upgrades acquired under the ARCS contract, which has had four phases.

The National Science Foundation, NCAR's primary sponsor, funded the computer through a special award from its Division of Atmospheric Sciences, with research support from the NSF Office of Polar Programs.