CISL High-Performance Computing (HPC) Benchmarks

31 March 2020 - Benchmark Q&As updated
For questions and answers regarding the NWSC-3 HPC Benchmarks, refer to the updated NWSC-3 Benchmarks Q&As document.  
 
19 March 2020 - Updated benchmarks released
Please note that the NWSC-3 HPC Benchmarks have been updated to include changes to the GOES and OSU MPI benchmarks. Prospective Offerors for the NWSC-3 Request for Proposal (RFP), which will be released 2 April 2020, should download the updated benchmark code, input cases, and instructions (see below). NCAR does not plan to make any additional changes to the HPC Benchmarks unless there are issues with the ones provided here.  
 
19 July 2019 - Benchmarks released
The NWSC-3 HPC Benchmarks are available ahead of an anticipated release of the NWSC-3 Request for Proposal (RFP) in Q1 of 2020. NCAR does not plan to make any changes to the HPC Benchmarks unless there are issues with the ones provided here.  

Questions?
If you have any questions or issues with the benchmarks, please contact Alison Propes, UCAR's Subcontract/Procurement Manager.

NCAR benchmarking applications - 2019/2020 release

Release Date: July 19, 2019
Last Updated: March 19, 2020
Previous Releases

Ensuring that real applications perform well on NCAR computing platforms is critical for getting the best value out of complex and costly high-performance computing and storage resources. Climate and weather applications are large, often with millions of lines of code, and are generally difficult to configure in a way that permits ease of use for things such as system deployments, upgrades, and procurements.

Thus, NCAR has developed a suite of application kernels, micro-benchmarks, and full applications with moderate input cases that can be used as proxies for the full applications and still provide meaningful information and insights into system performance. A few of these are well-known benchmarks that are commonly used in HPC for characterizing system performance.

NCAR's benchmarking applications are listed in the table below, along with file names, sizes, and checksums. These packages include source files, build scripts, and input data sets required to compile and run the applications. In cases where the benchmarks depend on applications and libraries that are not part of the package distributions, you will find version number and download details in the README files.

Documentation and benchmark download packages are available through the Globus-based NCAR Data Sharing Service. Instructions are given below for obtaining a Globus account, installing the required Globus software, and downloading the benchmark packages via the NCAR HPC Benchmarks endpoint. See the Globus instructions below for more information.

Benchmarks

  Name     Description   File Contents   File Name   Size   (Bytes)   MD5 Checksum
CLUBB Physics Kernel Instructions CLUBB_2019-05-19.pdf 63246 006fb83f72d3cc4042c361701f837ea4
Benchmark Files CLUBB_2019-05-19.tar.gz 184931304 626d0c5108f76c662d8e722fa64cfe82
DART_WRF Model Kernel Instructions DART_WRF_2019-05-20.pdf 59749 cbe78c13a80073910c0275467c52ee2c
Benchmark Files DART_WRF_2019-05-20.tar.gz 258607013 a388e961fce0960d096417cd45e1f00f
GOES ML Benchmark Instructions GOES16_2020_03_18.pdf 71352 cc23b36be5e105c2469bcfe573d16ae6
Benchmark Files GOES16_2020_03_18.tar.gz 3578031606 1cf8aaff6eadbb407e0da29649bc6694
MG2 Physics Kernel Instructions MG2_2019-05-20.pdf 61804 c13a288f425993504ab9ce5db692c008
Benchmark Files MG2_2019-05-20.tar.gz 85366943 25ebc145a374d3ccd1d410f9d495261a
MPAS-A* GPU-capable Atmospheric Model Instructions MPAS_2019-06-26.pdf 274813 cf523aa8e3a9d889d11817d0d07edca9
 
Benchmark Files (For access, follow the MPAS-A instructions below)
Input Data MPAS_2019-08-09_data.tar.gz 17406567725 0d0e9ad261178f256812e592f36f380f
Stream Memory Bandwidth Instructions Stream_2019-05-22.pdf 69436 577d9da38eed93d782d6a046c36d7353
Benchmark Files Stream_2019-05-22.tar.gz 20743 23d9d58f8d709553c7e409ab1b1e44cc
WACCM Physics Kernel Instructions WACCM_2019-05-19.pdf 62391 6bb2d8bd1df3471ad93a17540e2c2c17
Benchmark Files WACCM_2019-05-19.tar.gz 18289529 b31c36c58f5ecbbb613caaa39b663b32
WRF Weather Research and Forecasting (WRF) Model Instructions WRF_2019-09-06.pdf 93213 2a47071030e4417d4873938333a60af9
Benchmark Files WRF_2019-09-06.tar.gz 9778202346 bec5bf5cc682b14ebb30a2da51d381ab
OSU MPI MPI Communications Benchmark Instructions osu-micro-benchmarks-5.5_2020-03-12.pdf 62732 4baea167d0698973e751b75e578ac6bb
Benchmark Files osu-micro-benchmarks-5.5_2020-03-12.tar.gz 765369 bcb970d5a1f3424e2c7302ff60611008

Globus instructions

Step 1: Obtain a Globus account

Go to www.globus.org and click the Sign Up button in the upper-right corner.

Step 2: Install Globus Connect Personal

Go to https://www.globus.org/globus-connect-personal and install the version of Globus Connect Personal appropriate for your computer. Versions are available for Mac OS X, Linux, and Windows.

Step 3: Use Globus to download benchmarks

  1. Access the NCAR HPC Benchmarks folder on Globus (you will need to log in to Globus with the account created in Step 1).
  2. Select the files you wish to download and click Transfer or Sync to in the right-hand pane.
  3. Select the endpoint you wish to transfer files to. This can be the computer where you installed Globus Connect Personal in Step 2, or another Globus endpoint to which you have access.
  4. Click on the Start button below the file manager to initiate the transfer.

Your download will be submitted through the Globus Transfer service. You will receive an email when your transfer has completed. You can monitor the transfer by clicking Activity in the left-hand menu to bring up the Globus Activity view.

* Note: Access to the MPAS-A benchmark code is restricted. To obtain access, follow the instructions below.  

Instructions for obtaining NWSC-3 MPAS-A benchmark source code

Code releases for the MPAS-A GPU project will occur through the open-source GitHub site. However, before you may access the site, you are required to sign the MPAS-A Confidentiality Agreement. To obtain access to the MPAS-A GPU GitHub site, send both your signed MPAS-A Confidentiality Agreement and your GitHub account/login to Alison Propes, UCAR's Subcontract/Procurement Manager.

Note that all materials (including source code, products derived from source code, and documents) related to NWSC-3 MPAS should not be distributed, either formally or informally, in any form. Publishing any kind of results obtained from the NWSC-3 MPAS-A source code requires written consent from UCAR.