Getting started with HPSS

Access | Project codes | Archiving and managing files | Permissions

1/20/2020 - HPSS is now read-only.
11/14/2019 - HPSS will be put into read-only mode on January 20, 2020.
10/15/2019 - Users advised to move or delete all HPSS data now
 
NCAR’s High-Performance Storage System (HPSS) will reach its end of life on October 1, 2021. Users are advised to begin moving their data to an alternative storage system and deleting it from HPSS immediately.
 

 

Access

Principal investigators and project leads for HPSS storage allocations have user accounts that give them access to the system. They can request user accounts for other individuals who need access to HPSS files by contacting CISL.

HPC users

To use HPSS on NCAR systems inside the high-performance computing (HPC) environment that CISL manages, authorized users – those who have HPSS accounts – log in through the Cheyenne or Casper login nodes. No additional authentication is required at that point.

Other users

To use HPSS on NCAR systems that are outside of the HPC environment, see Kerberos and HSI.


Project codes

To use HPSS, you must have an HPSS account and project code to which your HPSS usage can be charged. Your HPSS project code may be the same as the one associated with your allocation for use of computing resources.

While you might have more than one project code, one is designated as your HPSS default project code. Unless you specify otherwise, files written to HPSS use that project code.

See Projects and charges for information on changing project codes.


Archiving and managing files

HPSS is shared by a large number of users, so efficiency and best practices are critical. The number of file actions that users can execute concurrently is subject to individual and global limits. See Use and storage policies for details regarding those limits.

Also see these pages for how to manage and transfer files:

Warning: Transferring numerous files warning

Use HTAR if you need to archive large numbers of files. Do not use HSI to move hundreds or thousands of individual files to HPSS.

See File size guidelines for HPSS for additional information that will help you use this system most efficiently.

Failure to follow these guidelines can result in suspension of your access to HPSS.


Permissions

Properly setting up directory and file permissions can help you prevent unintentional overwriting or deletion of the data that you store in HPSS.

As explained in our HPSS permissions documentation, it is how you set permissions on the parent directory that protects your HPSS files, not permissions on the files themselves. This is important to keep in mind because of the way some HSI commands are implemented.

While directory permissions are important for protecting data, file permissions serve other purposes. We recommend that you always set the permissions on files directly after writing them.