Campaign Storage file system

Data retention policy | Allocations | File transfers | Reports

The NCAR Campaign Storage file system is available to the NCAR labs and to universities that have project allocations. It is for the storage of data on publication timescales.

This disk-based file system’s initial storage capacity is 20 PB with plans to increase the capacity each year. It is the preferred alternative to the older, tape-based High Performance Storage System (HPSS).


Data retention policy

Campaign Storage has a five-year retention policy, meaning files will be purged five years after they are copied to the file system. NCAR lab users are expected to collaborate with CISL’s Digital Asset Services Hub (DASH) to develop data migration plans for storage needs that exceed five years.


Allocations

Each NCAR lab has an allocation of Campaign Storage space and the labs manage how those allocations are used. University users can request Campaign Storage space through the NCAR Resource Allocation System as supplements to their project allocations. Requests must include detailed justification for the amount of space requested.


File transfers and management

Globus transfers screen

Users transfer and manage their Campaign Storage files by:

  • using Globus as described here (with either a web or command line interface).
  • logging in to the CISL data-access nodes and using POSIX commands.

Globus

The Globus endpoint established for the file space is NCAR Campaign Storage.

How to make transfers using the command line interface is covered in detail here:

Globus CLI tutorial

NOTE: The Globus interface for transferring data does not handle symbolic links and does not create symbolic links on a destination endpoint.

POSIX

The Campaign Storage file system is mounted on the data-access nodes as /glade/campaign to:

  • enable users to manage file and directory permissions.
  • facilitate transfers of small files to and from GLADE spaces such as /glade/scratch and /glade/work.

Reports

The Systems Accounting Manager (SAM) provides overall summary information about the use of Campaign Storage allocations and other allocations.

CISL plans to develop additional tools for use in allocation management.

 

Related training courses