Campaign Storage file system

Data retention policy | Allocations | File transfers and management | Reports

The NCAR Campaign Storage file system is available to the NCAR labs and to universities that have project allocations. It is for the storage of data on publication timescales.

This disk-based file system’s initial storage capacity is 20 PB with plans to increase the capacity each year. It is the preferred alternative to the older, tape-based High Performance Storage System (HPSS).


Data retention policy

Campaign Storage has a five-year retention policy, meaning files will be purged five years after they are copied to the file system. NCAR lab users are expected to collaborate with CISL’s Digital Asset Services Hub (DASH) to develop data migration plans for storage needs that exceed five years.


Allocations

Each NCAR lab has an allocation of Campaign Storage space and the labs manage how those allocations are used. University users can request Campaign Storage space through the NCAR Resource Allocation System as supplements to their project allocations. Requests must include detailed justification for the amount of space requested.


File transfers and management

Globus transfers screen

Users transfer and manage their Campaign Storage files by:

  • using Globus with either a web or command line interface.
  • using POSIX commands after logging in to the CISL data-access nodes.

Globus transfers

The Globus endpoint established for the file system is NCAR Campaign Storage. How to make transfers to and from that endpoint is documented here:

Globus file transfers 

How to make transfers using the command line interface also is covered in detail in this tutorial:

Globus CLI tutorial

NOTE: The Globus interface for transferring data does not handle symbolic links and does not create symbolic links on a destination endpoint.

POSIX and data-access nodes

The Campaign Storage file system is mounted on the data-access nodes as /glade/campaign to:

  • enable users to manage file and directory permissions using POSIX commands.
  • facilitate transfers of small files to and from GLADE spaces such as /glade/scratch and /glade/work.

Reports

The Systems Accounting Manager (SAM) provides overall summary information about the use of Campaign Storage allocations and other allocations.

CISL plans to develop additional tools for use in allocation management.

 

Related training courses