- Data Portals
- User Support
- About Us
As announced in the CISL Daily Bulletin, the /glade/scratch file space for the Yellowstone, Geyser, and Caldera clusters has been integrated with the new, larger GLADE scratch system.
The old scratch space remains available in read-only mode as /glade/scratch_old until June 1. Yellowstone users who have not already done so should now move any scratch files that they want to keep from /glade/scratch_old to /glade/scratch.
The /glade/scratch_old space will be removed from all systems on June 1.
The Globally Accessible Data Environment—a centralized file service known as GLADE—uses high-performance GPFS shared file system technology to give users a common view of their data across the HPC, analysis, and visualization resources that CISL manages.
GLADE file spaces are intended as work areas for day-to-day tasks and are well suited for managing software projects, scripts, code, and data sets. They are available by default except for project spaces.
|25 GB*||Yes||Not purged||User home directory|
|10 TB||No||See below||Temporary
|512 GB||No||Not purged||User work space|
|N/A||No||Not purged||Project space allocations
(via allocation request)
|GLADE system status report|
Data can remain in each of these spaces in accordance with the policies detailed below. The policies are subject to change; any changes necessary will be announced in advance.
CISL backs up the GLADE home file space several times a week and also creates snapshots to enable users to recover deleted files quickly and easily.
CISL does not provide backups of scratch, work, and project spaces. You are responsible for the safe storage of any data that must be preserved.
You can conserve GLADE space by storing large files, such as tar files, rather than numerous small, individual files. This is because the system allocates a minimum amount of space for each file, no matter how small.
The /glade/u/home space has a block size of 1 MB and a minimum sub-block allocation of 32 KB.
In the scratch, work, and project spaces, the block size is 4 MB and the smallest amount of space the system can allocate to a file, including directories and symlinks, is 128 KB (the "sub-block" size). Any files smaller than 128 KB are still allocated 128 KB, so they require more space than you might expect.
Each user has a 25-GB* /glade/u/home/username space that is ideal for managing scripts, source code, and small data sets.
CISL creates snapshots of this space to enable users to recover deleted files quickly and easily.
Each user has a /glade/scratch/username space by default, with an individual quota of 10 TB. The scratch file space is intended to support output from large-scale capability runs as well as computing and analysis workflows across CISL-managed resources. It is a temporary space for data that will be analyzed and removed within a short amount of time.
If you will need to occupy more than your quota of scratch space at some point, request a temporary increase by emailing email@example.com. Include a paragraph justifying your need for additional space.
Individual files are removed from /glade/scratch/username automatically if they have not been accessed (for example: modified, read, or copied) in more than 45 days. A file's access time (atime) is updated at most once per day for purposes of I/O efficiency. To check a file's atime, run ls -ul filename.
Moving files to your scratch directory does not change their access times, so files with older access times are prone to being purged unexpectedly. If you need to move old files to your scratch directory, contact the CISL Help Desk for assistance in adjusting the access times appropriately. Also, be careful when untarring files so they do not show older access times and get scrubbed prematurely.
Users may not run “touch” commands or similar commands for the purpose of altering their files' timestamps to circumvent this purge policy. CISL staff will reduce the scratch quotas of users who violate this policy; running jobs may be killed as a result.
Best practice: To help us avoid the need to shorten the retention period, please use this space conscientiously.
Delete files that you no longer need as soon as you're done with them rather than leave large amounts of data sitting untouched for the full 45 days. If you need to retain data on disk for more than 45 days, consider using your /glade/p/work space or requesting an allocated project space.
Your /glade/p/work/username space is best suited for actively working with data sets over time periods greater than what is permitted in the scratch space.
The default quota for these spaces is 512 GB. If space is available, you may request a temporary increase of up to 1 TB.
Dedicated project spaces are available through our allocations process to support longer-term disk needs that are not easily accommodated by the scratch or work spaces. Allocations for project spaces are made to collaborative groups of users through the University/CHAP, CSL, or NCAR allocations processes. The allocations are based on project needs and resource availability. Requests are reviewed according to the various allocation schedules.
If you have an allocation, you can request a project space via the online request form. Justifications for project spaces are scrutinized closely. Carefully document which parts of your data and data workflow cannot be accommodated within the home, scratch, or work file spaces and policies.
If you have a user account and project space but lack the directory permissions you need for that space, email your request for changes to firstname.lastname@example.org. Identify the directories and the permissions you are requesting.
Project spaces and others are not intended as permanent storage, and the limited disk is a community resource. These spaces should be used as efficiently as possible so that other projects can take advantage of the storage resource.
All files that you own are counted against your GLADE quota, regardless of the directory in which they are stored. If you write files to another user's home or scratch space, for example, they still count against your own individual user quota for that space.
If you reach your disk quotas for the GLADE file spaces, you may encounter problems until you remove files to make more space available. For example, you may not be able to log in, the system may appear hung, you may not be able to access some of your directories or files, your batch jobs may fail, and commands may not work as expected.
If you cannot log in or execute commands, contact email@example.com. You can check your space usage as shown below.
This command will generate a report showing your quota and usage information:
The ncarbinlibs module, which is loaded by default when you log in, must remain loaded for this to work.
* The gladequota output will show the home space quota as 50 GB rather than 25 GB because the system stores dual copies of users' data for increased data integrity and safety.