Script for moving data between GLADE spaces

CISL recommends using rsync -av (or cp -rp) rather than Globus for moving data between GLADE spaces. Neither rsync nor cp alone preserves permissions, ownership and so on, but they will with the options shown. (Globus does not preserve the symbolic links that are common in working directories, and it does not create symbolic links on destination endpoints.)

These examples show how to use rsync -av in a batch script, in this case for transferring files from /glade/scratch_old to /glade/scratch.

For tcsh users

Insert your own project code where indicated and customize other settings as needed for your own job.

#!/bin/tcsh
#SBATCH -J rsync_scratch
#SBATCH -n 1
#SBATCH --ntasks-per-node=1
#SBATCH -t 24:00:00
#SBATCH -A project_code
#SBATCH -p dav
#SBATCH -o rsync.%j
#SBATCH -C caldera|pronghorn

setenv TMPDIR /glade/scratch/$USER/temp
mkdir -p $TMPDIR

### Run rsync -av
cd /glade/scratch_old/$USER
rsync -av . /glade/scratch/$USER

### End job

For bash users

Insert your own project code where indicated and customize other settings as needed for your own job.

#!/bin/bash -l
#SBATCH -J rsync_scratch
#SBATCH -n 1
#SBATCH --ntasks-per-node=1
#SBATCH -t 24:00:00
#SBATCH -A project_code
#SBATCH -p dav
#SBATCH -o rsync.%j
#SBATCH -C caldera|pronghorn

export TMPDIR=/glade/scratch/$USER/temp
mkdir -p $TMPDIR

### Run rsync -av
cd /glade/scratch_old/$USER
rsync -av . /glade/scratch/$USER

### End job