Globus file transfers

Web interface transfers | CLI transfers | Globus Connect Personal

CISL recommends Globus as the most efficient way to transfer files between NCAR file systems (e.g., GLADE) and non-NCAR resources, including XSEDE facilities and other sites. It has both a web interface and a command line interface (CLI).

GLADE exception: CISL recommends using the rsync or cp commands to move data between GLADE spaces. The Globus interface for transferring data does not handle symbolic links and does not create a symbolic link on a destination endpoint. Example batch scripts.

To use either interface, the first step is to create a personal Globus ID account. (UCAR and NCAR staff should not use the NCAR RDA organizational login.)

Several public endpoints are set up on the Globus system for transferring files to and from the GLADE disk storage system.

They include:

  • NCAR GLADE (legacy name: ncar#gridftp)
  • NCAR Campaign Storage
  • NCAR Data Sharing Service (legacy name: ncar#datashare)
  • XSEDE NCAR GLADE (legacy name: xsede#ncar) for users who have XSEDE credentials.

Globus also offers a feature called Globus Connect Personal for moving files to and from a laptop or desktop computer and other "endpoints."


Transferring files with the web interface

When transferring files between systems, keep in mind that your username might not be the same on each system.

Follow these steps to transfer files.

  1. Go to the main Globus page (globus.org) and use your personal Globus ID to log in. 
  2. Go to File Transfer.
  3. Enter your source endpoint on one side of the panel (see screen image below).
  4. Specify the path where your source files are located and click Go.
  5. Enter your username and token response or password when you are asked to authenticate.
  6. Identify your target endpoint in the other panel.
  7. Specify a destination path.
  8. Select the files you want to copy.
  9. Click the arrow button to initiate the transfer.

You can check the status of your transfers any time through the web interface and will be notified when they are complete.

Globus

Globus provides access to users' GLADE file spaces and the system's data-access nodes. Include your username in the path as shown. Changing to the parent directories /glade/scratch or /glade/u/home can result in long delays and timeouts because of the numerous child directories in each of those spaces.

Video demonstration


Transferring files with the command line interface

The Globus CLI application is installed on the NCAR/CISL data-access nodes. It can be used to make both manual and unattended file transfers.

To begin, log in as shown. (If your UCAR username and your username on your local computer are different, follow the alternative example.)

ssh data-access.ucar.edu (alternative: ssh username@data-access.ucar.edu)

Run globus login and follow the on-screen instructions. 

globus login

Output example:

Please log into Globus here:

------------------------------------

URL to copy and paste into your browser.

------------------------------------

Enter the resulting Authorization Code here:

Copy the lengthy URL and paste it into your browser. It will take you to a consent form that looks like this:

Click Allow to give the CLI app the necessary access and you will receive an authorization code.

After entering the code, you will be logged in to the Globus CLI and in your /glade/u/home directory.

Video demonstration

Using the command line interface

See these Globus resources for how to make batch transfers and single item transfers, how to manage endpoints, and additional information.

Your default shell on the data-access nodes is tcsh. To change your current shell, just enter bash or another preferred shell.


Globus Connect Personal

To set up your laptop or desktop computer to use Globus Connect Personal:

  • Go to Globus Connect Personal and follow the instructions to download and install it on your local system.
  • Add your local system as an endpoint by following the instructions from the Globus Connect website.
  • Start Globus Connect, and then sign in to globus.org.

Your local system should now appear as an endpoint that can be used for transferring files.

Related training courses