Globus file transfers

Web interface transfers | CLI transfers | Globus Connect Personal

Updated 5/17/2019

Globus is the most efficient way to transfer files between NCAR file systems such as GLADE and non-NCAR resources, including XSEDE facilities and other sites. Transferring files between GLADE and the separate NCAR Campaign Storage file system requires the use of Globus.

Globus has both a web interface and a command line interface (CLI). To use either interface, the first step is to create a personal Globus ID account. (UCAR and NCAR staff should not use the NCAR RDA organizational login.)

Several public endpoints are set up on the Globus system for transferring files to and from the GLADE disk storage system. They include:

  • NCAR GLADE
  • NCAR Campaign Storage
  • NCAR Data Sharing Service
  • XSEDE NCAR GLADE for users who have XSEDE credentials.

Globus also offers a feature called Globus Connect Personal for moving files to and from a laptop or desktop computer and other endpoints.

Cautions

  • It is possible to corrupt data when performing a transfer if you accidentally specify the same files for the source and destination. To avoid inadvertently deleting data when using the web interface, activate the "sync" option in the Transfer & Sync Options menu. Sync will allow a file transfer only if the file does not exist on the destination or if the source copy is newer than the destination copy.
  • The Globus interface for transferring data does not handle symbolic links and does not create symbolic links on a destination endpoint.
  • Transferred files assume the user's umask permissions on the destination system, regardless of permissions on the source system.

Transferring files with the web interface

When transferring files between systems, keep in mind that your username might not be the same on each system.

Follow these steps to transfer files. See the image below for reference.

  1. Go to the main Globus page (globus.org) and log in using your personal Globus ID
  2. Go to File Manager.
  3. Use the Panels button to display two endpoint panels side by side.
  4. Enter the name of your source endpoint in the Collection field on one panel.
  5. Specify the path where your source files are located.
  6. Enter your username and token response or password when you are asked to authenticate.
    The default credential lifetime is 24 hours. Here's how to extend it.
  7. Identify your target endpoint in the other panel.
  8. Specify a destination path.
  9. Select the files you want to copy.
  10. Click the Start button to initiate the transfer.

You can check the status of your transfers any time through the web interface and will be notified when they are complete.

Globus


Transferring files with the command line interface

The Globus CLI application can be used to make both manual and unattended file transfers.

The application is:

To begin, log in as shown in this example for using the data-access nodes. (If your UCAR username and your username on your local computer are different, follow the alternative example.)

ssh data-access.ucar.edu (alternative: ssh username@data-access.ucar.edu)

Run globus login and follow the on-screen instructions.

globus login

Output example (if you are not already logged in):

Please authenticate with Globus here:

------------------------------------

URL to copy and paste into your browser.

------------------------------------

Enter the resulting Authorization Code here:

Copy the lengthy URL and paste it into your browser. It will take you to a consent form that looks like this:

Globus CLI access permissions

Click Allow to give the CLI app the necessary access and you will receive an authorization code.

After entering the code, you will be logged in to the Globus CLI and in your /glade/u/home directory.

The default credential lifetime is 24 hours. Here's how to extend it.

Your default shell on the data-access nodes is tcsh. To change your current shell, just enter bash or another preferred shell.

Executing CLI transfers

For details regarding how to make batch transfers and single-item transfers, manage endpoints, and more, see these resources:


Globus Connect Personal

To set up your laptop or desktop computer to use Globus Connect Personal:

  • Go to Globus Connect Personal and follow the instructions to download and install it on your local system.
  • Add your local system as an endpoint by following the instructions from the Globus Connect website.
  • Start Globus Connect, and then sign in to globus.org.

Your local system should now appear as an endpoint that can be used for transferring files.

Related training courses