Data transfers and sharing

Transferring files | Sharing data with colleagues

Transferring files between systems

Globus

CISL recommends using Globus to transfer files between systems—for example, between XSEDE facilities and the resources that CISL manages. Globus has both web and command line interfaces, and its Globus Connect feature enables you to move files to and from laptop or desktop computers and other systems.

Globus has a typical transfer rate that ranges from 100 to 200 MBps. As a fire-and-forget utility, it enables users to start a transfer job and walk away. If a transfer fails, the system tries again and provides helpful feedback.

More information.

SCP and SFTP

The installed Secure Copy Protocol (SCP) and Secure FTP (SFTP) utilities are best suited to transferring small numbers of small files to or from a local computer and the resources that CISL manages.

More information.

These other tools also can be used to make secure transfers between your local computer and CISL systems:

  • FileZilla—SFTP for Windows, Linux, and Mac OS X users
  • PSCP / PSFTP—PuTTY utilities including a Windows version
  • WinSCP—SCP and SFTP transfers with graphical user interface for Windows users

Also see the SCP and SFTP man pages on the computer from which you will initiate transfers.

BBCP

BBCP is a multi-streaming utility for transferring large files. It splits files into multiple streams that are transferred simultaneously, so it is faster than the single-streaming SCP and SFTP utilities.

More information.

HPSS

To transfer files to and from our tape-based High Performance Storage System, use the HTAR and HSI commands that are described in detail in our HPSS documentation. As noted there, transferring data between HPSS and computers that are outside the UCAR security perimeter requires two steps: Using HTAR or HSI to move files between HPSS and CISL's GLADE file spaces; then using another method such as Globus for transfers between GLADE and outside systems.


Sharing data with colleagues

NCAR Data Sharing Service

This service enables users of NCAR's GLADE storage and HPC systems for scientific computing to share files with colleagues and others who do not have NCAR user accounts.

More information.

Sharing via Google Drive and other services

Individuals who have GLADE access can copy files from GLADE and then share them with others via Google Drive or services such as DropBox and Amazon Web Services.

More information.

UCAR FTP server

CISL recommends using the NCAR Data Sharing Service for sharing files, but the FTP server is available to authorized UCAR and NCAR personnel who need it.

More information.

Related training courses