Uploading / Downloading Files
Using a Secure File Transfer (sftp) client
We recommend using FileZilla to transfer files to and from the cluster. You can get this software from this link. Once you've installed and opened the FileZilla Client, use the following instructions to connect.
In the "Quickconnect" bar at the top of FileZilla, enter the following information:
- Host: sftp://login.cac.queensu.ca
- User: (your username)
- Password: (your password, don't use temporary passwords)
- Port: (leave blank, in case of problems use "22")
- Hit "Quickconnect" to connect
Once connected, you should see your files on the cluster along the right hand side, and the files from your computer on the left. To transfer files between your computer and the cluster, drag-and-drop the files from one side to the other (or to and from your desktop).
Globus provides a means to transfer large amounts of data in a batch framework, i.e. without "standing by" while the transfer is on-going. Since the setup of an individual "endpoint" is required for this, we don't recommend this method if only small amounts of data need to be transferred. However, if you are planning to move large amounts (in the TB range), then Globus is a reliable and convenient method.
If you decide to go this route, follow the following steps.
Installing Globus Command-Line Interface (CLI)
We reccommend to do the following installs in a spearate directory.
$ mkdir globus $ cd globus
The Globus CLI needs to be installed individually by the user. This is very simple using the python "pip" tool:
$ module load python $ pip install --upgrade --user globus-cli Collecting globus-cli [...response from pip installer...]
In addition, the "Globus Connect Personal CLI" needs to be installed too. We're adding the directory it's in to the path.
$ wget https://downloads.globus.org/globus-connect-personal/linux/stable/globusconnectpersonal-latest.tgz [...download response from wget...] 2018-11-12 09:57:00 (24.3 MB/s) - ‘globusconnectpersonal-latest.tgz’ saved [14501379/14501379] $ tar xzf globusconnectpersonal-latest.tgz $ cd globusconnectpersonal-2.3.6/ $ export PATH=`pwd`:$PATH
Login to Globus
Once the CLI is installed it can be used to login to your Globus account. You need a Globus ID which you can make yourself, or (more likely) obtain through Compute Canada. Authentication is done through a browser. Thge globus login command will provide a link to a Globus page, which you cut-and-paste. At the page you will be required to provide your Globus ID and authorize some access. Eventually you will be give an authorization code which you can cu-and-paste back into the login session:
hpc1005@caclogin03$ globus login --no-local-server Please authenticate with Globus here: ------------------------------------ https://auth.globus.org/v2/oauth2/authorize?[...etc...] ------------------------------------ Enter the resulting Authorization Code here: qLdfgbsbhdfugisbsusidfgsdbu You have successfully logged in to the Globus CLI! You can check your primary identity with globus whoami For information on which of your identities are in session use globus session show Logout of the Globus CLI with globus logout
"globus --help" provides a list of available commands that are used from the Globus CLI to initiate transfer sessions etc.