How do I transfer large files using rsync?

FTP uploads on our system have a maximum size of 5GB.  If you need to upload files larger than this, you'd need to upload these via SSH.

If you are using Windows, we would recommend uploading these using WinSCP.

If you are using a Mac OS, Linux or another Unix-based system you will be able to make use of rsync to move these files over. Firstly you will need to make sure rsync is installed and is ready to be used, rsync is pre-installed on the Mac OS X system and is pre-installed on most Linux Distributions, however, you can verify this yourself by running: 

$ rsync --version #

If this command fails to run, then it likely is not installed, and you can install it by running a command such as (depending on your system):

$ sudo apt-get install rsync (For Debian and Ubuntu)

Or

$ sudo yum install rsync (For CentOS, Fedora, Rocky and AlmaLinux)

How to use rsync to upload files to a web hosting package

Generating an SSH Key Pair to use

Firstly, you will want to generate some SSH keys to be able to access your web hosting package. You can do this by running:

$ ssh-keygen

After this command is run, you should see something similar to: 

Generating public/private rsa key pair.
Enter file in which to save the key (/yourHomePath/.ssh/id_rsa):

We recommend you just press Enter here and it will write your SSH key pair into the /.ssh/ folder in your home directory. Alternatively, you can provide it with a custom directory and key name that it will use to create your key pair.

It then may ask you to overwrite the previous keys there. If you choose to do so your previous keys will be lost.

The last step before your key is generated is to set a passphrase for the private key, however, this is optional.

Enter passphrase (empty for no passphrase):

Your keys should now be generated, the public key will need to be uploaded into the SSH Access section of the package you are looking to move your files into.

You can verify that this is set up correctly by running the following which should connect you to your package via SSH:

$ ssh -i /path/to/privatekey domain.com@ssh.gb.stackcp.com
  ...
-bash-4.2$

You can find a comprehensive guide to accessing your package using SSH here.

Using rsync to move files from your local machine to a package on 20i

Once you have SSH Access set, you can then look to begin moving your large backup files using rsync. This will be a case of simply running the rsync command and giving it a file to move and a directory to move it into.

rsync [options] [file to move] domain.com@ssh.gb.stackcp.com:[directory of package to move to]

More specifically, to move your file to your hosting package at 20i, the rsync command will look something like this:

$ rsync -v path/to/backup.zip domain.com@ssh.gb.stackcp.com:/path/to/you/packages/home/directory/[optional path in the package]

To find your package's /path/to/you/packages/home/directory/ it will be a case of going to the Manage Hosting page, finding the package you are looking to rsync to and going to Options > Manage and then looking for the Account Information section. Under this section you should see information such as the domain, package type and data centre location, however, we are specifically looking for the Home Path. 

In the rsync command, you can set a further optional path to somewhere in your package. For example, this could be something such as /path/to/you/packages/home/directory/public_html/new-site/, however, if you do not provide this the file you are moving will be written into your package's root directory.

The -v option sets the output to verbose, which will tell you which file is being moved, and will provide confirmation once it has been successfully moved.

Please do note, that if you are moving files to a package on our US data centre you will need to use ssh.us.stackcp.com instead of ssh.gb.stackcp.com.

When creating your SSH keys, if you set a custom directory or name for your key pair, then it may be the case you need to specify which private key to use in your rsync command. To do so we will need to make use of the -e option which allows you to specify the remove shell to be used whilst you are moving your files. This will look something like:

$ rsync -ve "ssh -i /path/to/key" path/to/backup.zip domain.com@ssh.gb.stackcp.com:/home/sites/path/to/package/[optional file path]

When you run either of the above commands it will connect to your package via SSH and synchronise the file from your local machine to your package at 20i. If you are using the -v option, you will know once the file has been successfully moved as the following will be outputted:

Backup.zip

sent 467,930,820 bytes  received 35 bytes  25,293,559.73 bytes/sec
total size is 467,816,526  speedup is 1.00

Once the rsync has finished running, you can verify the file has been uploaded successfully by going to the File Manager of your package where you should be able to locate the file you transferred using rsync.