Skip to navigation

Remote backups using ssh and rsync

Here’s a mini tutorial on how to do fast backups of important files to a remote server.

This involves three steps:

  1. Getting ssh access to a remote machine,
  2. setting up keypair authentication,
  3. backing up data with rsync.

Requirements:

  • rsync and ssh server on the remote machine,
  • rsync and ssh client on the local machine.

Step one: ssh access

First, check that you can get ssh access to the remote machine, using your login on the remote machine. If you use the same login on your local and remote machine, you can omit everything up to and including the @ sign:

ssh login@remote.machine.domain

If you get an error message like this:

ssh: connect to host remote.machine.domain port 22: Connection refused

it may mean the remote machine is listening for incoming ssh connections on a different port than the standard 22 port. In that case, find out which port to use (from an administrator of the remote machine), then specify that port using the -p switch, as in the example below, where we specify port 75:

ssh -p 75 login@remote.machine.domain

Step two: keypair authentication

You can optionally skip this step, if you’re willing to type the password to your remote machine every time you back up data. In that case, you also won’t be able to schedule backups using crontab (something I’ll hopefully post about later).

This is described very nicely by Daniel Robbins. For the impatient, in Daniel’s description, scroll down to “ssh-keygen up close.”

Daniel talks about the security implications of setting up the private key without a password, and gives instructions on a workaround using ssh-agent in part 2. It seems to me that for most users who are running Linux on a fairly well-protected (single user) private machine, it is okay and much less hassle to set up the private key without a password.

Step three: rsync

Rsync is a very handy tool because it can figure out which files have changed since the last backup, and then upload only the changes (yes, apparently, not just “only the files that have changed,” but also “only the changed parts of the file(s).” Nifty.

To use rsync correctly, you should first figure out what you want it to do. Here’s my situation: I have access to a computer at the university running Linux. I want to back up directories with important data to that computer from time to time. Currently, I do this manually whenever I’ve made significant changes. However, I also sometimes work on that university computer locally, so when I backup data from my laptop, I don’t want to overwrite a newer version of the file on the university machine.

I do all this using rsync wih the following options:

rsync -auv

The -a option stands for archive mode, which does the following:

  • recurses into directories,
  • copies symlinks as symlinks,
  • preserves owner, groups, permissions, and time.

The -u option skips files that are newer on the receiver, in my case, on the remote machine. Note that this does not protect from human error. If I make important changes to the file on the remote machine, then make different changes to an older version on the local machine, the file on the local machine is the newer version, and rsync -u will still overwrite the version on the remote machine.

The -v option stands for verbose and shows you what’s going on.

While you’re experimenting with rsync, you should definitely first use it with the -n option as well, as this will do a “dry run” without changing anything. Say -v as well, and you’ll see what rsync would do without the -n option.

Now that we have the options we like, let’s say we want to backup a directory called “ex” in the home directory of the local machine to the home directory of the remote machine:

rsync -auv /home/mpromber/ex remote.machine.com:

Note that there is no trailing slash after “ex”. If you write “ex/”, this will copy over the contents of the directory, so all the files would be dumped directly into the home directory on the remote machine. You could explicitly state that you want the files to go into the folder “ex” on the remote machine, so it would be equivalent to the line above to write:

rsync -auv /home/mpromber/ex/ remote.machine.com:ex

As a last step, we have to tell rsync to use an ssh connection for the transfer:

rsync -e "ssh" -auv /home/mpromber/ex remote.machine.com:

You can pass options to ssh inside the quotation marks. So if you have a nonstandard port, e.g. 75:

rsync -e "ssh -p 75" -auv /home/mpromber/ex remote.machine.com:

Optional: create a function in ~/.bashrc

The following assumes you are using bash as a shell:

echo $SHELL

will tell you the shell you are using.
Put the following lines in your ~/.bashrc, of course substituting your own username, remote host, and nostandard ssh port if needed:

mybackup () {
    rsync -e "ssh -p 75" -auv /home/username/$1 remote.machine.com:
}

This is a so-called function. I think it has to be preceded and followed by a blank line in the .bashrc, and anyway, that probably doesn’t hurt.

Tell your terminal about the changes:

source ~/.bashrc

Now, you can just type:

mybackup ex

to back up the directory ex to the remote machine. Note that when you are in your home directory and you use tab completion to finish the directory name after “mybackup,” this will usually add a trailing slash. You’ll want to remove this if you alias is set up like mine, to move over the directory and not just its contents.

Addendum

Corrections applied, see comment #1.

3 Responses to “Remote backups using ssh and rsync”

  1. zuth Says:

    Thank you for the useful info.

    In the step three, your command example:

    ssh -auv /home/mpromber/ex remote.machine.com:
    ssh -e “ssh” -auv /home/mpromber/ex remote.machine.com:
    ssh -e “ssh -p 75” -auv /home/mpromber/ex remote.machine.com:

    shouldn’t it be

    rsync -auv /home/mpromber/ex remote.machine.com:
    rsync -e “ssh” -auv /home/mpromber/ex remote.machine.com:
    rsync -e “ssh -p 75” -auv /home/mpromber/ex remote.machine.com:

  2. Marianne Says:

    Thanks, zuth, you are of course completely right. I’ll correct this in the original.

  3. depert Says:

    well, marianne and zuth thank you for your useful info. And it works in my network.