Ubuntu Rsync Tutorial (ubuntu)

Rsync is a great lightweight backup utility for Linux and windows, its selling point over other backup solutions is that it only copies the diffs of the files that have changed, therefore only transmitting the bare minimum of data. This makes it great for running backups into the cloud & to over internet services.

Rsync has two modes, daemon & client. The Rsync server runs in daemon mode, this listens for requests on port 873 and can serve these requests in Rsync’s own protocol or via remote shell (RSH or SSH), thus making it a very secure way to backup your files. The Rsync client is pretty simple and normally runs as a simple executable. What makes Rsync more remarkable is its huge array of features, to name a few:

  • Only Transferring DiffsOnly changed pieces of files are transferred, instead of the whole file. This makes the upload much faster.
  • Compression – These tiny pieces of diffs are then compressed further increasing transfer speeds.
  • Secure Shell – The connection from Rsync is passed through the ssh protocol to encrypt your session, this is a good an option to use as i doubt you will want anyone reading the files you are backing up.
  • Support for copying links, devices, owners, groups and permissions
  • Exclude and exclude-from options similar to GNU tar
  • A CVS exclude mode for ignoring the same files that CVS would ignore
  • Support for anonymous or authenticated Rsync servers (ideal for mirroring)
  • Pipelining of file transfers to minimize latency
  • Does not require root privileges thanks to its daemon mode
  • Mirroring or Backup for multiple clients to a central server
  • Automated encrypted rsync-based mirroring between multiple hosts and a central server
  • any many more great features that you may or may nether use, hay its always good to know they are there.

    So how does this great piece of software work, well first you have to setup a central computer that will run Rsync in daemon mode, we’ll call this the “Rsync Server“, in Linux this server will be configured using a config file (usually /etc/rsyncd.conf). Now several computers, with the Rsync client installed, can synchronize to and from the Rsync server. These clients can be configured to run backups, distribute files, mirror file systems, and much more.

    I’m sure by now you want to know how to actually use Rsync, well lets start with a simple local backup, type this into your terminal:

    sudo rsync -azvv ~/ /backup

    This will copy your entire home directory into the new folder /backup, if you run this command again you’ll notice that it will run much faster, this is because there is no new data to copy as it should all be up to date. This is good for local backups but Rsync is even better when you run it over a network as it gives you the ability to have a centralised backup server. Before we start with the backup server we’ll run the same command as before but run it over a ssh connection the same computer, try (substitute [USERNAME] for your local user):

    sudo rsync -azvv -e "ssh" /home/[USERNAME]/ [USERNAME]@[USERNAME]/backup

    An explanation of above options in the command:

  • -a preserves the date and times, and permissions of the files (same as -rlptgoD).
  • -z compresses the data
  • -vv increases the verbosity of the reporting process
  • -e specifies remote shell to use
  • /folder1 and folder2 In the examples above, folder1 and 2 are placeholders for the directories to be synchronized. /home/[USERNAME]/ is the original folder, and
    /home/[USERNAME]/backup is the new folder, or existing one to be brought in sync with the first. Replace them with the folders you’d like. A / was added after /home/[USERNAME] so that only the contents, rather than whole folder, would be moved into the second.
  • If you use a key file to login visa ssh substitute “ssh” for “ssh -i keyfile.pem”
  • Now all you need to do to setup a centralised backup server is have a running Linux box and run this command on your client machines substituting the [USERNAME] & [ADDR]:

    sudo rsync -azvv -e "ssh" /home/[USERNAME]/ [USERNAME]@[ADDR]:/home/[USERNAME]/backup

    If you don’t want to use ssh (not sure why you wouldn’t) you can setup a Rsync server to use Rsync’s own protocol, a good tutorial for this can be found here.

    Please leave any comments or feedback.

    , , , , , , , , , ,

    Comments are closed.