Syncing data between two Ubuntu servers using rsync : Simplysimple.info

Syncing data between two Ubuntu servers using rsync

By | Posted, August 27th, 2010 and modified on September 27th, 2011.

Back to HIS Installation Home

This is the continuation of the post ‘Setting up automatic backups for MySQL‘ Keeping in mind the previous documentation, the IP address of the main server is 192.168.0.222 (This is the main/live server also called the HMS server) and that of the backup server is 192.168.0.111 (This is the backup server or HMS backup server). Both the machines have a user called ad who has permissions to  sudo.Lets begin!

On the HMS server,  login as user ad and create the required directories in your home folder

mkdir  -p ~/.ssh/authorized_keys

chmod -R 755 ~/.ssh

We also need to change permissions for the backup folder we created in the previous post so that rsync can access it.

sudo chmod -R 755 /backup

On the HMS backup server

ssh-keygen -t dsa (accept the default location for saving the key and  do not enter a password when prompted, press enter twice)

The output should look something like this…

Generating public/private dsa key pair.
Enter file in which to save the key (/home/ad/.ssh/id_dsa):
Created directory ‘/home/ad/.ssh’.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/ad/.ssh/id_dsa.
Your public key has been saved in /home/ad/.ssh/id_dsa.pub.
The key fingerprint is:
6c:65:71:71:97:07:ef:1b:8d:bc:84:c5:d4:3c:2d:89 ad@hms-backup
The key’s randomart image is:
+–[ DSA 1024]—-+
|          . oo+=+|
|           oE+o==|
|          o   o.+|
|       . o   + o.|
|        S   . +.o|
|       .     . .o|
|              .. |
|                 |
|                 |
+—————–+

We now need to copy this key to the HMS Server…

scp ~/.ssh/id_dsa.pub ad@192.168.0.222:~/.ssh/authorized_keys

The output should look something like this.

The authenticity of host ’192.168.0.222 (192.168.0.222)’ can’t be established.
RSA key fingerprint is e1:83:a3:94:ac:eb:ae:59:cf:10:3c:fd:82:28:83:97.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ’192.168.0.222′ (RSA) to the list of known hosts.

Answer yes if you are prompted to continue connecting as shown above. Then enter the password for the “remote machine”  (HMS Server) password

Test from the backup server by typing ssh ad@192.168.0.222 it should now log you in without prompting for the password, if not, check whether you followed the above instructions correctly and redo them.

Type exit to return back to the server shell prompt. You should see Connection to 192.168.0.222 closed on the screen and the prompt change back from ad@hms-server:~$ to ad@hms-backup:~$

We will now perform a dry run to test our settings.

rsync -e ssh -avzn –delete-after ad@192.168.0.222:/backup ~

If it is successful, i.e. no errors displayed, then proceed to the next step to run the sync manually…

rsync -e ssh -avz –delete-after  ad@192.168.0.222:/backup ~

You should see some output such as this… this output could be very long depending on how many backups exist on your  server.

receiving file list … done
backup/
backup/db/
backup/db/daily/
backup/db/daily/hms_eha/
backup/db/daily/hms_eha/hms_eha_2010-08-26_12h40m.Thursday.sql.gz
backup/db/daily/hms_eha/hms_eha_2010-08-26_12h50m.Thursday.sql.gz
backup/db/daily/hms_eha/hms_eha_2010-08-26_13h00m.Thursday.sql.gz
backup/db/monthly/
backup/db/weekly/
backup/db/weekly/hms_eha/

sent 89 bytes  received 6331270 bytes  383718.73 bytes/sec
total size is 6328752  speedup is 1.00

If you check the /home/ad/backup directory on the backup server, you should be able to see the created backups

Now we are ready to  automate this backup using cron!

crontab -e

If you are prompted to choose an editor, choose nano.

Add the following line to the file and press Ctrl + X and follow the instructions to save and exit

*/60 * * * * rsync -e ssh -az –delete-after  ad@192.168.0.222:/backup~ # (Note: runs only on minutes divisible by 10: 10, 20, 30, etc.)

Check in some time whether the backups on the server are being synced to the backup server. If it is, then your backup server is now operational. You now have two backups, one on the main server, and another on the remote backup server. If it did not work, check whether you followed the instructions on this page correctly and also on the previous post Setting up automatic backups for MySQL

For those who cannot afford a backup server or who use Ubuntu Desktop and want to have an additional layer of security by the way of an additional copy of the data, do the following.

In your home folder, or any other suitable location, right click and Create Document > Empty file and call it hms-backup. Copy and paste the following lines of text into the file and save it.

#! /bin/bash

rsync -e ssh -avzn –delete-after  ad@192.168.0.222:/backup ~

Next right click the file, Properties > Permissions and tick the check box Allow executing file as program.

We are now ready to automate this  backup using cron

crontab -e

If you are prompted to choose an editor, choose nano.

Add the following lines to the file and press Ctrl + X and follow the instructions to save and exit

*/60 * * * * ~/hms-backup

This will create an updated backup folder in your home folder (on your Ubuntu Desktop) every hour.

In case for any reason you cannot get the rsync to take place without prompting for a password (It happened to us after JKT mauled our server for the umpteenth time ) then another workaround is to add the created shell script (The file you created ) to startup applications. This will however take a backup only whenever you log in to Ubuntu, so more or less you get about a backup a  day. This also means that there is no reason why anyone should stay without a proper backup.

 

Related Posts

  1. Syncing your data on multiple machines
  2. Ubuntu Server 10.04 LTS Installation
  3. Webhosting and your MYSQL data
  4. Setting up automated backups for MySQL on Linux
  5. Moving your WordPress blog
  6. Backing up your data
  7. Recovering from a broken array : Ubuntu Software RAID
  8. HIS/HMS Installation Docs
  9. Using Tomcat6 on Ubuntu Server
  10. Accelerated browsing through Squid on Ubuntu
4 Comments so far
  1. Suddarsi October 15, 2010 11:52 AM

    it is very useful ….

  2. Benjy October 15, 2010 12:04 PM

    Thanks Buddy!

  3. James Sebastian March 10, 2011 7:09 PM

    Hi Benjy, Great blog.
    Also trying to re-establish the contacts. James who was part of DBF-South.

    James

  4. Benjy March 10, 2011 8:40 PM

    Hey James! Thanks. Was nice catching up over the phone. Your blog is great too! Being a software guy you will have a better understanding of the internals than me :-)

Comment on this post

If you would like to make a comment, please fill out the form below.

Name (required)

Email (required)

Website

Comments

IMPORTANT! To be able to proceed, you need to solve the following simple math (so we know that you are a human) :-)

What is 11 + 4 ?
Please leave these two fields as-is:

+(reset)-
Follow me
© 2009 Simplysimple.info webmaster simplysimple.info