Borg Backup
Sun, Oct 23, 2016I was looking for a good backup solution for a long time. Just recently I got the tip to have a look at borgbackup and think it is a simple and strong backup software. Today I set it up and start to test it for my server.
My backup setup is based on a local borg repository, which will be synced to my local Synology Diskstation as a remote backup. In this blogpost I will show how I setup borg and configured my daily backup.
Installation and configuration
The first step is the installation of borg. I installed the version provided by the CentOS repository. (1.0.7 as time of writing)
sudo yum install borgbackup
Then I needed to initiate a backup-repository for borg. You will need to create the folder first.
mkdir -p /var/backups/borgrepo
borg init /var/backups/borgrepo
Afterwards I defined the locations, which I would like to backup daily. In case of my server it is /opt, /etc, /home and /root. Also I looked for some excludes for files or folders, which I won’t backup and excluded them from the backup.
I play a lot with vagrant on my server but it’s really more playing and not that important for a backup. So I excluded Virtualbox VMs and .vagrant.d. This saved around 50gb of space for the backup.
With this information I modified the example configuration and saved it on my server (of cause in a location, which will be backed up).
I also edited the amount of backups which will be stored. Due to the fact, that borg has an excellent deduplication, the higher amount should not make large differences.
#!/bin/bash
REPOSITORY=/var/backups/borgrepo
TAG=daily
# Backup all of /home and some other folders
# except a few excluded directories
borg create -v --stats \
$REPOSITORY::$TAG'-{now:%Y-%m-%d}' \
/opt \
/etc \
/home \
/root \
--exclude '/home/*/.cache' \
--exclude '/home/*/.local' \
--exclude '/home/*/.pki' \
--exclude '/home/*/Virtualbox VMs' \
--exclude '/home/*/.vagrant.d' \
--exclude '/root/.cache' \
--exclude '/root/.local' \
--exclude '/root/.pki'
# Use the `prune` subcommand to maintain 14 daily, 8 weekly and 12 monthly
# archives of THIS machine. The 'daily-' prefix is very important to
# limit prune's operation to this machine's archives and not apply to
# other machine's archives also.
borg prune -v $REPOSITORY --prefix $TAG'-' \
--keep-daily=14 --keep-weekly=8 --keep-monthly=12
I executed the script with the TAG=init once. It backed up around 40GB in 15 minutes on my mashine. I changed the TAG back to daily and re-run the backup. It only needed 7 seconds this time, which is blazing fast I think.
[root@gerl1ng ~]# ./backup_daily.sh
------------------------------------------------------------------------------
Archive name: init-2016-10-23
Archive fingerprint: 7c05576dc8f821c22532d4da384899edd38aea16cb4ce29b8fe5456dd3e716d4
Time (start): Sun, 2016-10-23 23:32:28
Time (end): Sun, 2016-10-23 23:47:53
Duration: 15 minutes 24.18 seconds
Number of files: 38294
------------------------------------------------------------------------------
Original size Compressed size Deduplicated size
This archive: 40.48 GB 40.48 GB 32.49 GB
All archives: 40.48 GB 40.48 GB 32.49 GB
Unique chunks Total chunks
Chunk index: 39616 51329
------------------------------------------------------------------------------
[root@gerl1ng ~]# vi backup_daily.sh
[root@gerl1ng ~]# ./backup_daily.sh
------------------------------------------------------------------------------
Archive name: daily-2016-10-23
Archive fingerprint: c6eed8e023943fd410b8e38737334adc3a4533e34863d7adfd75d1c4220e09b9
Time (start): Sun, 2016-10-23 23:55:12
Time (end): Sun, 2016-10-23 23:55:19
Duration: 7.82 seconds
Number of files: 38294
------------------------------------------------------------------------------
Original size Compressed size Deduplicated size
This archive: 40.48 GB 40.48 GB 3.48 MB
All archives: 80.96 GB 80.96 GB 32.49 GB
Unique chunks Total chunks
Chunk index: 39644 102656
------------------------------------------------------------------------------
Restore tests
The best backup tool is worthless, if the restore is not working. I made some quick checks how a restore could be done. There are two different types of restores. You can extract the files directly into your current folder or you can mount a backup, browse the files and copy / save them.
First you can list the different backups with borg list $REPO
[root@gerl1ng ~]# borg list /var/backups/borgrepo/
init-2016-10-23 Sun, 2016-10-23 23:32:28
daily-2016-10-23 Sun, 2016-10-23 23:55:12
So I tried to mount the backup and have a look at the backup_daily.sh skript, as it is the only file which I changed between the backups.
[root@gerl1ng ~]# borg mount /var/backups/borgrepo::init-2016-10-23 /mnt/borg
[root@gerl1ng ~]# vi /mnt/borg/root/backup_daily.sh
[root@gerl1ng ~]# umount /mnt/borg
[root@gerl1ng ~]# borg mount /var/backups/borgrepo::daily-2016-10-23 /mnt/borg
[root@gerl1ng ~]# vi /mnt/borg/root/backup_daily.sh
[root@gerl1ng ~]# umount /mnt/borg
As expected it the first mount showed TAG=init and the secont mount showed TAG=daily.
Afterwards I extracted a folder and compared the extracted data to the originating files. The folder is /home/mgerling/perl_tsi
[root@gerl1ng ~]# du -hs /home/mgerling/perl_test/
1.1G /home/mgerling/perl_test/
[root@gerl1ng ~]# borg extract -v /var/backups/borgrepo::daily-2016-10-23 home/mgerling/perl_test
[root@gerl1ng ~]# du -hs home/mgerling/perl_test/
1.1G home/mgerling/perl_test/
[root@gerl1ng ~]# diff -r /home/mgerling/perl_test/ home/mgerling/perl_test/
And again as expected there was no different in the files. So I can say that the backups are restoreable. The extract of the 1.1 GB only took 40 seconds after all.
Cronjob
As I want to backup daily, I will include the script to the crontab.
[root@gerl1ng ~]# crontab -l
# mm hh dom mon dow command
00 3 * * * /root/backup_daily.sh
Now I will let borg work for some time and check it at one time or another. So far I can recommend borg.