How to use Duplicity in Linux

To Create Encrypted and Bandwidth-efficient Backups in Linux

Duplicity is backup tool used to create Bandwidth-efficient and Encrypted backup in Linux based distributions. To connect to a file server for storing backup ssh, scp and sftp protocols are preferable methods, then rsync and ftp used for local file access. Procedure to use duplicity is explained in this article.

Testing Environment

Local Server &ndash 192.168.5.88
Backup Server &ndash 192.168.5.89

To Install Duplicity

Enable the EPEL repository before installing duplicity.

[root@linuxhelp ~]# yum install epel-release -y
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: ftp.iitm.ac.in
* extras: ftp.iitm.ac.in
* updates: ftp.iitm.ac.in
Resolving Dependencies
-->  Running transaction check
--->  Package epel-release.noarch 0:7-6 will be installed
.
.
.
Transaction test succeeded
Running transaction
Installing : epel-release-7-6.noarch 1/1
Verifying : epel-release-7-6.noarch 1/1
Installed:
epel-release.noarch 0:7-6
Complete!

To install duplicity execute the following command.
For Debian Derivatives

root@linuxhelp:~# aptitude update & &  aptitude install duplicity

For Redhat Derivatives

[root@linuxhelp ~]# yum install duplicity -y
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: ftp.iitm.ac.in
* epel: ftp.cuhk.edu.hk
* extras: ftp.iitm.ac.in
* updates: ftp.iitm.ac.in
Resolving Dependencies
-->  Running transaction check
--->  Package duplicity.x86_64 0:0.6.26-1.el7 will be installed
.
.
.
requests.noarch 0:2.6.0-1.el7_1
python-six.noarch 0:1.9.0-2.el7 python-urllib3.noarch 0:1.10.2-2.el7_1 python2-boto.noarch 0:2.39.0-1.el7
python2-crypto.x86_64 0:2.6.1-9.el7 python2-ecdsa.noarch 0:0.13-4.el7 python2-rsa.noarch 0:3.4.1-1.el7
Dependency Updated:
python-chardet.noarch 0:2.2.1-1.el7_1
Complete!

Creating SSH keys and GPG keys

Create SSH Keys to access remote servers and GPG keys for encryption. sshd daemon listen to the default port 22 in Backup server. The actual IP of the remote server is 192.168.5.89 and it should be replaced with your server IP. Now generate the ssh key with an encrypted Passphrase.

[root@linuxhelp ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
36:6f:80:a1:fc:ef:ce:93:17:2f:65:7c:2b:05:c3:e6 root@linuxhelp
The key' s randomart image is:
+--[ RSA 2048]----+
| |
| |
| . . |
| . . o = |
| o . S + o |
| . . +. E o |
| . .o= o . |
| oo.o o . |
| o=o . . |
+-----------------+

Now copy the ssh key to the Backup server.

[root@linuxhelp ~]# ssh-copy-id root@192.168.5.89
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.5.89' s password:
Number of key(s) added: 1
Now try logging into the machine, with: " ssh ' root@192.168.5.89' " 
and check to make sure that only the key(s) you wanted were added.

Connect to the backup server via ssh without using a password. For first time it will ask you to enter the ssh passphrase that we have used while generating the ssh key.

[root@linuxhelp ~]# ssh root@192.168.5.89

Last login: Fri May 20 15:19:11 2016 from 192.168.5.88

Enter the ssh key passphrase and click “ unlock” . Now you are logged into the backup server. Check the IP by using “ ifconfig” command. After that it will not ask any password for connecting to the backup server via ssh.
Create GPG keys to encrypt and decrypt data. You need to choose the type of key, key size, expiration for key and passphrase while generating the gpgkey.

[root@linuxhelp ~]# gpg --gen-key
gpg (GnuPG) 2.0.22  Copyright (C) 2013 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
gpg: directory `/root/.gnupg'  created
gpg: new configuration file `/root/.gnupg/gpg.conf'  created
gpg: WARNING: options in `/root/.gnupg/gpg.conf'  are not yet active during this run
gpg: keyring `/root/.gnupg/secring.gpg'  created
gpg: keyring `/root/.gnupg/pubring.gpg'  created
Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048)
Requested keysize is 2048 bits
Please specify how long the key should be valid.
0 = key does not expire
= key expires in n days
w = key expires in n weeks
m = key expires in n months
y = key expires in n years
Key is valid for? (0)
Key does not expire at all
Is this correct? (y/N) y
GnuPG needs to construct a user ID to identify your key.
Real name: linuxhelp
Email address:
Comment:
You selected this USER-ID:
" linuxhelp" 
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
You need a Passphrase to protect your secret key.

Enter the ssh key passphrase and click “ unlock” . Wait until the gpg key is generated.

You will receive an output as follows.

We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation  this gives the random number
generator a better chance to gain enough entropy.
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key 7DB2D34A marked as ultimately trusted
public and secret key created and signed.
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
pub 2048R/7DB2D34A 2016-05-20
Key fingerprint = D671 674C F48A 391E 579A 8FC8 EE43 BB54 7DB2 D34A
uid linuxhelp
sub 2048R/C18FCF85 2016-05-20

List the keys, once it has been generated.

[root@linuxhelp ~]# gpg --list-keys
/root/.gnupg/pubring.gpg
------------------------
pub 2048R/7DB2D34A 2016-05-20
uid linuxhelp
sub 2048R/C18FCF85 2016-05-20

Public key ID (7DB2D34A) is highlighted in the above output.

To Create a backup with Duplicity

Backup the ' /var/log' directory, with the exception of ' /var/log/anaconda' and ' /var/log/sa' .

Syntax
PASSPHRASE=" YourPassphraseHere" duplicity --encrypt-key YourPublicKeyIdHere --exclude /var/log/anaconda --exclude /var/log/sa /var/log scp://root@RemoteServerIP:port//localbackup/files

[root@linuxhelp ~]# PASSPHRASE=" linuxhelp"  duplicity --encrypt-key 7DB2D34A --exclude /var/log/anaconda --exclude /var/log/sa /var/log scp://root@192.168.5.89:22//localbackup/files
Duplicity 0.6 series is being deprecated:
See http://www.nongnu.org/duplicity/
The authenticity of host ' 192.168.5.89'  can' t be established.
SSH-RSA key fingerprint is 68:9a:c8:6f:bd:41:e2:84:18:8f:4a:7d:cb:45:7a:b1.
Are you sure you want to continue connecting (yes/no)? yes
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
--------------[ Backup Statistics ]--------------
StartTime 1463744436.19 (Fri May 20 17:10:36 2016)
EndTime 1463744436.81 (Fri May 20 17:10:36 2016)
ElapsedTime 0.62 (0.62 seconds)
SourceFiles 41
SourceFileSize 1352942 (1.29 MB)
NewFiles 41
NewFileSize 1352942 (1.29 MB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 41
RawDeltaSize 1348590 (1.29 MB)
TotalDestinationSizeChange 144862 (141 KB)
Errors 0
-------------------------------------------------

The backup is stored in the path ' /localbackup/files' . Verify it on the backup server by running the following command.

[root@linuxhelp ~]# ls /localbackup/files/
duplicity-full.20160520T114036Z.manifest.gpg duplicity-full-signatures.20160520T114036Z.sigtar.gpg
duplicity-full.20160520T114036Z.vol1.difftar.gpg

To Restore Backup

Delete yum.log in directory /var/log of the local server to make sure that the backup file doesn' t exist.

[root@linuxhelp ~]# cd /var/log
[root@linuxhelp log]# ls
anaconda btmp cups gdm maillog pm-powersave.log sa speech-dispatcher tuned Xorg.0.log.old
audit chrony dmesg lastlog messages ppp samba spooler wtmp Xorg.9.log
boot.log cron dmesg.old libvirt pluto qemu-ga secure tallylog Xorg.0.log yum.log

[root@linuxhelp log]# rm -rf yum.log
[root@linuxhelp log]# ls
anaconda btmp cups gdm maillog pm-powersave.log sa speech-dispatcher tuned Xorg.0.log.old
audit chrony dmesg lastlog messages ppp samba spooler wtmp Xorg.9.log
boot.log cron dmesg.old libvirt pluto qemu-ga secure tallylog Xorg.0.log

Run the following command to restore a single file from remote server.

Syntax
# PASSPHRASE=" YourPassphraseHere" duplicity --file-to-restore filename sftp://root@RemoteHostIP//localbackup/files /where/to/restore/filename

[root@linuxhelp log]# PASSPHRASE=" linuxhelp"  duplicity --file-to-restore yum.log sftp://root@192.168.5.89:22//localbackup/files /var/log/yum.log
Duplicity 0.6 series is being deprecated:
See http://www.nongnu.org/duplicity/
Synchronizing remote metadata to local cache...
Copying duplicity-full-signatures.20160520T114036Z.sigtar.gpg to local cache.
Copying duplicity-full.20160520T114036Z.manifest.gpg to local cache.
Last full backup date: Fri May 20 17:10:36 2016
Other Features of Duplicity:

By using the following command you may display list of archived files.

[root@linuxhelp ~]# duplicity list-current-files sftp://root@192.168.5.89:22//localbackup/files

For deleting 6 months older backups, run the following command.

[root@linuxhelp ~]# duplicity remove-older-than 6M sftp://root@192.168.5.89:22//localbackup/files