Continuous Backup of a Server

This config example shows how we can backup our ZFS-based server to another machine using a zrepl push job.

Production server prod with filesystems to back up:
- The entire pool zroot
- except zroot/var/tmp and all child datasets of it
- and except zroot/usr/home/paranoid which belongs to a user doing backups themselves.
Backup server backups with a dataset sub-tree for use by zrepl:
- In our example, that will be storage/zrepl/sink/prod.

Our backup solution should fulfill the following requirements:

Periodically snapshot the filesystems on prod every 10 minutes
Incrementally replicate these snapshots to storage/zrepl/sink/prod/* on backups
Keep only very few snapshots on prod to save disk space
Keep a fading history (24 hourly, 30 daily, 6 monthly) of snapshots on backups
The network is untrusted - zrepl should use TLS to protect its communication and our data.

Analysis

We can model this situation as two jobs:

A push job on prod
- Creates the snapshots
- Keeps a short history of local snapshots to enable incremental replication to backups
- Connects to the zrepl daemon process on backups
- Pushes snapshots backups
- Prunes snapshots on backups after replication is complete
A sink job on backups
- Accepts connections & responds to requests from prod
- Limits client prod access to filesystem sub-tree storage/zrepl/sink/prod

Generate TLS Certificates

We use the TLS client authentication transport to protect our data on the wire. To get things going quickly, we skip setting up a CA and generate two self-signed certificates as described here. For convenience, we generate the key pairs on our local machine and distribute them using ssh:

(name=backups; openssl req -x509 -sha256 -nodes \
 -newkey rsa:4096 \
 -days 365 \
 -keyout $name.key \
 -out $name.crt -addext "subjectAltName = DNS:$name" -subj "/CN=$name")

(name=prod; openssl req -x509 -sha256 -nodes \
 -newkey rsa:4096 \
 -days 365 \
 -keyout $name.key \
 -out $name.crt -addext "subjectAltName = DNS:$name" -subj "/CN=$name")

ssh root@backups "mkdir /etc/zrepl"
scp  backups.key backups.crt prod.crt root@backups:/etc/zrepl

ssh root@prod "mkdir /etc/zrepl"
scp  prod.key prod.crt backups.crt root@prod:/etc/zrepl

Note that alternative transports exist, e.g. via TCP without TLS or ssh.

Configure server `prod`

We define a push job named prod_to_backups in /etc/zrepl/zrepl.yml on host prod :

jobs:
- name: prod_to_backups
  type: push
  connect:
    type: tls
    address: "backups.example.com:8888"
    ca: /etc/zrepl/backups.crt
    cert: /etc/zrepl/prod.crt
    key:  /etc/zrepl/prod.key
    server_cn: "backups"
  filesystems: {
    "zroot<": true,
    "zroot/var/tmp<": false,
    "zroot/usr/home/paranoid": false
  }
  snapshotting:
    type: periodic
    prefix: zrepl_
    interval: 10m
  pruning:
    keep_sender:
    - type: not_replicated
    - type: last_n
      count: 10
    keep_receiver:
    - type: grid
      grid: 1x1h(keep=all) | 24x1h | 30x1d | 6x30d
      regex: "^zrepl_"

Configure server `backups`

We define a corresponding sink job named sink in /etc/zrepl/zrepl.yml on host backups :

jobs:
- name: sink
  type: sink
  serve:
      type: tls
      listen: ":8888"
      ca: "/etc/zrepl/prod.crt"
      cert: "/etc/zrepl/backups.crt"
      key: "/etc/zrepl/backups.key"
      client_cns:
        - "prod"
  root_fs: "storage/zrepl/sink"

Go Back To Quickstart Guide

Click here to go back to the quickstart guide.