Continuous Backup of a Server

This config example shows how we can backup our ZFS-based server to another machine using a zrepl push job.

  • Production server prod with filesystems to back up:

    • The entire pool zroot

    • except zroot/var/tmp and all child datasets of it

    • and except zroot/usr/home/paranoid which belongs to a user doing backups themselves.

  • Backup server backups with a dataset sub-tree for use by zrepl:

    • In our example, that will be storage/zrepl/sink/prod.

Our backup solution should fulfill the following requirements:

  • Periodically snapshot the filesystems on prod every 10 minutes

  • Incrementally replicate these snapshots to storage/zrepl/sink/prod/* on backups

  • Keep only very few snapshots on prod to save disk space

  • Keep a fading history (24 hourly, 30 daily, 6 monthly) of snapshots on backups

  • The network is untrusted - zrepl should use TLS to protect its communication and our data.

Analysis

We can model this situation as two jobs:

  • A push job on prod

    • Creates the snapshots

    • Keeps a short history of local snapshots to enable incremental replication to backups

    • Connects to the zrepl daemon process on backups

    • Pushes snapshots backups

    • Prunes snapshots on backups after replication is complete

  • A sink job on backups

    • Accepts connections & responds to requests from prod

    • Limits client prod access to filesystem sub-tree storage/zrepl/sink/prod

Generate TLS Certificates

We use the TLS client authentication transport to protect our data on the wire. To get things going quickly, we skip setting up a CA and generate two self-signed certificates as described here. For convenience, we generate the key pairs on our local machine and distribute them using ssh:

(name=backups; openssl req -x509 -sha256 -nodes \
 -newkey rsa:4096 \
 -days 365 \
 -keyout $name.key \
 -out $name.crt -addext "subjectAltName = DNS:$name" -subj "/CN=$name")

(name=prod; openssl req -x509 -sha256 -nodes \
 -newkey rsa:4096 \
 -days 365 \
 -keyout $name.key \
 -out $name.crt -addext "subjectAltName = DNS:$name" -subj "/CN=$name")

ssh root@backups "mkdir /etc/zrepl"
scp  backups.key backups.crt prod.crt root@backups:/etc/zrepl

ssh root@prod "mkdir /etc/zrepl"
scp  prod.key prod.crt backups.crt root@prod:/etc/zrepl

Note that alternative transports exist, e.g. via TCP without TLS or ssh.

Configure server prod

We define a push job named prod_to_backups in /etc/zrepl/zrepl.yml on host prod :

jobs:
- name: prod_to_backups
  type: push
  connect:
    type: tls
    address: "backups.example.com:8888"
    ca: /etc/zrepl/backups.crt
    cert: /etc/zrepl/prod.crt
    key:  /etc/zrepl/prod.key
    server_cn: "backups"
  filesystems: {
    "zroot<": true,
    "zroot/var/tmp<": false,
    "zroot/usr/home/paranoid": false
  }
  snapshotting:
    type: periodic
    prefix: zrepl_
    interval: 10m
  pruning:
    keep_sender:
    - type: not_replicated
    - type: last_n
      count: 10
    keep_receiver:
    - type: grid
      grid: 1x1h(keep=all) | 24x1h | 30x1d | 6x30d
      regex: "^zrepl_"

Configure server backups

We define a corresponding sink job named sink in /etc/zrepl/zrepl.yml on host backups :

jobs:
- name: sink
  type: sink
  serve:
      type: tls
      listen: ":8888"
      ca: "/etc/zrepl/prod.crt"
      cert: "/etc/zrepl/backups.crt"
      key: "/etc/zrepl/backups.key"
      client_cns:
        - "prod"
  root_fs: "storage/zrepl/sink"

Go Back To Quickstart Guide

Click here to go back to the quickstart guide.