Fan-out replication

This quick-start example demonstrates how to implement a fan-out replication setup where datasets on a server (A) are replicated to multiple targets (B, C, etc.).

This example uses multiple source jobs on server A and pull jobs on the target servers.

Warning

Before implementing this setup, please see the caveats listed in the fan-out replication configuration overview.

Overview

On the source server (A), there should be:

A snap job
- Creates the snapshots
- Handles the pruning of snapshots
A source job for target B
- Accepts connections from server B and B only
Further source jobs for each additional target (C, D, etc.)
- Listens on a unique port
- Only accepts connections from the specific target

On each target server, there should be:

A pull job that connects to the corresponding source job on A
- prune_sender should keep all snapshots since A’s snap job handles the pruning
- prune_receiver can be configured as appropriate on each target server

Generate TLS Certificates

Mutual TLS via the TLS client authentication transport can be used to secure the connections between the servers. In this example, a self-signed certificate is created for each server without setting up a CA.

source=a.example.com
targets=(
    b.example.com
    c.example.com
    # ...
)

for server in "${source}" "${targets[@]}"; do
    openssl req -x509 -sha256 -nodes \
        -newkey rsa:4096 \
        -days 365 \
        -keyout "${server}.key" \
        -out "${server}.crt" \
        -addext "subjectAltName = DNS:${server}" \
        -subj "/CN=${server}"
done

# Distribute each host's keypair
for server in "${source}" "${targets[@]}"; do
    ssh root@"${server}" mkdir /etc/zrepl
    scp "${server}".{crt,key} root@"${server}":/etc/zrepl/
done

# Distribute target certificates to the source
scp "${targets[@]/%/.crt}" root@"${source}":/etc/zrepl/

# Distribute source certificate to the targets
for server in "${targets[@]}"; do
    scp "${source}.crt" root@"${server}":/etc/zrepl/
done

Configure source server A

jobs:
# Separate job for snapshots and pruning
- name: snapshots
  type: snap
  filesystems:
    'tank<': true # all filesystems
  snapshotting:
    type: periodic
    prefix: zrepl_
    interval: 10m
  pruning:
    keep:
      # Keep non-zrepl snapshots
      - type: regex
        negate: true
        regex: '^zrepl_'
      # Time-based snapshot retention
      - type: grid
        grid: 1x1h(keep=all) | 24x1h | 30x1d | 12x30d
        regex: '^zrepl_'

# Source job for target B
- name: target_b
  type: source
  serve:
    type: tls
    listen: :8888
    ca: /etc/zrepl/b.example.com.crt
    cert: /etc/zrepl/a.example.com.crt
    key: /etc/zrepl/a.example.com.key
    client_cns:
      - b.example.com
  filesystems:
    'tank<': true # all filesystems
  # Snapshots are handled by the separate snap job
  snapshotting:
    type: manual

# Source job for target C
- name: target_c
  type: source
  serve:
    type: tls
    listen: :8889
    ca: /etc/zrepl/c.example.com.crt
    cert: /etc/zrepl/a.example.com.crt
    key: /etc/zrepl/a.example.com.key
    client_cns:
      - c.example.com
  filesystems:
    'tank<': true # all filesystems
  # Snapshots are handled by the separate snap job
  snapshotting:
    type: manual

# Source jobs for remaining targets. Each one should listen on a different port
# and reference the correct certificate and client CN.
# - name: target_c
#   ...

Configure each target server

jobs:
# Pull from source server A
- name: source_a
  type: pull
  connect:
    type: tls
    # Use the correct port for this specific client (eg. B is 8888, C is 8889, etc.)
    address: a.example.com:8888
    ca: /etc/zrepl/a.example.com.crt
    # Use the correct key pair for this specific client
    cert: /etc/zrepl/b.example.com.crt
    key: /etc/zrepl/b.example.com.key
    server_cn: a.example.com
  root_fs: pool0/backup
  interval: 10m
  pruning:
    keep_sender:
      # Source does the pruning in its snap job
      - type: regex
        regex: '.*'
    # Receiver-side pruning can be configured as desired on each target server
    keep_receiver:
      # Keep non-zrepl snapshots
      - type: regex
        negate: true
        regex: '^zrepl_'
      # Time-based snapshot retention
      - type: grid
        grid: 1x1h(keep=all) | 24x1h | 30x1d | 12x30d
        regex: '^zrepl_'

Go Back To Quickstart Guide

Click here to go back to the quickstart guide.