Fan-out replication

This quick-start example demonstrates how to implement a fan-out replication setup where datasets on a server (A) are replicated to multiple targets (B, C, etc.).

This example uses multiple source jobs on server A and pull jobs on the target servers.

Warning

Before implementing this setup, please see the caveats listed in the fan-out replication configuration overview.

Overview

On the source server (A), there should be:

• A snap job

  • Creates the snapshots

  • Handles the pruning of snapshots

• A source job for target B

  • Accepts connections from server B and B only

• Further source jobs for each additional target (C, D, etc.)

  • Listens on a unique port

  • Only accepts connections from the specific target

On each target server, there should be:

• A pull job that connects to the corresponding source job on A

  • prune_sender should keep all snapshots since A’s snap job handles the pruning

  • prune_receiver can be configured as appropriate on each target server

Generate TLS Certificates

Mutual TLS via the TLS client authentication transport can be used to secure the connections between the servers. In this example, a self-signed certificate is created for each server without setting up a CA.

source=a.example.com
targets=(
    b.example.com
    c.example.com
    # ...
)

for server in "${source}" "${targets[@]}"; do
    openssl req -x509 -sha256 -nodes \
        -newkey rsa:4096 \
        -days 365 \
        -keyout "${server}.key" \
        -out "${server}.crt" \
        -addext "subjectAltName = DNS:${server}" \
        -subj "/CN=${server}"
done

# Distribute each host's keypair
for server in "${source}" "${targets[@]}"; do
    ssh root@"${server}" mkdir /etc/zrepl
    scp "${server}".{crt,key} root@"${server}":/etc/zrepl/
done

# Distribute target certificates to the source
scp "${targets[@]/%/.crt}" root@"${source}":/etc/zrepl/

# Distribute source certificate to the targets
for server in "${targets[@]}"; do
    scp "${source}.crt" root@"${server}":/etc/zrepl/
done

Configure source server A

jobs:
# Separate job for snapshots and pruning
- name: snapshots
  type: snap
  filesystems:
    'tank<': true # all filesystems
  snapshotting:
    type: periodic
    prefix: zrepl_
    interval: 10m
  pruning:
    keep:
      # Keep non-zrepl snapshots
      - type: regex
        negate: true
        regex: '^zrepl_'
      # Time-based snapshot retention
      - type: grid
        grid: 1x1h(keep=all) | 24x1h | 30x1d | 12x30d
        regex: '^zrepl_'

# Source job for target B
- name: target_b
  type: source
  serve:
    type: tls
    listen: :8888
    ca: /etc/zrepl/b.example.com.crt
    cert: /etc/zrepl/a.example.com.crt
    key: /etc/zrepl/a.example.com.key
    client_cns:
      - b.example.com
  filesystems:
    'tank<': true # all filesystems
  # Snapshots are handled by the separate snap job
  snapshotting:
    type: manual

# Source job for target C
- name: target_c
  type: source
  serve:
    type: tls
    listen: :8889
    ca: /etc/zrepl/c.example.com.crt
    cert: /etc/zrepl/a.example.com.crt
    key: /etc/zrepl/a.example.com.key
    client_cns:
      - c.example.com
  filesystems:
    'tank<': true # all filesystems
  # Snapshots are handled by the separate snap job
  snapshotting:
    type: manual

# Source jobs for remaining targets. Each one should listen on a different port
# and reference the correct certificate and client CN.
# - name: target_c
#   ...

Configure each target server

jobs:
# Pull from source server A
- name: source_a
  type: pull
  connect:
    type: tls
    # Use the correct port for this specific client (eg. B is 8888, C is 8889, etc.)
    address: a.example.com:8888
    ca: /etc/zrepl/a.example.com.crt
    # Use the correct key pair for this specific client
    cert: /etc/zrepl/b.example.com.crt
    key: /etc/zrepl/b.example.com.key
    server_cn: a.example.com
  root_fs: pool0/backup
  interval: 10m
  pruning:
    keep_sender:
      # Source does the pruning in its snap job
      - type: regex
        regex: '.*'
    # Receiver-side pruning can be configured as desired on each target server
    keep_receiver:
      # Keep non-zrepl snapshots
      - type: regex
        negate: true
        regex: '^zrepl_'
      # Time-based snapshot retention
      - type: grid
        grid: 1x1h(keep=all) | 24x1h | 30x1d | 12x30d
        regex: '^zrepl_'

Go Back To Quickstart Guide

Click here to go back to the quickstart guide.