Warning: This document is for an old version of zrepl. The main version is stable.

GitHub license Language: Go User Docs Donate via PayPal Donate via Liberapay Twitter

zrepl - ZFS replication

zrepl is a one-stop, integrated solution for ZFS replication.

  Job: prod_to_backups
  Type: push
  Replication:
      Attempt #1
      Status: fan-out-filesystems
      Progress: [=========================\----] 246.7 MiB / 264.7 MiB @ 11.5 MiB/s
        zroot              STEPPING (step 1/2, 624 B/1.2 KiB) next: @a => @b
        zroot/ROOT         DONE (step 2/2, 1.2 KiB/1.2 KiB)
        zroot/ROOT/default STEPPING (step 1/2, 123.4 MiB/129.3 MiB) next: @a => @b
        zroot/tmp          STEPPING (step 1/2, 29.9 KiB/44.2 KiB) next: @a => @b
        zroot/usr          STEPPING (step 1/2, 624 B/1.2 KiB) next: @a => @b
        zroot/usr/home     STEPPING (step 1/2, 123.3 MiB/135.3 MiB) next: @a => @b
        zroot/var          STEPPING (step 1/2, 624 B/1.2 KiB) next: @a => @b
        zroot/var/audit    DONE (step 2/2, 1.2 KiB/1.2 KiB)
        zroot/var/crash    DONE (step 2/2, 1.2 KiB/1.2 KiB)
        zroot/var/log      STEPPING (step 1/2, 22.0 KiB/29.2 KiB) next: @a => @b
        zroot/var/mail     STEPPING (step 1/2, 624 B/1.2 KiB) next: @a => @b
  Pruning Sender:
      ...
  Pruning Receiver:

Getting started

The 10 minutes tutorial setup gives you a first impression.

Main Features

  • Filesystem replication
    • [x] Pull & Push mode
    • [x] Multiple transport transports: TCP, TCP + TLS client auth, SSH
    • Advanced replication features
      • [x] Automatic retries for temporary network errors
      • [ ] Resumable send & receive
      • [ ] Compressed send & receive
      • [ ] Raw encrypted send & receive
  • Automatic snapshot management
    • [x] Periodic filesystem snapshots
    • [x] Flexible pruning rule system
      • [x] Age-based fading (grandfathering scheme)
      • [x] Bookmarks to avoid divergence between sender and receiver
  • Sophisticated Monitoring & Logging
    • [x] Live progress reporting via zrepl status subcommand
    • [x] Comprehensive, structured logging
      • human, logfmt and json formatting
      • stdout, syslog and TCP (+TLS client auth) outlets
    • [x] Prometheus monitoring endpoint
  • Maintainable implementation in Go
    • [x] Cross platform
    • [x] Type safe & testable code

Attention

zrepl as well as this documentation is still under active development. There is no stability guarantee on the RPC protocol or configuration format, but we do our best to document breaking changes in the Changelog.

Contributing

We are happy about any help we can get!

  • Explore the codebase
    • These docs live in the docs/ subdirectory
  • Document any non-obvious / confusing / plain broken behavior you encounter when setting up zrepl for the first time
  • Check the Issues and Projects sections for things to do. The good first issues and docs are suitable starting points.

Development Workflow

The GitHub repository is where all development happens. Make sure to read the Developer Documentation section and open new issues or pull requests there.