You're reading an old version of this documentation. If you want up-to-date information, please have a look at stable.

Send & Recv Options

Send Options

Source and push jobs have an optional send configuration section.

jobs:
- type: push
  filesystems: ...
  send:
    # flags from the table below go here
  ...

The following table specifies the list of (boolean) options. Flags with an entry in the zfs send column map directly to the zfs send CLI flags. zrepl does not perform feature checks for these flags. If you enable a flag that is not supported by the installed version of ZFS, the zfs error will show up at runtime in the logs and zrepl status. See the upstream man page (man zfs-send) for their semantics.

send.	zfs send	Comment
encrypted		Specific to zrepl, see below.
raw	-w	Use encrypted to only allow encrypted sends.
send_properties	-p	Be careful, read the note on property replication below.
backup_properties	-b	Be careful, read the note on property replication below.
large_blocks	-L	Potential data loss on OpenZFS < 2.0, see warning below.
compressed	-c
embbeded_data	-e
saved	-S

encrypted

The encrypted option controls whether the matched filesystems are sent as OpenZFS native encryption raw sends. More specifically, if encrypted=true, zrepl

• checks for any of the filesystems matched by filesystems whether the ZFS encryption property indicates that the filesystem is actually encrypted with ZFS native encryption and

• invokes the zfs send subcommand with the -w option (raw sends) and

• expects the receiving side to support OpenZFS native encryption (recv will fail otherwise)

Filesystems matched by filesystems that are not encrypted are not sent and will cause error log messages.

If encrypted=false, zrepl expects that filesystems matching filesystems are not encrypted or have loaded encryption keys.

Note

Use encrypted instead of raw to make your intent clear that zrepl must only replicate filesystems that are actually encrypted by OpenZFS native encryption. It is meant as a safeguard to prevent unintended sends of unencrypted filesystems in raw mode.

properties

Sends the dataset properties along with snapshots. Please be careful with this option and read the note on property replication below.

backup_properties

When properties are modified on a filesystem that was received from a send stream with send.properties=true, ZFS archives the original received value internally. This also applies to inheriting or overriding properties during zfs receive.

When sending those received filesystems another hop, the backup_properties flag instructs ZFS to send the original property values rather than the current locally set values.

This is useful for replicating properties across multiple levels of backup machines. Example: Suppose we want to flow snapshots from Machine A to B, then from B to C. A will enable the properties send option. B will want to override critical properties such as mountpoint or canmount. But the job that replicates from B to C should be sending the original property values received from A. Thus, B sets the backup_properties option.

Please be careful with this option and read the note on property replication below.

large_blocks

This flag should not be changed after initial replication. Prior to OpenZFS commit 7bcb7f08 it was possible to change this setting which resulted in data loss on the receiver. The commit in question is included in OpenZFS 2.0 and works around the problem by prohibiting receives of incremental streams with a flipped setting.

Warning

This bug has not been fixed in the OpenZFS 0.8 releases which means that changing this flag after initial replication might cause data loss on the receiver.

Recv Options

Sink and pull jobs have an optional recv configuration section:

jobs:
- type: pull
  recv:
    properties:
      inherit:
        - "mountpoint"
      override: {
        "org.openzfs.systemd:ignore": "on"
      }
  ...

properties

override maps directly to the zfs recv -o flag. Property name-value pairs specified in this map will apply to all received filesystems, regardless of whether the send stream contains properties or not.

inherit maps directly to the zfs recv -x flag. Property names specified in this list will be inherited from the receiving side’s parent filesystem (e.g. root_fs).

With both options, the sending side’s property value is still stored on the receiver, but the local override or inherit is the one that takes effect. You can send the original properties from the first receiver to another receiver using send.backup_properties.

A Note on Property Replication

If a send stream contains properties, as per send.properties or send.backup_properties, the default ZFS behavior is to use those properties on the receiving side, verbatim.

In many use cases for zrepl, this can have devastating consequences. For example, when backing up a filesystem that has mountpoint=/ to a storage server, that storage server’s root filesystem will be shadowed by the received file system on some platforms. Also, many scripts and tools use ZFS user properties for configuration and do not check the property source (local vs. received). If they are installed on the receiving side as well as the sending side, property replication could have unintended effects.

zrepl currently does not provide any automatic safe-guards for property replication:

• Make sure to read the entire man page on zfs recv (man zfs recv) before enabling this feature.

• Use recv.properties.override whenever possible, e.g. for mountpoint=none or canmount=off.

• Use recv.properties.inherit if that makes more sense to you.

Below is an non-exhaustive list of problematic properties. Please open a pull request if you find a property that is missing from this list. (Both with regards to core ZFS tools and other software in the broader ecosystem.)

Mount behaviour

• mountpoint

• canmount

• overlay

Note: inheriting or overriding the mountpoint property on ZVOLs fails in zfs recv. This is an issue in OpenZFS . As a workaround, consider creating separate zrepl jobs for your ZVOL and filesystem datasets. Please comment at zrepl issue #430 if you encounter this issue and/or would like zrepl to automatically work around it.

Systemd

With systemd, you should also consider the properties processed by the zfs-mount-generator .

Most notably:

• org.openzfs.systemd:ignore

• org.openzfs.systemd:wanted-by

• org.openzfs.systemd:required-by

Encryption

If the sender filesystems are encrypted but the sender does plain sends and property replication is enabled, the receiver must inherit the following properties:

• keylocation

• keyformat

• encryption