Logging

zrepl uses structured logging to provide users with easily processable log messages.

Logging outlets are configured in the global section of the main configuration file.

global:
  logging:

    - type: OUTLET_TYPE
      level: MINIMUM_LEVEL
      format: FORMAT

    - type: OUTLET_TYPE
      level: MINIMUM_LEVEL
      format: FORMAT

    ...

jobs: ...

Attention

The first outlet is special: if an error writing to any outlet occurs, the first outlet receives the error and can print it. Thus, the first outlet must be the one that always works and does not block, e.g. stdout, which is the default.

Default Configuration

By default, the following logging configuration is used

global:
  logging:

    - type: "stdout"
      level:  "warn"
      format: "human"

Building Blocks

The following sections document the semantics of the different log levels, formats and outlet types.

Levels

Level SHORT Description
error ERRO immediate action required
warn WARN symptoms for misconfiguration, soon expected failure, etc.
info INFO explains what happens without too much detail
debug DEBG tracing information, state dumps, etc. useful for debugging.

Incorrectly classified messages are considered a bug and should be reported.

Formats

Format Description
human prints job and subsystem into brackets before the actual message, followed by remaining fields in logfmt style
logfmt logfmt output. zrepl uses this Go package.
json JSON formatted output. Each line is a valid JSON document. Fields are marshaled by encoding/json.Marshal(), which is particularly useful for processing in log aggregation or when processing state dumps.

Outlets

Outlets are the destination for log entries.

stdout Outlet

Parameter Comment
type stdout
level minimum log level
format output format
time always include time in output (true or false)
color colorize output according to log level (true or false)

Writes all log entries with minimum level level formatted by format to stdout. If stdout is a tty, interactive usage is assumed and both time and color are set to true.

Can only be specified once.

syslog Outlet

Parameter Comment
type syslog
level minimum log level
format output format
retry_interval Interval between reconnection attempts to syslog (default = 0)

Writes all log entries formatted by format to syslog. On normal setups, you should not need to change the retry_interval.

Can only be specified once.

tcp Outlet

Parameter Comment
type tcp
level minimum log level
format output format
net tcp in most cases
address remote network, e.g. logs.example.com:10202
retry_interval Interval between reconnection attempts to address
tls TLS config (see below)

Establishes a TCP connection to address and sends log messages with minimum level level formatted by format. If tls is not specified, an unencrypted connection is established. If tls is specified, the TCP connection is secured with TLS + Client Authentication. The latter is particularly useful in combination with log aggregation services.

Parameter Description
ca PEM-encoded certificate authority that signed the remote server’s TLS certificate
cert PEM-encoded client certificate identifying this zrepl daemon toward the remote server
key PEM-encoded, unencrypted client private key identifying this zrepl daemon toward the remote server

Warning

zrepl drops log messages to the TCP outlet if the underlying connection is not fast enough. Note that TCP buffering in the kernel must first run full becfore messages are dropped.

Make sure to always configure a stdout outlet as the special error outlet to be informed about problems with the TCP outlet (see above ).

Note

zrepl uses Go’s crypto/tls and crypto/x509 packages and leaves all but the required fields in tls.Config at their default values. In case of a security defect in these packages, zrepl has to be rebuilt because Go binaries are statically linked.