fogserv.cloud fogserv.cloud

Restic Backups - Encrypted, Deduplicated Backups Done Right

Status: Active
Last Updated: 2026-08-26
Category: Cloud - Phase 4: Backups & Disaster Recovery
Prerequisites: cloud-storage-concepts, minio-setup, docker-volumes
Time: 2-3 hours
Tags: restic, backup, restore, snapshots, deduplication, encryption, systemd-timer, s3, minio, disaster-recovery

Summary

restic is a modern backup program that does the three things most backup tools get wrong out of the box: it encrypts everything client-side before data leaves the machine, deduplicates across every snapshot so daily backups of a 100 GB server cost megabytes not gigabytes, and stores versioned snapshots you can browse and mount like a filesystem. This guide recaps the 3-2-1 rule, explains restic's core concepts, walks through initializing repositories on a local directory and on S3/MinIO, covers the essential command set (backup, snapshots, restore, check, forget --prune), and assembles a production-grade automated backup job with systemd service + timer plus monitoring so silent failures can't happen.

🎯 What You'll Learn

By the end of this article, you'll be able to:

Table of Contents

  1. Backup Strategy: The 3-2-1 Rule
  2. How restic Thinks: Snapshots, Dedup, Encryption
  3. Preparing a Repository
  4. The Essential Command Set
  5. Retention: forget --keep and Pruning
  6. Automation with systemd Service + Timer
  7. Monitoring Backup Success
  8. Troubleshooting & Common Pitfalls
  9. Key Takeaways
  10. Sources & Related

Backup Strategy: The 3-2-1 Rule

Before any tooling: the strategy that survives every failure mode.

Digit Rule Example in this stack
3 Three copies of your data Live data + local restic repo + offsite MinIO/cloud repo
2 Two different media/systems Local disk + object storage on another host
1 One copy offsite restic → MinIO on a remote server or B2/Wasabi

Two additions worth adopting:

What to back up on a typical Docker host: bind-mount and named-volume data (databases need dump-first handling — see pitfalls), configs, compose files, and secrets. Not container images (rebuildable) and not /proc-style ephemera.

How restic Thinks: Snapshots, Dedup, Encryption

Three concepts cover 90% of the mental model:

Snapshots

Each restic backup run creates a snapshot: a point-in-time tree of the directories you backed up, tagged with host, time, and labels. Snapshots are cheap because they share data.

Content-Defined Deduplication

Files are split into variable-size chunks based on content (not fixed offsets). Each chunk is stored once, addressed by its SHA-256 hash. Consequences:

This is why "snapshot per day" is a sane default rather than a storage bomb.

Encryption

Everything — data chunks, filenames, metadata — is encrypted on the client with AES-256-CTR + Poly1305 MAC before it's written to the repository. The backend never sees plaintext. That means:

Repository layout is a flat structure of data/, index/, keys/, snapshots/, locks/ — identical whether the repo sits on disk, SFTP, S3, or any supported backend, which is why tools work interchangeably.

Preparing a Repository

Install first:

# Debian/Ubuntu
sudo apt install restic          # often older; prefer binary for latest features
# Official binary (recommended)
sudo wget -O /usr/local/bin/restic https://github.com/restic/restic/releases/download/v0.17.3/restic_0.17.3_linux_amd64.bz2

(Grab the current release filename from https://github.com/restic/restic/releases; decompress with bunzip2 and chmod +x.)

Option A: Local Directory Repository

export RESTIC_REPOSITORY=/srv/backups/server1
export RESTIC_PASSWORD='your-strong-passphrase'   # see env-file pattern below

restic init
# created restic repository b61f2b5c9a at /srv/backups/server1

Good for fast local copies (second copy in 3-2-1). Put it on a separate disk from the live data, not inside /home.

Option B: S3 / MinIO Backend

restic speaks native S3 against Amazon, MinIO, Wasabi, Backblaze B2 (via its S3 API), etc. Per the official docs, set credentials and init with an s3: URL:

export AWS_ACCESS_KEY_ID=restic-backup-key       # MinIO access key from the console
export AWS_SECRET_ACCESS_KEY=restic-backup-secret
export RESTIC_REPOSITORY=s3:http://localhost:9000/restic-backups   # path-style: host/bucket
export RESTIC_PASSWORD='your-strong-passphrase'

restic init
# created restic repository 6ad29560f5 at s3:http://localhost:9000/restic-backups

Notes grounded in the restic docs:

Credential Hygiene: Env File Pattern

Never put the passphrase in scripts that might leak. Use a root-only env file:

sudo install -m 600 /dev/null /etc/restic-env
sudo tee /etc/restic-env >/dev/null <<'EOF'
RESTIC_REPOSITORY=s3:http://localhost:9000/restic-backups
AWS_ACCESS_KEY_ID=restic-backup-key
AWS_SECRET_ACCESS_KEY=restic-backup-secret
RESTIC_PASSWORD=your-strong-passphrase
EOF

Every command then becomes:

sudo bash -c 'source /etc/restic-env && restic snapshots'

The Essential Command Set

All examples assume the env file sourced (shown as $RESTIC ... shorthand).

backup — create a snapshot

$RESTIC backup /etc /home /srv/docker/volumes --tag daily
# summary prints: scanned files, new/changed bytes ("added to the repo"),
# processed bytes — tiny numbers prove dedup is working

Useful flags:

Databases need consistent dumps, not raw file copies: run pg_dump/mysqldump into a staging dir first and back that up (script shown later).

snapshots — list what exists

$RESTIC snapshots                          # all, grouped table
$RESTIC snapshots --tag daily --last       # filtered
$RESTIC snapshots --json                   # machine-readable (for monitoring scripts)

restore — get data back

# Whole snapshot to a directory (use a scratch dir, not over live data!)
$RESTIC restore latest --target /tmp/restore-test

# Single path out of a snapshot
$RESTIC restore latest:/home/alice/thesis.pdf --target /tmp/out

# Restore a specific snapshot by short ID from `snapshots`
$RESTIC restore 4a5b6c7d --target /tmp/out --include /etc/nginx

You can also browse without restoring:

$RESTIC ls latest                      # list files in snapshot
$RESTIC mount /mnt/restic              # FUSE-mount all snapshots read-only
ls /mnt/restic/snapshots/latest/home/alice/

check — verify integrity

$RESTIC check                    # metadata/index consistency (fast-ish)
$RESTIC check --read-data        # reads ALL data packs; slow, schedule monthly/quarterly
$RESTIC check --read-data-subset=10%    # spread full verification across runs

This is your "3-2-1-0" verification digit — an unverified backup is a hope, not a backup.

Diffing and stats (nice-to-haves)

$RESTIC diff <snap-id-1> <snap-id-2>
$RESTIC stats latest --mode restore-size
$RESTIC stats --mode raw-data            # actual repo size after dedup

Retention: forget --keep and Pruning

Snapshots accumulate forever unless told otherwise. forget removes snapshots from the index; --prune additionally deletes now-unreferenced data from the repo (recovering space).

# Keep: 7 daily, 5 weekly, 12 monthly, 3 yearly — plus ALL snapshots from the last 24h.
# Tags/hosts filter applies per policy group.
$RESTIC forget \
  --host "$HOSTNAME" \
  --keep-daily 7 \
  --keep-weekly 5 \
  --keep-monthly 12 \
  --keep-yearly 3 \
  --keep-within 24h \
  --prune

Key semantics:

Grandfather-father-son (GFS) as above gives roughly: a week of dailies, a month+ of weeklies, a year+ of monthlies — a few hundred snapshots total for pennies of deduplicated storage.

Automation with systemd Service + Timer

Cron works, but systemd timers give you logging to the journal, dependency ordering, and missed-run catch-up. Three files:

/etc/systemd/system/restic-backup.service

[Unit]
Description=Restic backup to MinIO S3
Wants=network-online.target
After=network-online.target docker.service

[Service]
Type=oneshot
EnvironmentFile=/etc/restic-env
ExecStartPre=/usr/local/bin/dump-databases.sh /var/backups/dbdump
Nice=10
IOSchedulingClass=idle
ExecStart=/usr/bin/restic backup /etc /home /srv/docker/volumes /var/backups/dbdump --tag auto
ExecStartPost=/usr/local/bin/restic-forget.sh
ExecStartPost=/usr/local/bin/notify-backup-result success
OnFailure=/usr/local/bin/notify-backup-result failure

/etc/systemd/system/restic-backup.timer

[Unit]
Description=Nightly restic backup

[Timer]
OnCalendar=*-*-* 02:30:00
RandomizedDelaySec=15m
Persistent=true          # run missed jobs after downtime/reboot

[Install]
WantedBy=timers.target

Supporting scripts

/usr/local/bin/restic-forget.sh (retention + verify in one place):

#!/usr/bin/env bash
set -euo pipefail
source /etc/restic-env
restic forget --host "$(hostname)" \
  --keep-daily 7 --keep-weekly 5 --keep-monthly 12 \
  --keep-within 24h --prune
restic check

/usr/local/bin/notify-backup-result (hook for Uptime Kuma/Discord/email):

#!/usr/bin/env bash
STATE="$1"
KUMA_URL="${KUMA_PUSH_URL:-}"     # e.g. http://kuma.local:3001/api/push/AbCdEf?status=up&msg=OK&ping=
if [[ -n "$KUMA_URL" ]]; then
  status=$([[ "$STATE" == success ]] && echo up || echo down)
  msg=$([[ "$STATE" == success ]] && echo ok || echo FAILED)
  curl -fsS --max-time 10 "${KUMA_URL%%\?*}?status=${status}&msg=${msg}&ping=" >/dev/null || true
fi
logger -t restic "backup result: $STATE"

Enable:

chmod +x /usr/local/bin/{dump-databases.sh,restic-forget.sh,notify-backup-result}
sudo systemctl daemon-reload
sudo systemctl enable --now restic-backup.timer
systemctl list-timers restic-backup.timer     # confirm next run time
sudo systemctl start restic-backup.service    # fire once now, watch journalctl -u restic-backup -f

Monitoring Backup Success

Backups fail quietly (full disk, expired key, network blip) and nobody notices until restore day. Layer three checks:

  1. Push heartbeat — the notify script pings Uptime Kuma's push monitor; Kuma alerts if no ping arrives within N hours (catches "job never ran"). See uptime-kuma-setup.

  2. Journal-based alerting — Netdata or a log watcher flags logger -t restic failure lines. Quick manual audit:

    journalctl -u restic-backup.service --since "7 days ago" | grep -c "backup result: success"

  3. Restore drill — quarterly, actually restore something and diff it:

    $RESTIC restore latest --target /tmp/drill && diff -r /tmp/drill/etc /etc | head

Bonus freshness check for scripts/cron-emails:

LAST=$(restic snapshots --latest 1 --json | jq -r '.[0].time')
AGE_HOURS=$(( ($(date +%s) - $(date -d "$LAST" +%s)) / 3600 ))
[[ $AGE_HOURS -gt 26 ]] && echo "ALERT: last backup ${AGE_HOURS}h ago" >&2

Troubleshooting & Common Pitfalls

Fatal: wrong password or no key found Passphrase mismatch — usually a stale env var in your shell overriding the env file. Check with env | grep RESTIC. There is no reset; this is why the passphrase lives in two places.

Backing up databases directly from data dirs Raw volume copies of a running Postgres/MySQL may be inconsistent. Always pg_dump/mysqldump (or stop the container) first — the ExecStartPre hook above exists exactly for this.

Repo grows unboundedly Forgetting to run forget --prune (snapshots pile up), or huge churn files (VM images, logs being rewritten) defeating dedup. Check what changed: restic diff two snapshots. Consider excluding volatile paths.

check --read-data takes forever It re-reads every byte. Use --read-data-subset=10% monthly instead, or schedule a full read annually.

Stale lock after crash Fatal: unable to create lock in backend when nothing else is running → restic unlock. Verify no concurrent restic processes first (pgrep restic).

S3 backend errors: SignatureDoesNotMatch / access denied MinIO access-key rotation broke the env file, or clock skew between hosts. Also confirm the key's policy allows the repo bucket (minio-setup).

Timer shows "Trigger: n/a" Timer unit not enabled or Persistent=true missing while the machine was off. systemctl enable --now restic-backup.timer and check list-timers.

Restored files have odd ownership Restic preserves uid/gid; restoring as non-root maps them oddly. Restore as root, or use --user/--group options on restore.

Key Takeaways

  1. 3-2-1 (or 3-2-1-1-0) is the plan; restic is just the tool. Offsite + verified beats fancy.
  2. Client-side encryption means the password is everything — protect it like the data itself.
  3. Content-defined chunking makes daily snapshots nearly free; watch "added to repo" shrink after run #1.
  4. Same commands everywhere: swap RESTIC_REPOSITORY between local dir and s3:http://minio:9000/bucket.
  5. Rehearse retention with --dry-run; verify with check; prove value with test restores.
  6. systemd service + timer + push heartbeat = backups that tattle on themselves instead of failing silently.

Sources & Related

Web Sources

Related KB Articles

Change Log

Choose Theme

Your selection is saved locally.

Neural Cacophony
Aperture v2
Flux v1
Mosaic Chaos
Nexus v1
Nexus Zest
Prism v2
Synapse