Caching Strategies - Faster Woodpecker Pipelines with Layer, Dependency, and Volume Caches

Status: Active
Last Updated: 2026-08-26
Category: CI/CD - Phase 2: Performance
Prerequisites: woodpecker-first-pipeline, docker-volumes, multi-stage-builds
Time: 2 hours
Tags: caching, woodpecker, docker, buildkit, performance, pipelines

Summary

Cut pipeline times from minutes to seconds with three cache layers: Docker layer caches for image builds, mounted volume caches on runners for tool/dependency directories, and registry-based caches. This article shows where each applies in a Woodpecker setup and how to avoid the classic stale-cache traps.

🎯 What You'll Learn

By the end of this article, you'll be able to:

✅ Enable BuildKit layer caching for pipeline image builds
✅ Mount persistent volumes and use a cache plugin on agents
✅ Cache language dependencies (npm, pip, Go modules) correctly
✅ Invalidate caches safely when they go stale

Context / Why This Matters
Implementation / Core Content
Practical Examples
Troubleshooting & Common Pitfalls
Next Steps / Ops Actions
Sources & Related

Context / Why This Matters

Every pipeline that starts from zero — download Node modules, re-resolve pip deps, rebuild all fifteen Docker layers — wastes runner CPU and, worse, your attention: slow pipelines get run less often, and infrequent merges are how integration pain compounds. After woodpecker-first-pipeline your builds are correct; this article makes them fast enough to run on every push.

Caching happens at distinct layers, each with its own mechanism:

Container image layers — cached by the Docker daemon doing the build (BuildKit).
Dependency/tool directories — persisted between runs via agent volumes or a cache plugin.
Scanner databases (Trivy, etc.) — same mechanism as #2 but worth calling out (security-scanning).

Implementation / Core Content

1. Docker layer caching

The plugins/docker step runs its own DinD daemon whose cache dies with the workflow unless you attach a volume or use cache_from:

steps:
  build:
    image: plugins/docker
    settings:
      repo: registry.fogserv.cloud/homelab/myapp
      tags: "${CI_COMMIT_SHA}"
      username:
        from_secret: registry_user
      password:
        from_secret: registry_pass
      # pull previous images as cache sources before building
      cache_from:
        - "registry.fogserv.cloud/homelab/myapp:cache"
      # after build, retag result as :cache so next run benefits
      tags_cache: "cache"

This works best combined with well-layered Dockerfiles from dockerfile-guide: copy lockfile + install first, source code last, so dependency layers survive code-only changes. multi-stage-builds keeps final images small regardless of builder cache size.

For self-built images with plain docker build, mount a persistent builder cache on the agent host:

  docker-build:
    image: docker:27
    commands:
      - docker buildx build --cache-from type=local,src=/cache/buildkit --cache-to type=local,dest=/cache/buildkit,mode=max .
    volumes:
      - /var/lib/woodpecker/cache/buildkit:/cache/buildkit

2. Persistent volumes on the agent

Anything under an agent-mounted directory survives between workflows. Reserve one directory per purpose and mount it into steps:

steps:
  test:
    image: node:22
    environment:
      npm_config_cache: /woodpecker/cache/npm
    commands:
      - npm ci --prefer-offline
    volumes:
      - /var/lib/woodpecker/cache:/woodpecker/cache

Rules that keep shared caches safe:

One subdirectory per package manager (npm, pip, go-build), never share across projects.
Content-addressed caches (npm/pip/Go) tolerate concurrent access poorly — if you run multiple agents, give each agent its own host path rather than sharing over NFS.
The cache is disposable: any step must succeed with an empty cache.

3. Language-specific cache configuration

# npm — cache dir keyed by content hash automatically
environment:
  npm_config_cache: /woodpecker/cache/npm

# pip
commands:
  - pip cache dir                       # discover location per version
  - PIP_CACHE_DIR=/woodpecker/cache/pip pip install -r requirements.txt

# Go
environment:
  GOCACHE: /woodpecker/cache/go-build
  GOMODCACHE: /woodpecker/cache/go-mod

# Gradle
environment:
  GRADLE_USER_HOME: /woodpecker/cache/gradle

4. The Woodpecker cache plugin (archive-style restore/save)

When steps run on ephemeral executors without shared storage, use the cache plugin to pack/restore directories as tarballs:

steps:
  restore-cache:
    image: meltwater/drone-cache
    environment:
      AWS_ACCESS_KEY_ID:
        from_secret: cache_s3_key
      AWS_SECRET_ACCESS_KEY:
        from_secret: cache_s3_secret
    settings:
      backend: s3
      archive_format: gzip
      rebuild: false
      cache_key: '{{ checksum "package-lock.json" }}'
      mount:
        - node_modules

  test:
    image: node:22
    commands:
      - npm ci --prefer-offline

  rebuild-cache:
    image: meltwater/drone-cache
    when:
      event: push          # don't churn cache on PRs
    settings:
      backend: s3
      archive_format: gzip
      rebuild: true
      cache_key: '{{ checksum "package-lock.json" }}'
      mount:
        - node_modules

The key includes checksum "package-lock.json", so changing the lockfile produces a fresh key automatically — old entries age out of S3 lifecycle rules. On a homelab stack you can point the S3 backend at MinIO (minio-setup).

Choosing between them

Situation	Use
Single agent, trusted workloads	Host-path volumes (simplest, fastest)
Multiple agents / no shared disk	Cache plugin + S3/MinIO backend
Image builds	`cache_from` registry tag or BuildKit local cache

Practical Examples

Example: Node project, single agent, before/after

steps:
  test:
    image: node:22
    environment:
      npm_config_cache: /woodpecker/cache/npm
    commands:
      - time npm ci
      - npm test
    volumes:
      - /var/lib/woodpecker/cache:/woodpecker/cache

Measure the effect directly:

# First run (cold):
real  0m48s   # npm ci downloads everything
# Second run (warm, unchanged lockfile):
real  0m9s    # packages restored from npm cache

Housekeeping cron on the agent host to stop unbounded growth:

#!/bin/sh
# prune cache entries older than 14 days
find /var/lib/woodpecker/cache -type f -atime +14 -delete 2>/dev/null
du -sh /var/lib/woodpecker/cache/* | sort -h

Run it weekly; alert on cache dir exceeding ~10GB using the pattern from ci-monitoring.

Troubleshooting & Common Pitfalls

Problem	Cause	Fix
Tests pass locally, fail in CI after caching added	Stale/corrupted cache entries	Wipe that subdir once (`rm -rf /var/lib/woodpecker/cache/npm`); if it recurs, add cache-busting to the key
Cache grows forever	No pruning, keys never expire	Weekly find/prune cron or S3 lifecycle rules on the plugin backend
Two agents corrupt each other's caches	Shared host path across agents	Per-agent paths, or switch to S3-backed plugin
`npm ci` ignores the cache	npm ci always removes node_modules	Cache the npm cache dir, not node_modules itself (or accept reinstall-from-cache speedup only)
`plugins/docker` cache_from has no effect	Base image changed or previous :cache tag missing	Verify `:cache` tag exists in registry; ensure Dockerfile early layers match
Secrets leaked into cached dirs	Token files written inside cache path	Never mount caches over `$HOME`; scope env vars explicitly
Permission errors on mounted volume	UID mismatch between container user and host dir	`chown` the host dir to the container UID used by your step image

Next Steps / Ops Actions

Apply the same cache discipline to security scanners: security-scanning
Watch queue-depth metrics to prove the improvement: ci-monitoring
Keep matrix legs from multiplying cold-start costs: matrix-builds

Sources & Related

External references consulted:

Change Log

2026-08-26

Initial creation by KB writing session.