fogserv.cloud fogserv.cloud

S3 API Usage - Working with MinIO via aws CLI and mc

Status: Active
Last Updated: 2026-08-26
Category: Cloud - Object Storage Operations
Prerequisites: minio-setup, cloud-storage-concepts
Time: 2-3 hours
Tags: s3, minio, aws-cli, mc, presigned-urls, lifecycle

Summary

Hands-on guide to speaking the S3 API against the fogserv.cloud MinIO deployment: configuring both the aws CLI and MinIO's native mc client, bucket and object operations, access policies, time-limited presigned URLs, and lifecycle rules for automatic expiration. Everything here works identically against AWS S3 later — that portability is why S3 is the lingua franca of storage.

🎯 What You'll Learn

By the end of this article, you'll be able to:

Table of Contents

  1. Why the S3 API Matters
  2. Client Setup: aws CLI
  3. Client Setup: mc
  4. Everyday Object Operations
  5. Policies & Access Control
  6. Presigned URLs
  7. Lifecycle Rules

Context / Why This Matters

MinIO gets the server running; this article is how everything else in the stack talks to it. Restic uses S3 as a backend (backup-to-object-storage), application code can use any S3 SDK, and scripts use aws or mc. Learning the API once pays off across every tier described in storage-backup-strategies.

Implementation / Core Content

Client Setup: aws CLI

sudo apt install awscli        # or pipx install awscli
aws configure                  # then enter credentials interactively

For MinIO, the endpoint must be passed on every call (or set per-profile):

# ~/.aws/credentials
[minio]
aws_access_key_id = YOURACCESSKEY
aws_secret_access_key = your-secret-key

# ~/.aws/config
[profile minio]
region = us-east-1             # MinIO ignores it but the CLI requires one
s3 =
    addressing_style = path    # required: no virtual-host DNS on internal names
endpoint_url = http://minio.internal:9000

Usage:

aws --profile minio s3 ls
aws --profile minio s3 ls s3://backups/

Note: older aws-cli v1 does not support endpoint_url in config; pass --endpoint-url http://minio.internal:9000 explicitly instead.

Client Setup: mc

mc is MinIO's purpose-built admin-and-file client. It understands both S3 semantics and MinIO-specific administration.

curl -o /usr/local/bin/mc https://dl.min.io/client/mc/release/linux-amd64/mc \
  && chmod +x /usr/local/bin/mc

# Register an alias ("remote") once:
mc alias set local http://minio.internal:9000 YOURACCESSKEY your-secret-key
mc alias list

Everything else becomes mc <verb> alias/...:

mc ls local                          # list buckets
mc mb local/photos                   # make bucket
mc rb local/old-bucket               # remove bucket
mc admin info local                  # server health, drives, uptime

Everyday Object Operations

Both clients, side by side:

Task aws cli mc
List objects aws s3api list-objects-v2 --bucket photos mc ls local/photos --recursive
Upload file aws s3 cp f.jpg s3://photos/f.jpg mc cp f.jpg local/photos/
Download aws s3 cp s3://photos/f.jpg . mc cp local/photos/f.jpg .
Mirror dir aws s3 sync ./dir s3://photos/dir mc mirror ./dir local/photos/dir
Delete aws s3 rm s3://photos/f.jpg mc rm local/photos/f.jpg
Disk usage aws s3 ls --recursive --summarize s3://photos | tail -1 mc du local/photos

Tips:

Server-side copy between buckets (no client bandwidth):

aws s3api copy-object \
  --bucket archive --key 2026/report.pdf \
  --copy-source photos/2026/report.pdf

Policies & Access Control

MinIO users are distinct from root credentials. Principle: applications get their own user with the narrowest policy that works.

# Create a user + key pair
mc admin user add local backup-agent 'S3curePassw0rd'

# Attach a built-in policy...
mc admin policy attach local readwrite --user backup-agent
# ...or better: define a custom least-privilege one
cat > backup-policy.json <<'EOF'
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["s3:PutObject", "s3:GetObject", "s3:ListBucket",
               "s3:DeleteObject", "s3:GetBucketLocation"],
    "Resource": [
      "arn:aws:s3:::backups",
      "arn:aws:s3:::backups/*"
    ]
  }]
}
EOF
mc admin policy create local backup-policy backup-policy.json
mc admin policy attach local backup-policy --user backup-agent

Anonymous public read for a specific prefix (e.g., static assets):

mc anonymous set download local/public-assets
mc anonymous set none   local/public-assets   # revoke

Audit what a user can actually do:

mc admin user info local backup-agent

Presigned URLs

Presigned URLs embed authentication in the query string so an unauthenticated client can perform exactly one operation until expiry — ideal for giving a phone app, a colleague, or a CI job temporary upload/download rights without sharing keys.

# Download link, valid 1 hour
mc share download --expire 1h local/photos/vacation.zip

# Upload-capable URL
mc share upload --expire 24h local/incoming/report.pdf
# Client uploads with: curl -T report.pdf '<returned-url>'

# Same via aws cli
aws s3 presign s3://photos/vacation.zip --expires-in 3600

Rules:

Lifecycle Rules

Lifecycle rules age data out without cron jobs. Classic patterns:

# Expire noncurrent versions after 30 days (bucket versioning enabled)
mc ilm rule add local/backups \
  --noncurrent-expire-days 30

# Expire objects under logs/ after 14 days
mc ilm rule add local/appdata --prefix "logs/" --expire-days 14

# Transition to a colder remote tier (tiering via ILM)
mc admin tier add remote minio local COLD http://coldstore:9000 cold-bucket ACCESSKEY SECRETKEY
mc ilm rule add local/backups --transition-days 90 --transition-tier COLD

# Inspect & manage
mc ilm rule ls local/backups
mc ilm rule rm local/backups --id "<rule-id>"

The equivalent JSON document via aws s3api put-bucket-lifecycle-configuration also works — MinIO implements the standard S3 lifecycle schema.

Practical Examples

Example 1: Provision a bucket for a new restic repo

mc mb local/restic-nextcloud
mc version enable local/restic-nextcloud          # ransomware safety net
mc quota set local/restic-nextcloud --size 500GiB
mc admin user add local nc-backup '<generated-password>'
mc admin policy attach local backup-policy --user nc-backup

Then point restic at s3:http://minio.internal:9000/restic-nextcloud using this user's keys (see backup-to-object-storage).

Example 2: One-off secure handoff of a large file

mc cp big-dump.tar.zst local/incoming/
mc share download --expire 24h local/incoming/big-dump.tar.zst
# Send the printed URL over chat; recipient downloads without accounts.
mc rm local/incoming/big-dump.tar.zst   # clean up afterwards

Example 3: Verify lifecycle behavior

mc ilm rule ls local/appdata
mc ilm restore ls local/appdata     # check transitions
# Force-check by inspecting object timestamps vs rules:
mc ls --recursive local/appdata/logs/ | head

Lifecycle evaluation runs periodically (roughly daily), not instantly — don't panic when test objects survive an hour.

Troubleshooting & Common Pitfalls

Problem Cause Fix
SignatureDoesNotMatch with correct-looking keys Clock skew >15 min between client and MinIO Fix NTP on both hosts
NoSuchBucket although listed earlier Wrong alias/endpoint, or typo'd path-style mc alias list; always path-style
Virtual-host style errors (bucket.host unresolvable) Default DNS addressing on internal names Set addressing_style = path
Presigned URL rejected immediately Signed with different endpoint than the one used to fetch Use the identical hostname/port in the URL
Lifecycle rule seems ignored Evaluation is periodic; prefix mismatch Wait up to 24 h; verify exact prefix casing
Backup agent can read other buckets Attached broad readwrite canned policy Custom scoped policy like backup-policy.json
Deleted objects reappear in listing Bucket versioning shows old versions List with versions (mc ls --versions); rely on ILM noncurrent expiry

Next Steps / Ops Actions

Sources & Related Articles

External references consulted:

Related knowledge-base articles:

Change Log

2026-08-26

Choose Theme

Your selection is saved locally.

Neural Cacophony
Aperture v2
Flux v1
Mosaic Chaos
Nexus v1
Nexus Zest
Prism v2
Synapse