Cloud Storage Concepts - Understanding Cloud Storage Before Building It
Status: Active
Last Updated: 2026-08-14
Category: Cloud - Phase 1: File Storage & Sync
Prerequisites: None
Time: 1-2 hours
Tags: cloud, storage, concepts, sync-vs-backup, privacy, self-hosted, data-sovereignty
Summary
Before installing Nextcloud or MinIO, understand what "cloud storage" actually means: the difference between file storage and object storage, why sync is not backup, what you give up when you use commercial clouds, and why self-hosting gives you sovereignty over your own data. This conceptual foundation makes every hands-on lesson in this course make sense.
🎯 What You'll Learn
By the end of this article, you'll be able to:
- ✅ Explain the difference between file storage, block storage, and object storage
- ✅ Articulate why "sync" and "backup" are fundamentally different things
- ✅ List the real costs of commercial cloud services (money, privacy, lock-in)
- ✅ Decide which workloads belong on self-hosted infrastructure vs. commercial cloud
- ✅ Understand the threat model that drives encryption and replication decisions
- ✅ Map your personal or team workflows onto the self-hosted stack (Nextcloud + MinIO + Restic)
What is "The Cloud", Really?
Strip away the marketing and the cloud is simple:
"The cloud" = someone else's computer, rented by the hour.
Every commercial cloud product — Dropbox, Google Drive, iCloud, OneDrive — is a web application running on servers in a datacenter, with three components:
| Component | What it does | Example |
|---|---|---|
| Storage layer | Disks holding your bytes (often object storage underneath) | Amazon S3 behind Dropbox's early days |
| Application layer | The logic: sharing links, versioning, search | Dropbox's sync engine |
| Access layer | Web UI, desktop/mobile clients, APIs | The Dropbox folder on your laptop |
Here's the key insight of this entire course: you can run all three layers yourself.
Commercial cloud:
Your files → their servers → their rules → their prices → forever
Self-hosted cloud:
Your files → your server → your rules → one-time hardware cost → your responsibility
That last item — your responsibility — is the honest price of sovereignty. When you host your own cloud, you also inherit the jobs of backup operator, security engineer, and capacity planner. This course teaches those jobs alongside the software.
Why Self-Host?
People self-host for a mix of reasons. Rank these for yourself before choosing architecture:
- Data sovereignty — your documents, photos, and credentials live on hardware you physically control. No terms-of-service change can read, scan, or delete them.
- Cost predictability — 4TB on Google One is ~$20/month = $240/year forever. A 4TB NAS drive is ~$90 once.
- No artificial limits — no upload caps, throttled download speeds, or "file too large" errors.
- Privacy — no content scanning, no ad-profile building, no training-data ambiguity.
- Learning — running your own stack is the best infrastructure education there is (which is partly why this KB exists).
- Independence from vendors — a company killing a product line can't strand your data when the export button is on your own server.
And the honest counterweights:
- You are the SLA — if your server dies at 2 AM, nobody pages you but you.
- You handle off-site — a house fire destroys local disks; commercial clouds are inherently off-site.
- Security is yours — an unpatched Nextcloud exposed to the internet is worse than any commercial option.
The rest of this course exists to neutralize those three risks with backups (Phase 4), hardening (kb/security/), and monitoring (kb/observability/).
File Storage vs Object Storage vs Block Storage
Three storage models dominate. Knowing which problem each solves prevents years of architectural mistakes.
File Storage (NAS-style)
Data lives in a hierarchy: directories containing files containing bytes.
/photos/2026/vacation/beach-day.jpg
/documents/contracts/lease.pdf
- Accessed via POSIX filesystem semantics or network protocols: NFS, SMB/CIFS, or WebDAV
- Files have metadata: owner, permissions, timestamps
- Great for: documents, media libraries, home directories — anything a human browses
- Self-hosted examples: Nextcloud (file store + sync), a Samba share, NFS exports
Block Storage (Disk-style)
Raw fixed-size blocks presented as if they were a physical disk.
- The OS puts a filesystem on top (
mkfs.ext4 /dev/sdb) - Great for: VM disks, databases (PostgreSQL data directory), anything needing random I/O
- You never "browse" block storage directly
- Self-hosted examples: a Proxmox VM disk, an iSCSI LUN, a Kubernetes PV backed by Ceph RBD
Object Storage (S3-style)
A completely flat namespace: buckets contain objects, and objects are addressed by keys.
Bucket: my-app-uploads
Object key: images/2026/08/beach-day.jpg ← just a string, not a path!
Object = data + metadata + globally unique ID
Critical differences from file storage:
| Property | File storage | Object storage |
|---|---|---|
| Addressing | Path in a tree | Flat key string |
| Access protocol | POSIX/NFS/SMB | HTTP + S3 API |
| Partial writes | Yes (edit bytes in place) | No (objects are immutable — replace whole object) |
| Listing cost | Cheap (ls) |
Expensive at scale (paginated LIST) |
| Scale limit | Millions of files practically | Billions of objects by design |
| Rename | Cheap metadata op | Copy + delete (expensive) |
| Perfect for | Human-browsable data | App data, backups, images, logs, static assets |
What Happens conceptually when an app uploads to object storage:
1. Client computes: PUT https://s3.example.com/my-bucket/reports/q3.pdf
2. Request signed with access key (HMAC signature)
3. Server validates signature → stores object durably (multiple disks/nodes)
4. Returns HTTP 200 ETag="d41d8cd9..."
5. Later reads use GET on the same URL — any client, anywhere
Objects are immutable: to "change" q3.pdf you upload a new version of it. That immutability is exactly why object storage is fantastic for backups (old versions can't be silently corrupted) and why databases don't live on it directly.
The Rule of Thumb
Humans browsing files? → File storage (Nextcloud)
Applications storing data? → Object storage (MinIO)
Virtual machines / databases? → Block storage (ZFS/LVM/iSCSI)
Most real systems combine all three. In this course's final architecture: Nextcloud stores its data directory on disk, uses MinIO (object storage) as its external storage backend for big blobs, and Restic backs both up into another MinIO bucket.
Sync vs Backup: The Most Expensive Confusion in IT
This section may save your data someday. Read it twice.
Sync Mirrors Mistakes
Sync (Dropbox, Nextcloud desktop client, Syncthing) keeps N devices consistent with each other:
Laptop: thesis-final.docx ──sync──► Server copy ──sync──► Phone
But watch what happens during ransomware, a bad rm -rf, or a corrupted save:
Day 100, 09:00 Laptop has thesis.docx ✓ synced everywhere
Day 100, 14:32 Ransomware encrypts thesis.docx ✗ encrypted version synced everywhere
Day 100, 14:35 All devices + server now hold encrypted garbage
Sync faithfully propagated the disaster. Sync replicates state; it does not preserve history. If your only copy strategy is sync, you have zero protection against deletion, corruption, or encryption — because every copy becomes wrong simultaneously.
Backup Preserves History
A backup system (Restic, Borg) takes periodic point-in-time snapshots that are:
- Immutable-ish — old snapshots aren't modified when source files change
- Retention-managed — you keep "daily for 7 days, weekly for 8 weeks, monthly for 2 years"
- Restorable to any point — recover last Tuesday's version of the file
Backup timeline:
Snapshot Mon ── Snapshot Tue ── Snapshot Wed ── Snapshot Thu
│
Ransomware hits Wed 15:00 ◄──────────┘
Restore Wednesday-morning snapshot: lose 15 hours, not everything.
The Verdict Table
| Property | Sync | Backup |
|---|---|---|
| Protects against disk failure | ✅ (second copy) | ✅ |
| Protects against accidental delete | ❌ (deletes propagate) | ✅ (older snapshots retain) |
| Protects against ransomware | ❌ (encryption propagates) | ⚠️ partially (retention window) |
| Point-in-time recovery | ❌ | ✅ |
| Convenient daily file access | ✅ | ❌ (restore needed) |
You need both. Nextcloud for working files; Restic snapshots of Nextcloud's data directory underneath it. Phase 4 wires this together properly.
Privacy Considerations & Threat Modeling
"Privacy" is meaningless without naming who you're hiding data from. Build a small threat model:
| Threat actor | Commercial cloud risk | Self-hosted risk | Mitigation |
|---|---|---|---|
| Ad platforms / content scanning | High — ToS grants broad rights | Low — data never leaves | Self-host sensitive docs |
| Platform employees | Possible insider access | Only you (and your SSH keys) | Self-host + encryption |
| Hackers over the internet | Vendor's job (usually good) | Your job | Hardening: kb/security/ssh-security-hardening, fail2ban, TLS |
| Governments w/ legal process | Data handed over per jurisdiction | Warrants to your door | Depends on your threat model |
| Hardware failure | Vendor's redundancy | Your RAID/backups | Phase 4 + ZFS/RAID (lesson 20) |
| House fire / theft | ✅ inherently off-site | ❌ unless you replicate off-site | Off-site restic targets (lesson 18) |
Two design consequences follow, and this course enforces both:
- Encrypt everything that leaves your building. Restic repositories are encrypted client-side by design; MinIO supports SSE (server-side encryption); TLS protects data in transit.
- Assume the public internet is hostile. Anything exposed (Nextcloud web UI, MinIO S3 API, Harbor) gets TLS, strong auth, fail2ban, and ideally sits behind a reverse proxy with rate limiting.
Commercial vs Self-Hosted: An Honest Comparison
Cost Model (4 TB, 10 users, 5 years)
Commercial (Google Workspace-tier):
$20/user/month × 10 users × 60 months ≈ $12,000 (and counting)
Self-hosted:
Server (16GB RAM, 4× 4TB drives): ~$1,200 once
Electricity (~80W avg): ~$350 over 5 years
Occasional drive replacement: ~$300 over 5 years
Off-site VPS/storage (B2-class 1TB): ~$300–600 over 5 years
─────────────────────────────────────────────────────
Total ≈ $2,150–2,450 (~80% cheaper, and hardware stays yours)
Break-even typically lands between 12–24 months for small teams. But money isn't the whole ledger:
| Factor | Commercial | Self-hosted |
|---|---|---|
| Uptime accountability | Contractual SLA | Your monitoring + discipline |
| Admin effort | Zero | ~2–5 hrs/month maintenance |
| Feature velocity | Vendor ships updates | You apply updates |
| Data exit | Export tools, friction-free? varies | It's already yours |
| Compliance surface | Vendor certifications (SOC2 etc.) | Yours to build |
Honest recommendation used throughout this KB: hybrid. Keep truly critical archival copies replicated to cheap commercial cold storage (Backblaze B2, Wasabi) as your "offsite" leg — you get sovereignty day-to-day plus vendor-grade durability for disasters. Lesson 18 implements exactly this pattern with Restic.
Mapping Workloads to the Course Stack
Where does each kind of data go? Use this table as your north star; every later lesson builds one box:
┌─────────────────────────────────────────────────────────────┐
│ YOUR SELF-HOSTED CLOUD │
│ │
│ Working files & shares → Nextcloud (lessons 2–4) │
│ App data / blobs / media → MinIO (lessons 5–9) │
│ Container images → Harbor (lessons 10–14) │
│ Snapshots of ALL of it → Restic (lessons 15–19) │
│ Scale & polish → HA/CDN/tuning (lessons 20–24) │
└─────────────────────────────────────────────────────────────┘
Concrete examples:
| Data type | Where it lives | Why |
|---|---|---|
| Family photo library | Nextcloud (files) | Humans browse it; phone auto-upload |
| Team documents | Nextcloud + OnlyOffice | Collaborative editing (lesson 4) |
| Web app user uploads | MinIO bucket via S3 API | Apps talk S3 natively; scales |
| Backup repository | MinIO bucket + off-site copy | Immutable, encryppable, scriptable |
| Docker images for CI | Harbor | Private, scanned, fast pulls |
| VM disks | Block storage (ZFS/Proxmox) | Random I/O needs |
Vocabulary Check
Terms used constantly in the rest of the course:
- Bucket — top-level named container for objects in S3-compatible storage
- Object key — the string address of an object inside a bucket
- S3 API — de-facto standard HTTP API for object storage; MinIO speaks it exactly
- WebDAV — HTTP-based file-access protocol; how many clients mount Nextcloud
- Retention policy — rules for how long snapshots/versions are kept
- RPO / RTO — Recovery Point Objective (how much data loss is tolerable) / Recovery Time Objective (how long recovery may take); covered deeply in lesson 15
- Idempotent — safe to run repeatedly with same result; good backup scripts are idempotent
Common Gotchas
Gotcha 1: Treating the NAS as backup. RAID protects against a disk failing, not against deletion/ransomware/fire. RAID ≠ backup; snapshots ≠ backup until retention + off-site exist.
Gotcha 2: Choosing object storage for human-browsed files. Listing a million-object bucket is slow and awkward. If humans browse it daily, it belongs in Nextcloud/file storage.
Gotcha 3: Ignoring egress reality. Your home upload bandwidth (often 20–40 Mbps) is the ceiling for remote sync speed. Design expectations accordingly; seed initial data locally.
Gotcha 4: No name-spacing discipline.
Buckets named backup, backup2, new-backups become unmaintainable. Adopt <purpose>-<env> naming early (restic-prod, media-archive).
Gotcha 5: Assuming self-hosting is free. Time is the currency. Budget monthly maintenance hours like you budgeted the hardware.
Practical Exercises
- Inventory your data. List every category of file you care about (docs, photos, code, media). For each, note: size, who accesses it, from what devices, how often.
- Classify each into file/object/block using the rule of thumb above.
- Audit your current sync setup. Do you rely on Dropbox/Drive alone? Identify one file deleted >30 days ago you couldn't recover today — that's your gap.
- Sketch your 3-2-1 plan on paper: 3 copies, 2 media, 1 off-site. Don't implement yet — lessons 15–19 will.
- Check your upload bandwidth (
speedtest-cli). Compute how long one 50GB initial backup would take to reach off-site storage.
Resource Requirements
This lesson is conceptual — nothing to run. Planning numbers for what follows:
| Component | RAM | CPU | Disk | Notes |
|---|---|---|---|---|
| Nextcloud (personal) | 1–2 GB | 2 | 100 GB+ | Lessons 2–3 |
| Nextcloud (+Office) | +2 GB | +2 | — | Lesson 4 |
| MinIO standalone | 1–2 GB | 2 | As needed | Lessons 5–9 |
| Harbor | 4 GB | 4 | 500 GB+ | Lessons 10–14 |
| Restic | 256 MB–2 GB | 1 | Minimal (client) | Lessons 15–19 |
🔗 Related
- Next lesson: nextcloud-setup — install your personal cloud
- object-storage-concepts — deep dive into the S3 model
- backup-strategies — the 3-2-1 rule done right
- kb/basics/linux-fundamentals — CLI prerequisites
- kb/containers/docker-basics — Docker prerequisites for every install in this course
- kb/security/password-management — credential hygiene for services you're about to expose
- kb/security/disk-encryption-luks — encrypting the underlying storage layer
Change Log
- 2026-08-14 — Initial publication as part of the cloud course build-out.