fogserv.cloud fogserv.cloud

Netdata Basics - Real-Time System Monitoring

Status: Active
Last Updated: 2026-08-14
Category: Observability - Phase 1: Basic Monitoring
Prerequisites: linux-fundamentals
Time: 1-2 hours
Tags: netdata, metrics, real-time, system-monitoring, docker, alerts, zero-config

Summary

Netdata gives you per-second visibility into everything a Linux box is doing — CPU, memory, disks, network, containers, and every application it can auto-detect — with literally zero configuration and ~150 MB of RAM. This guide installs Netdata two ways (kickstart script and Docker), tours the dashboard, configures sensible alert notifications, and shows how to stream metrics between nodes.

🎯 What You'll Learn

By the end of this article, you'll be able to:

Table of Contents

  1. Why Netdata
  2. Installation
  3. The Dashboard Tour
  4. What Zero-Config Detects
  5. Container Monitoring
  6. Alert Configuration
  7. Streaming Between Nodes
  8. Netdata or Prometheus?
  9. Troubleshooting
  10. Key Takeaways
  11. Next Steps

Why Netdata

Every monitoring tool answers: how much detail, how fast, at what setup cost?

Tool Resolution Setup cost RAM
top/htop 1–3 s Zero Zero — but only while you watch
Uptime Kuma Per check Minutes ~100 MB
Netdata Per second ~10 minutes ~100–200 MB
Prometheus stack 15–60 s Hours 1–4 GB+

Netdata's niche is the top-right corner: per-second resolution with near-zero setup. It's the tool you install first on any new machine because within five minutes you can answer "what is this server actually doing right now?"

Its superpower is context during incidents: when an alert fires elsewhere, Netdata's dashboard shows you the exact second things went sideways and every metric that moved with it.

Installation

Option A: Kickstart Script (bare metal / VM)

The official one-liner installs a static build that auto-updates:

wget -O /tmp/kickstart.sh https://get.netdata.cloud/kickstart.sh
sh /tmp/kickstart.sh --stable-channel

What happens:

1. Detects your distro and architecture
2. Downloads the static Netdata binary bundle
3. Installs to /opt/netdata
4. Creates systemd service "netdata"
5. Starts Netdata listening on port 19999
6. Enables auto-updates via netdata-updater.timer

Verify:

sudo systemctl status netdata      # active (running)
curl -I http://localhost:19999     # HTTP 200

Open http://<server-ip>:19999 — the dashboard appears instantly, already full of charts.

🔒 Bind Netdata to localhost or keep it behind the firewall/VPN; the dashboard exposes detailed system internals. For remote access use an SSH tunnel (ssh -L 19999:localhost:19999 user@host) or reverse proxy with auth (kb/security/ssh-security-hardening).

Option B: Docker

services:
  netdata:
    image: netdata/netdata:stable
    container_name: netdata
    restart: unless-stopped
    pid: host                 # see all processes, not just container's
    cap_add:
      - SYS_PTRACE            # process-level metrics
      - SYS_ADMIN             # cgroup/network details (optional)
    security_opt:
      - apparmor:unconfined
    ports:
      - "127.0.0.1:19999:19999"   # localhost only!
    volumes:
      - ./config:/etc/netdata
      - ./lib:/var/lib/netdata
      - ./cache:/var/cache/netdata
      - /:/host/root:ro,rslave
      - /etc/passwd:/host/etc/passwd:ro
      - /etc/group:/host/etc/group:ro
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - DOCKER_HOST=unix:///var/run/docker.sock

This is the standard privileged-monitoring pattern: mount /proc, /sys, and the host root read-only under /host so Netdata sees the host's reality rather than the container's isolated view.

docker compose up -d
docker logs netdata --tail 20   # look for "ready" without error lines

First Configuration (both options)

Edit health_alarm_notify.conf early so test alerts have somewhere to go:

# bare metal
sudo nano /opt/netdata/etc/netdata/health_alarm_notify.conf
# docker
nano ~/apps/netdata/config/health_alarm_notify.conf

Set at minimum:

DEFAULT_RECIPIENT_EMAIL="ops@yourdomain.com"
role_recipients_email[sysadmin]="ops@yourdomain.com"

Then restart and force one notification round so sender credentials get built:

sudo systemctl restart netdata
sudo su -s /bin/bash netdata
/opt/netdata/usr/libexec/netdata/plugins.d/alarm-notify.sh test

You should receive three test notifications (raise/clear/warning). If nothing arrives, fix delivery now, before you rely on it at 2 AM.

The Dashboard Tour

Open the dashboard. It's overwhelming by design — hundreds of charts, all updating every second. Here's the reading order that matters:

1. CPU Section

2. Memory Section

3. Disk Section

4. Network Section

5. Apps & Containers Sections

Per-process-group and per-container breakdowns of CPU/RAM/disk — this answers "which container ate all the memory?" without running anything.

Reading Charts Like a Pro

Anomaly checklist for any suspicious graph:
1. When exactly did it change?         → hover for timestamp
2. What else changed at that second?   → scan other sections at same time
3. Is there a daily/weekly pattern?    → normal cycle vs true anomaly
4. Did it recover?                     → incident vs ongoing bleed

Correlating across sections at the same timestamp is Netdata's killer debugging move: disk.util spikes + iowait spikes + postgres writes = your backup job, probably.

What Zero-Config Detects

Out of the box, with no configuration files touched, Netdata auto-detects and starts collecting from:

Category Examples
System CPU, memory, swap, disks, filesystems, network, kernel
Containers All Docker/containerd/Podman cgroups
Databases PostgreSQL, MySQL/MariaDB, Redis, MongoDB
Web Nginx, Apache, Traefik, Caddy
Services systemd units, supervisord
Hardware SMART disk health, sensors, UPS (nut)
Apps Any app exposing StatsD or Prometheus endpoints

Detection works via local ports and sockets — if Postgres listens locally, Netdata finds it and starts charting connections, transactions, cache hits within a minute of first contact.

Check what yours found: scroll the dashboard sidebar, or:

sudo /opt/netdata/usr/libexec/netdata/plugins.d/go.d.plugin --list

If something you run isn't detected, its collector may need credentials (e.g., a Postgres monitoring user) — each collector has a config in /etc/netdata/go.d/.

Container Monitoring

With the Docker socket mounted (or bare-metal install), Netdata shows every container as a first-class citizen:

Practical checks to run once after install:

  1. Sort containers by memory — does the biggest match expectations?
  2. Look for throttling on CPU-limited containers (they're slower than they should be)
  3. Find any container with growing RSS over hours — future OOM candidate

This complements Uptime Kuma nicely: Kuma knows the container stopped; Netdata shows you the memory climb that caused it.

Alert Configuration

Netdata ships with hundreds of pre-written alarms — disk space, RAM pressure, TCP retransmits, app-specific health — tuned by the community. Your job is only to (a) route them and (b) silence the noise.

How Alarms Work

Each alarm definition (in .conf files under health.d/) follows this shape:

 template: 10min_ram_usage
       on: mem.available
     calc: ($avail_now) * 100 / ($avail_total)
     warn: $this <= 10
     crit: $this <= 5
     unit: %

Read: evaluate available memory every 10 s; warn below 10%, critical below 5%. You can override any value by copying the line into health.d/*.conf.edit and changing numbers — stock definitions stay intact across upgrades.

Route Notifications

In health_alarm_notify.conf (from Installation):

# Email
EMAIL_SENDER="monitor@yourdomain.com"
DEFAULT_RECIPIENT_EMAIL="ops@yourdomain.com"

# Discord
SEND_DISCORD="YES"
DISCORD_WEBHOOK_URL="https://discord.com/api/webhooks/..."
DEFAULT_RECIPIENT_DISCORD="alerts"

Restart, then verify with the alarm-notify.sh test command shown earlier.

Fighting Alert Fatigue

Default alarms are chatty on small systems. Tame them:

  1. Silence noisy non-critical alarms globally:
# health_alarm_notify.conf
DEFAULT_RECIPIENT_EMAIL="ops@yourdomain.com|critical"   # email only criticals
  1. Mute specific alarms by creating e.g. health.d/cpu.conf:
 # 10min_cpu_usage silenced: this box compiles things sometimes
 template: 10min_cpu_usage
           on: system.cpu
        calc: $this
        warn: $this > 999999

(Setting an impossible threshold = effective mute, while keeping the alarm visible.)

  1. Review monthly: like all alerts (why-monitor), any alarm that fired without requiring action gets demoted or muted.

Streaming Between Nodes

For 2–10 servers, Netdata can stream metrics from child machines into a parent — one dashboard, one place to check.

On each child, create stream.conf:

[stream]
    enabled = yes
    destination = parent-host:19999
    api key = 11111111-2222-3333-4444-555555555555

On the parent, same file:

[11111111-2222-3333-4444-555555555555]
    enabled = yes
    default history = 3600

Restart both. Children appear in the parent's dashboard node list. Benefits: children need almost no RAM (metrics buffered briefly then forwarded), and if a child dies, the parent still shows its last-known data — useful for post-mortems on machines that died completely.

Beyond ~10 nodes, graduate to Prometheus + Grafana (Phase 2–3) — see next section.

Netdata or Prometheus?

Both, at different stages:

Question Answer
One to five servers, want instant insight Netdata alone — done
Need year-long history and capacity trends Netdata's DB is short-term; add Prometheus
Many targets, custom apps, team dashboards Prometheus + Grafana (Phase 2–3)
Want both? They coexist happily Netdata exposes /api/v1/allmetrics?format=prometheus — scrape it as one target

That last row is the pragmatic path: keep Netdata for real-time forensics, let Prometheus pull its metrics for long-term storage, and Grafana graphs the history. Best of both with minimal duplication.

Troubleshooting

Dashboard loads but most charts are empty

High CPU from netdata itself

Notifications not sending

Can't see a specific container

Port 19999 occupied

Change [web] default port = 19998 in netdata.conf, restart, adjust any tunnel/proxy.

Key Takeaways

  1. Netdata = per-second, zero-config visibility — install it on every box as step one.
  2. Read dashboards in order: CPU (incl. iowait) → memory (available!) → disk (util + await) → network.
  3. Auto-detection covers databases, web servers, and containers out of the box.
  4. Container views include per-cgroup limits — catch OOM candidates early.
  5. Hundreds of alarms ship enabled; routing + selective muting is your whole alert job.
  6. Stream to a parent for multi-node dashboards; scrape into Prometheus when you need long history.
  7. Test notification delivery immediately — untested alerts don't exist.

🔗 Related

Choose Theme

Your selection is saved locally.

Neural Cacophony
Aperture v2
Flux v1
Mosaic Chaos
Nexus v1
Nexus Zest
Prism v2
Synapse