# Netdata - Docker Host Monitoring

Comprehensive real-time monitoring for Docker host and containers with alerting capabilities.

## Features

- **Container Monitoring**: Auto-detects and monitors all Docker containers
  - CPU, memory, network, disk I/O per container
  - Container state monitoring (running, stopped, unhealthy)
  - Pre-configured alerts for resource usage

- **Host Monitoring**: Full system metrics
  - CPU, memory, disk, network usage
  - Process monitoring
  - System health checks

- **Built-in Alerts**: Pre-configured alarms for:
  - Container CPU and memory usage
  - Container health status
  - System resource exhaustion
  - Disk space usage

## Access

Once deployed, access Netdata at: `http://your-host-ip:19999`

## Notifications: Telegram (configured)

Netdata's notification config lives at `/srv/netdata/config/health_alarm_notify.conf`
on the host (bind-mounted into the container at `/etc/netdata/health_alarm_notify.conf`).
This file is **host-only and not tracked in git** since it holds the bot token —
same pattern as Authelia's config. It's populated from the shared homelab
Telegram bot credentials in `.credentials` (`TELEGRAM_BOT_TOKEN` /
`TELEGRAM_CHAT_ID` — same bot used by Watchtower and Uptime Kuma):

```conf
SEND_TELEGRAM="YES"
TELEGRAM_BOT_TOKEN="<TELEGRAM_BOT_TOKEN from .credentials>"
DEFAULT_RECIPIENT_TELEGRAM="<TELEGRAM_CHAT_ID from .credentials>"
```

Netdata reads this file fresh each time it sends a notification, so no
container restart is needed after editing it — just re-run the test below.

To recreate this file (e.g. after a host rebuild), source `.credentials` and:
```bash
set -a; source .credentials; set +a
cat > /srv/netdata/config/health_alarm_notify.conf <<EOF
SEND_TELEGRAM="YES"
TELEGRAM_BOT_TOKEN="${TELEGRAM_BOT_TOKEN}"
DEFAULT_RECIPIENT_TELEGRAM="${TELEGRAM_CHAT_ID}"
EOF
```

Test the notification pipeline any time with:
```bash
docker exec netdata /usr/libexec/netdata/plugins.d/alarm-notify.sh test
```
(An email delivery error in the output is expected/harmless — Telegram is
the only channel actually configured.)

### Host CPU/load alerting (cryptominer detection)

Netdata ships a built-in alarm, `10min_cpu_usage` (in
`/usr/lib/netdata/conf.d/health.d/cpu.conf`), that watches the 10-minute
rolling average of total host CPU utilization and warns at >85%, critical at
>95%. It's core-count-agnostic (percentage-based) so it needs no tuning per
host. This is now wired to notify via Telegram above — it's the primary
signal for "something is pegging the CPU" (e.g. a cryptominer), and doesn't
require any custom `health.d` override.

Related stock alarms `load_average_1/5/15` (in `load.conf`) exist but are
intentionally routed `to: silent` upstream — they're informational only,
not part of the notified alert path.

## Alert Configuration

Netdata comes with sensible defaults, but you can customize:

- **Alert thresholds**: Edit individual alarm files in `/etc/netdata/health.d/`
- **Notification frequency**: Configure repeat intervals in `health_alarm_notify.conf`
- **Alert filtering**: Set up role-based notifications

## Monitoring Capabilities

- Real-time metrics with 1-second granularity
- Automatic anomaly detection
- Zero-configuration auto-detection of containers
- Historical data retention
- Beautiful web dashboard with interactive charts

## Resources

- Official docs: https://learn.netdata.cloud/
- Alert configuration: https://learn.netdata.cloud/docs/alerting/
- Notification setup: https://learn.netdata.cloud/docs/alerting/notifications/

## Notes

- The container requires privileged access and specific capabilities to monitor the host system
- Docker socket is mounted read-only for security
- Data persists across container restarts via Docker volumes