This alarm was originally the crude proxy for catching cryptominers, but
Falco now detects that directly via process/network behavior. It was
firing several times a day from Ollama's legitimate sustained CPU use
(subwave-controller driving chat completions), with no way to tell that
apart from a real problem using aggregate CPU % alone. Routed to netdata's
"silent" role so it stays visible on the dashboard but stops paging
Telegram/Cloud email.
No repeat intrusion - investigated all Falco alerts since deployment
(2026-08-08 to 2026-08-16); the only crypto-miner rule hits were the
original test alerts. The daily self-resolving Telegram alerts the user
was seeing came from four known-benign sources: Obsidian's Electron
self-re-exec pattern and s6-init sudoers setup (fires ~daily on container
recreation), ttyd/Synchronet's telnet/pv/busybox-extras BBS tooling, and
ollama's nvidia-ctk ldconfig-refresh hook (memfd_create by design). Added
scoped exceptions for each via Falco's own rule extension points, same
pattern as the earlier pg_isready/gitea fixes.
Also fixed netdata's used_swap alarm, which recalculated from the
instantaneous raw sample every 10s with only a 30s notification debounce -
a brief swap spike was enough to page both Telegram and (via Netdata
Cloud) email. Widened the up-delay to 5m so only sustained swap pressure
notifies.
Verified end-to-end via a fresh fake-xmrig test after redeploying Falco
and reloading netdata: CRITICAL alarm fired and Telegram delivery
succeeded.
- 'Drop and execute new binary in container' fired on every git push/pull
to gitea over SSH (gitea's own binary re-executing itself for git-shell
hooks looks like container drift). Scoped exception via
known_drop_and_execute_activities to proc.name=gitea on that specific
image, not a blanket container whitelist, so other unexpected binaries
in that container still get caught.
- netdata alarm text was a static generic blurb requiring a manual
'docker logs falco' every time to find out what actually happened.
Now interpolates ${label:rule_name}/${label:priority} (exposed by
Falco's Prometheus metric) so the Telegram message names the specific
rule directly. Note: literal double-quotes in the info/summary text
broke netdata's alarm-notify.sh (silent delivery failure, exit 1) -
avoided.
- Added netdata/health.d and netdata/go.d to the repo for documentation;
netdata does not auto-deploy these from git, same as gitea's app.ini -
copy to /srv/netdata/config/ manually and restart the container.
Two cryptominer incursions (inbox-zero, then gitea) went unnoticed until
found manually — there was no alerting on sustained host CPU. Wired
netdata's existing 10min_cpu_usage alarm to notify via the shared Telegram
bot (config is host-only, not committed). Also narrowed the dashboard port
from 0.0.0.0 to the host's LAN IP to reduce exposure.
poprhythm