falco/netdata: tune out obsidian/ttyd/ollama noise, fix swap alarm flapping

No repeat intrusion - investigated all Falco alerts since deployment
(2026-08-08 to 2026-08-16); the only crypto-miner rule hits were the
original test alerts. The daily self-resolving Telegram alerts the user
was seeing came from four known-benign sources: Obsidian's Electron
self-re-exec pattern and s6-init sudoers setup (fires ~daily on container
recreation), ttyd/Synchronet's telnet/pv/busybox-extras BBS tooling, and
ollama's nvidia-ctk ldconfig-refresh hook (memfd_create by design). Added
scoped exceptions for each via Falco's own rule extension points, same
pattern as the earlier pg_isready/gitea fixes.

Also fixed netdata's used_swap alarm, which recalculated from the
instantaneous raw sample every 10s with only a 30s notification debounce -
a brief swap spike was enough to page both Telegram and (via Netdata
Cloud) email. Widened the up-delay to 5m so only sustained swap pressure
notifies.

Verified end-to-end via a fresh fake-xmrig test after redeploying Falco
and reloading netdata: CRITICAL alarm fired and Telegram delivery
succeeded.
This commit is contained in:
2026-08-16 14:08:05 +00:00
parent 9c4a54c0cd
commit 4d9d15417a
2 changed files with 65 additions and 14 deletions
+41 -14
View File
@@ -5,20 +5,47 @@
# else. Add entries here as new noisy defaults turn up - don't let this list
# grow into blinding Falco to anything actually meaningful.
# "Sensitive file opened for reading by non-trusted program" fired every
# ~10s from inbox-zero-db's pg_isready healthcheck touching /etc/shadow -
# NSS/libc user-lookup behavior during process init, not credential
# harvesting. Scoped to just this binary, not a blanket rule disable.
# "Sensitive file opened for reading by non-trusted program" false-positives:
# - pg_isready: fired every ~10s from inbox-zero-db's healthcheck touching
# /etc/shadow - NSS/libc user-lookup behavior during process init, not
# credential harvesting.
# - lscr.io/linuxserver/obsidian: fires once per container start from
# `sed -i s/CORRUPT_FILE/NOPASSWD/g /etc/sudoers` - linuxserver.io's
# own s6-init NOPASSWD setup, same class of init-time file touch.
- macro: user_known_read_sensitive_files_activities
condition: (proc.name = pg_isready)
condition: >
(proc.name = pg_isready)
or (container.image.repository = "lscr.io/linuxserver/obsidian" and proc.name = sed)
# "Drop and execute new binary in container" fires on every git push/pull
# to gitea over SSH - gitea's own binary re-executes itself for git-shell
# commands (serv/hook/pre-receive/post-receive), which looks identical to
# drift-detection's "new binary written to the writable layer" pattern.
# Scoped to proc.name=gitea specifically (not the whole container via
# known_drop_and_execute_containers) so this container - the one that was
# actually compromised - still gets checked for any OTHER unexpected
# binary, just not its own expected self-invocation.
# "Fileless execution via memfd_create" fires from nvidia-ctk's
# ldconfig-refresh hook, which runs via memfd_create by design every time
# an NVIDIA-GPU container (e.g. ollama) starts - normal container-toolkit
# behavior, not defense evasion. Scoped to the hook's own parent process
# names rather than the ollama/ollama image, so drift detection stays live
# for anything else that image does.
- macro: known_memfd_execution_processes
condition: (proc.pname in (nvidia-containe, nvidia-ctk))
# "Drop and execute new binary in container" (output text: "Executing binary
# not part of base image") false-positives on three known-benign sources,
# all confirmed via docker logs falco (2026-08-08 to 2026-08-16):
# - gitea: re-executes itself for git-shell commands (serv/hook/pre-receive/
# post-receive) over SSH, which looks identical to a dropped binary.
# Scoped to proc.name=gitea, not the whole container, so this
# previously-compromised container still gets checked for any OTHER
# unexpected binary.
# - lscr.io/linuxserver/obsidian: Electron's normal /proc/self/exe
# self-re-exec pattern (zygote/renderer/utility processes), fires ~6x
# every time the container is recreated (~daily, likely watchtower).
# Scoped to the whole image since this is normal Electron startup
# behavior end to end, not one specific binary.
# - tsl0922/ttyd (Synchronet BBS web terminal): telnet/pv/busybox-extras
# installed via apk at container start and then used for the BBS's
# normal telnet workflow. Scoped to those three binaries, not the image,
# since this is a shell-access container where an unexpected binary is
# more worth knowing about.
- macro: known_drop_and_execute_activities
condition: (container.image.repository = "docker.gitea.com/gitea" and proc.name = gitea)
condition: >
(container.image.repository = "docker.gitea.com/gitea" and proc.name = gitea)
or (container.image.repository = "lscr.io/linuxserver/obsidian")
or (container.image.repository = "tsl0922/ttyd" and proc.name in (telnet, pv, busybox-extras))
+24
View File
@@ -0,0 +1,24 @@
# Override of netdata's stock used_swap alarm (health.d/swap.conf). The
# stock template recalculates $used/$free from the *instantaneous* raw
# sample every 10s with only a 30s notification debounce, so a swap spike
# lasting a couple of minutes was enough to page. Only the delay line
# changes here (up 30s -> up 5m) - the rest is copied verbatim from the
# stock template since netdata health.d overrides replace the whole
# template, not just one line. This requires sustained swap pressure
# (5+ continuous minutes above threshold) before notifying, instead of
# firing on every brief spike.
template: used_swap
on: mem.swap
class: Utilization
type: System
component: Memory
host labels: _os=linux freebsd
calc: (($used + $free) > 0) ? ($used * 100 / ($used + $free)) : 0
units: %
every: 10s
warn: $this > (($status >= $WARNING) ? (80) : (90))
crit: $this > (($status == $CRITICAL) ? (90) : (98))
delay: up 5m down 15m multiplier 1.5 max 1h
summary: System swap memory utilization
info: Swap memory utilization
to: sysadmin