falco/netdata: tune out obsidian/ttyd/ollama noise, fix swap alarm flapping
No repeat intrusion - investigated all Falco alerts since deployment (2026-08-08 to 2026-08-16); the only crypto-miner rule hits were the original test alerts. The daily self-resolving Telegram alerts the user was seeing came from four known-benign sources: Obsidian's Electron self-re-exec pattern and s6-init sudoers setup (fires ~daily on container recreation), ttyd/Synchronet's telnet/pv/busybox-extras BBS tooling, and ollama's nvidia-ctk ldconfig-refresh hook (memfd_create by design). Added scoped exceptions for each via Falco's own rule extension points, same pattern as the earlier pg_isready/gitea fixes. Also fixed netdata's used_swap alarm, which recalculated from the instantaneous raw sample every 10s with only a 30s notification debounce - a brief swap spike was enough to page both Telegram and (via Netdata Cloud) email. Widened the up-delay to 5m so only sustained swap pressure notifies. Verified end-to-end via a fresh fake-xmrig test after redeploying Falco and reloading netdata: CRITICAL alarm fired and Telegram delivery succeeded.
This commit is contained in:
+41
-14
@@ -5,20 +5,47 @@
|
||||
# else. Add entries here as new noisy defaults turn up - don't let this list
|
||||
# grow into blinding Falco to anything actually meaningful.
|
||||
|
||||
# "Sensitive file opened for reading by non-trusted program" fired every
|
||||
# ~10s from inbox-zero-db's pg_isready healthcheck touching /etc/shadow -
|
||||
# NSS/libc user-lookup behavior during process init, not credential
|
||||
# harvesting. Scoped to just this binary, not a blanket rule disable.
|
||||
# "Sensitive file opened for reading by non-trusted program" false-positives:
|
||||
# - pg_isready: fired every ~10s from inbox-zero-db's healthcheck touching
|
||||
# /etc/shadow - NSS/libc user-lookup behavior during process init, not
|
||||
# credential harvesting.
|
||||
# - lscr.io/linuxserver/obsidian: fires once per container start from
|
||||
# `sed -i s/CORRUPT_FILE/NOPASSWD/g /etc/sudoers` - linuxserver.io's
|
||||
# own s6-init NOPASSWD setup, same class of init-time file touch.
|
||||
- macro: user_known_read_sensitive_files_activities
|
||||
condition: (proc.name = pg_isready)
|
||||
condition: >
|
||||
(proc.name = pg_isready)
|
||||
or (container.image.repository = "lscr.io/linuxserver/obsidian" and proc.name = sed)
|
||||
|
||||
# "Drop and execute new binary in container" fires on every git push/pull
|
||||
# to gitea over SSH - gitea's own binary re-executes itself for git-shell
|
||||
# commands (serv/hook/pre-receive/post-receive), which looks identical to
|
||||
# drift-detection's "new binary written to the writable layer" pattern.
|
||||
# Scoped to proc.name=gitea specifically (not the whole container via
|
||||
# known_drop_and_execute_containers) so this container - the one that was
|
||||
# actually compromised - still gets checked for any OTHER unexpected
|
||||
# binary, just not its own expected self-invocation.
|
||||
# "Fileless execution via memfd_create" fires from nvidia-ctk's
|
||||
# ldconfig-refresh hook, which runs via memfd_create by design every time
|
||||
# an NVIDIA-GPU container (e.g. ollama) starts - normal container-toolkit
|
||||
# behavior, not defense evasion. Scoped to the hook's own parent process
|
||||
# names rather than the ollama/ollama image, so drift detection stays live
|
||||
# for anything else that image does.
|
||||
- macro: known_memfd_execution_processes
|
||||
condition: (proc.pname in (nvidia-containe, nvidia-ctk))
|
||||
|
||||
# "Drop and execute new binary in container" (output text: "Executing binary
|
||||
# not part of base image") false-positives on three known-benign sources,
|
||||
# all confirmed via docker logs falco (2026-08-08 to 2026-08-16):
|
||||
# - gitea: re-executes itself for git-shell commands (serv/hook/pre-receive/
|
||||
# post-receive) over SSH, which looks identical to a dropped binary.
|
||||
# Scoped to proc.name=gitea, not the whole container, so this
|
||||
# previously-compromised container still gets checked for any OTHER
|
||||
# unexpected binary.
|
||||
# - lscr.io/linuxserver/obsidian: Electron's normal /proc/self/exe
|
||||
# self-re-exec pattern (zygote/renderer/utility processes), fires ~6x
|
||||
# every time the container is recreated (~daily, likely watchtower).
|
||||
# Scoped to the whole image since this is normal Electron startup
|
||||
# behavior end to end, not one specific binary.
|
||||
# - tsl0922/ttyd (Synchronet BBS web terminal): telnet/pv/busybox-extras
|
||||
# installed via apk at container start and then used for the BBS's
|
||||
# normal telnet workflow. Scoped to those three binaries, not the image,
|
||||
# since this is a shell-access container where an unexpected binary is
|
||||
# more worth knowing about.
|
||||
- macro: known_drop_and_execute_activities
|
||||
condition: (container.image.repository = "docker.gitea.com/gitea" and proc.name = gitea)
|
||||
condition: >
|
||||
(container.image.repository = "docker.gitea.com/gitea" and proc.name = gitea)
|
||||
or (container.image.repository = "lscr.io/linuxserver/obsidian")
|
||||
or (container.image.repository = "tsl0922/ttyd" and proc.name in (telnet, pv, busybox-extras))
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
# Override of netdata's stock used_swap alarm (health.d/swap.conf). The
|
||||
# stock template recalculates $used/$free from the *instantaneous* raw
|
||||
# sample every 10s with only a 30s notification debounce, so a swap spike
|
||||
# lasting a couple of minutes was enough to page. Only the delay line
|
||||
# changes here (up 30s -> up 5m) - the rest is copied verbatim from the
|
||||
# stock template since netdata health.d overrides replace the whole
|
||||
# template, not just one line. This requires sustained swap pressure
|
||||
# (5+ continuous minutes above threshold) before notifying, instead of
|
||||
# firing on every brief spike.
|
||||
template: used_swap
|
||||
on: mem.swap
|
||||
class: Utilization
|
||||
type: System
|
||||
component: Memory
|
||||
host labels: _os=linux freebsd
|
||||
calc: (($used + $free) > 0) ? ($used * 100 / ($used + $free)) : 0
|
||||
units: %
|
||||
every: 10s
|
||||
warn: $this > (($status >= $WARNING) ? (80) : (90))
|
||||
crit: $this > (($status == $CRITICAL) ? (90) : (98))
|
||||
delay: up 5m down 15m multiplier 1.5 max 1h
|
||||
summary: System swap memory utilization
|
||||
info: Swap memory utilization
|
||||
to: sysadmin
|
||||
Reference in New Issue
Block a user