Root cause of nas_audiobooks still flapping after the automount fix: the
health check finds the first file in each share via `find -quit` and
`cat`s the whole thing within a 10s timeout. That's fine for immich's
13-byte marker and owncloud's 0-byte one, but audiobooks has no such
marker - find picked a real 2.9GB audiobook file, which can never fully
transfer in 10s regardless of mount health. Switched to `head -c 64k`,
which only needs to prove the file handle/NFS path is alive. Confirmed:
the mount was never actually unhealthy post the automount fix, this test
methodology was generating the "stale" verdicts (cat exit=124) itself.
Also captures and logs/alerts the actual find/head error text now instead
of a generic "stale/unreachable" guess, so any future real failure is
diagnosable without SSH archaeology.
Adds the 4 permanent .mount units (family/books/owncloud/audiobooks) and
audiobookshelf-mount-ready.service to the repo backup, which weren't
tracked here yet, and updates the README to document the autofs->permanent
mount fix applied to nas_audiobooks today so a host rebuild doesn't
recreate the broken automount version.
find -maxdepth 3 couldn't reach immich's thumbs/<uuid>/XX/YY/file (4
levels deep), so it exhausted the 256x256 hash-bucket fan-out with no
match and timed out on every run -- 3 spurious remounts/restarts of
immich_server within 15 minutes of deploy, all falsely alerted as
staleness. Drop -maxdepth entirely; -quit already stops at the first
match via depth-first search, so it's fast regardless of tree depth.
Stale NFS handles inside containers (ESTALE/errno -116) can persist even
after the host-side mount looks healthy, requiring a manual remount +
container restart. Adds systemd mount-ready hooks (BindsTo) to restart
immich_server/calibre/ocis when their NFS mount unit restarts, plus a
5-minute health-check timer that does a real nested read and force-remounts
on staleness. Host files backed up under system-config/nfs-self-heal/ so
they can be reinstalled after a host rebuild.
poprhythm