Remove obico's GPU reservation to free VRAM for ollama
obico's own comment already notes its :cuda tag falls back to CPU inference on this driver/GPU combo, but it was still reserving 826MB of VRAM it never used productively. That squeeze was forcing ollama to offload only 20/29 model layers to GPU, pushing the rest onto CPU and causing severe latency (multi-minute LLM calls) that stalled SUB/WAVE's track picking. Claude-Session: https://claude.ai/code/session_01L7Rwa6guD5wK8F8tWQwcJX
This commit is contained in:
@@ -7,9 +7,10 @@ services:
|
||||
- obico-internal
|
||||
|
||||
obico:
|
||||
# :cuda tag attempts GPU but falls back to CPU if CUDA version mismatch
|
||||
# (libcudart.so.11.0 required; GTX 1660 SUPER with driver 590 has CUDA 12).
|
||||
# ML inference still works on CPU - adequate for a single printer.
|
||||
# :cuda tag falls back to CPU anyway on this GPU (libcudart.so.11.0
|
||||
# required; GTX 1660 SUPER with driver 590 has CUDA 12) - adequate for a
|
||||
# single printer. No GPU reservation here so it doesn't reserve VRAM
|
||||
# ollama needs for its own model (was forcing partial CPU offload there).
|
||||
image: ghcr.io/imagegenius/obico:cuda
|
||||
container_name: obico
|
||||
restart: unless-stopped
|
||||
@@ -33,13 +34,6 @@ services:
|
||||
- /srv/obico/config/model_cache:/model_cache/ml_api
|
||||
ports:
|
||||
- "3334:3334"
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: 1
|
||||
capabilities: [gpu]
|
||||
networks:
|
||||
- npm-network
|
||||
- obico-internal
|
||||
|
||||
Reference in New Issue
Block a user