Remove obico's GPU reservation to free VRAM for ollama
obico's own comment already notes its :cuda tag falls back to CPU inference on this driver/GPU combo, but it was still reserving 826MB of VRAM it never used productively. That squeeze was forcing ollama to offload only 20/29 model layers to GPU, pushing the rest onto CPU and causing severe latency (multi-minute LLM calls) that stalled SUB/WAVE's track picking. Claude-Session: https://claude.ai/code/session_01L7Rwa6guD5wK8F8tWQwcJX
This commit is contained in:
@@ -7,9 +7,10 @@ services:
|
|||||||
- obico-internal
|
- obico-internal
|
||||||
|
|
||||||
obico:
|
obico:
|
||||||
# :cuda tag attempts GPU but falls back to CPU if CUDA version mismatch
|
# :cuda tag falls back to CPU anyway on this GPU (libcudart.so.11.0
|
||||||
# (libcudart.so.11.0 required; GTX 1660 SUPER with driver 590 has CUDA 12).
|
# required; GTX 1660 SUPER with driver 590 has CUDA 12) - adequate for a
|
||||||
# ML inference still works on CPU - adequate for a single printer.
|
# single printer. No GPU reservation here so it doesn't reserve VRAM
|
||||||
|
# ollama needs for its own model (was forcing partial CPU offload there).
|
||||||
image: ghcr.io/imagegenius/obico:cuda
|
image: ghcr.io/imagegenius/obico:cuda
|
||||||
container_name: obico
|
container_name: obico
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
@@ -33,13 +34,6 @@ services:
|
|||||||
- /srv/obico/config/model_cache:/model_cache/ml_api
|
- /srv/obico/config/model_cache:/model_cache/ml_api
|
||||||
ports:
|
ports:
|
||||||
- "3334:3334"
|
- "3334:3334"
|
||||||
deploy:
|
|
||||||
resources:
|
|
||||||
reservations:
|
|
||||||
devices:
|
|
||||||
- driver: nvidia
|
|
||||||
count: 1
|
|
||||||
capabilities: [gpu]
|
|
||||||
networks:
|
networks:
|
||||||
- npm-network
|
- npm-network
|
||||||
- obico-internal
|
- obico-internal
|
||||||
|
|||||||
Reference in New Issue
Block a user