Set ollama's server-level context default to 11264, fix compose drift
The prior fix dropped numCtx to 8192 to guarantee full GPU offload, but real multi-turn djAgentPick conversations peak around 7.5k tokens -- leaving almost no headroom before llama.cpp's context-shift drops the front of the prompt (system instructions + tool defs, including `done`). Result: "agent stopped without calling done" recurring every few picks. 11264 is the largest context that still offloads all 29 layers on this GPU (tested empirically -- 12288 fell 15MB short and dropped to 28/29), leaving ~3.7k tokens of margin over the observed peak. Also: OLLAMA_CONTEXT_LENGTH had drifted onto the running container via an earlier manual `docker run` and was never in this compose file, so a prior git-redeploy silently kept the manual value instead of the committed one. Committing it here closes that drift. Claude-Session: https://claude.ai/code/session_01L7Rwa6guD5wK8F8tWQwcJX
This commit is contained in:
@@ -17,6 +17,13 @@ services:
|
|||||||
# attention + an explicitly quantized KV cache keep it small and stable.
|
# attention + an explicitly quantized KV cache keep it small and stable.
|
||||||
- OLLAMA_FLASH_ATTENTION=1
|
- OLLAMA_FLASH_ATTENTION=1
|
||||||
- OLLAMA_KV_CACHE_TYPE=q8_0
|
- OLLAMA_KV_CACHE_TYPE=q8_0
|
||||||
|
# Server-level default context, kept in step with SUB/WAVE's own
|
||||||
|
# llm.numCtx (settings.json) -- 11264 is the largest value that still
|
||||||
|
# fits the full 29-layer GPU offload on this 6GB card with margin, while
|
||||||
|
# staying above the ~7.5k-token peak seen in real multi-turn DJ-agent
|
||||||
|
# calls (a lower ceiling truncates the front of the prompt -- including
|
||||||
|
# the tool definitions -- and the agent stops calling `done`, #291).
|
||||||
|
- OLLAMA_CONTEXT_LENGTH=11264
|
||||||
volumes:
|
volumes:
|
||||||
- /srv/ollama:/root/.ollama
|
- /srv/ollama:/root/.ollama
|
||||||
runtime: nvidia
|
runtime: nvidia
|
||||||
|
|||||||
Reference in New Issue
Block a user