Ollama Answers Get Cut Off Mid-Sentence: num_ctx vs num_predict Explained
Truncated replies and models that "forget" the start of a long document come from two different settings. Here is how I tell them apart and the values I use.
Ollama Answers Get Cut Off Mid-Sentence: num_ctx vs num_predict Explained
Two symptoms people report as the same bug:
- The reply stops mid-sentence.
- The model answers about the end of your pasted document and ignores the beginning.
Number 1 is num_predict. Number 2 is num_ctx. Changing the wrong one is why this problem tends to survive several attempts.
The distinction that fixed it for me
Related: Midjourney vs DALL-E vs Flux: A Practical Image-Tool Comparison Framework →
Related: Ollama Filled My Disk: How I Moved the Models Directory Safely →
num_ctx— total window: your prompt + history + the answer. Default is small (2048 in older builds). Overflow silently drops the oldest tokens.num_predict— maximum tokens in this one response. Hit it and generation stops wherever it is.
Fix for truncated replies
Related: 9 Free AI Coding Tools Every Developer Should Try in 2026 →
Related: Ollama "connection refused on 127.0.0.1:11434": The 5 Causes I Have Actually Hit →
/set parameter num_predict 2048Or in an API call:
ollama.chat(model="llama3.1:8b",
messages=msgs,
options={"num_predict": 2048, "num_ctx": 8192})Check done_reason in the response: length means you hit the cap, stop means the model finished on its own. That single field ends the guessing.
Fix for "it ignored the first half of my document"
Related: Sora 2 Review: OpenAI's Video Model Is Finally Useful for Real Work →
Related: Ollama Pull Fails with "max retries exceeded" or EOF: How I Get Downloads to Finish →
OLLAMA_CONTEXT_LENGTH=16384 ollama serveRough budget I use: 1 token ≈ 0.75 English words. A 6,000-word document needs ~8,000 tokens for input alone, so 16k context with room for the answer.
Bigger context is not free: 32k on an 8B model cost me ~3 GB extra and slowed first-token time noticeably. When it stops fitting, see the memory error fix.
Make it permanent with a Modelfile
Related: How to Clone Your Voice with AI in 2026 (Free and Paid Options) →
Related: DeepSeek-R1 Repeats Itself or Outputs Gibberish: The 4 Settings That Fixed It →
FROM llama3.1:8b
PARAMETER num_ctx 16384
PARAMETER num_predict 2048ollama create llama-long -f Modelfile
ollama run llama-longMy working values
Related: Ollama Not Using My NVIDIA GPU in WSL2: The Fix That Finally Worked →
| Job | num_ctx | num_predict |
|---|---|---|
| Short Q&A | 4096 | 512 |
| Article drafting | 8192 | 2048 |
| Document analysis | 16384 | 1024 |
| DeepSeek-R1 reasoning | 8192 | 4096 |
R1 needs a high num_predict because the reasoning block is generated tokens too — cap it too low and you get a truncated think block and no answer at all. Details in stripping R1 think tags.
FAQ
Why is 128k context advertised but 2048 in practice? The model supports it; the runner defaults lower to protect your RAM. You must raise it explicitly.
Does a huge context improve quality? No. Beyond what the task needs it slows generation and can dilute attention. Right-size per job.
Related Ollama running slow, the step-by-step local LLM guide, and the ultimate AI tools guide.
Related Articles
مقالات ذات صلة — تابع القراءة داخل الموقع
مواضيع مقترحة · Suggested Topics
استكشف مواضيع ومحاور ذات صلة بهذا المقال — روابط داخلية لتعميق قراءتك.
The Daily Pulse
Newsletter delivery is not connected yet. This form only saves your address in this browser; no email is sent.
Get concise, source-linked technology notes without the hype.