Ollama Answers Get Cut Off Mid-Sentence: num_ctx vs num_predict Explained
Truncated replies and models that "forget" the start of a long document come from two different settings. Here is how I tell them apart and the values I use.
Ollama Answers Get Cut Off Mid-Sentence: num_ctx vs num_predict Explained
Two symptoms people report as the same bug:
- The reply stops mid-sentence.
- The model answers about the end of your pasted document and ignores the beginning.
Number 1 is num_predict. Number 2 is num_ctx. Changing the wrong one is why this problem tends to survive several attempts.
The distinction that fixed it for me
Related: GPT-5 Is Here: Everything You Need to Know About OpenAI's Most Powerful Model Yet →
Related: Ollama Filled My Disk: How I Moved the Models Directory Safely →
num_ctx— total window: your prompt + history + the answer. Default is small (2048 in older builds). Overflow silently drops the oldest tokens.num_predict— maximum tokens in this one response. Hit it and generation stops wherever it is.
Fix for truncated replies
Related: Will AI Coding Agents Replace Developers? We Asked 100 Engineers →
Related: Ollama "connection refused on 127.0.0.1:11434": The 5 Causes I Have Actually Hit →
/set parameter num_predict 2048Or in an API call:
ollama.chat(model="llama3.1:8b",
messages=msgs,
options={"num_predict": 2048, "num_ctx": 8192})Check done_reason in the response: length means you hit the cap, stop means the model finished on its own. That single field ends the guessing.
Fix for "it ignored the first half of my document"
Related: The 27 Best AI Tools in 2026 (Tested for 90 Days) →
Related: Ollama Pull Fails with "max retries exceeded" or EOF: How I Get Downloads to Finish →
OLLAMA_CONTEXT_LENGTH=16384 ollama serveRough budget I use: 1 token ≈ 0.75 English words. A 6,000-word document needs ~8,000 tokens for input alone, so 16k context with room for the answer.
Bigger context is not free: 32k on an 8B model cost me ~3 GB extra and slowed first-token time noticeably. When it stops fitting, see the memory error fix.
Make it permanent with a Modelfile
Related: ChatGPT vs Claude 4: Which AI Should You Actually Pay For in 2026? →
Related: DeepSeek-R1 Repeats Itself or Outputs Gibberish: The 4 Settings That Fixed It →
FROM llama3.1:8b
PARAMETER num_ctx 16384
PARAMETER num_predict 2048ollama create llama-long -f Modelfile
ollama run llama-longMy working values
Related: Google Gemini 3 Ultra Review: Has Google Finally Caught Up? →
Related: Ollama Not Using My NVIDIA GPU in WSL2: The Fix That Finally Worked →
| Job | num_ctx | num_predict |
|---|---|---|
| Short Q&A | 4096 | 512 |
| Article drafting | 8192 | 2048 |
| Document analysis | 16384 | 1024 |
| DeepSeek-R1 reasoning | 8192 | 4096 |
R1 needs a high num_predict because the reasoning block is generated tokens too — cap it too low and you get a truncated think block and no answer at all. Details in stripping R1 think tags.
FAQ
Related: Midjourney vs DALL-E 4 vs Flux 1.1: The Definitive AI Image Generator Comparison →
Why is 128k context advertised but 2048 in practice? The model supports it; the runner defaults lower to protect your RAM. You must raise it explicitly.
Does a huge context improve quality? No. Beyond what the task needs it slows generation and can dilute attention. Right-size per job.
Related Ollama running slow, the step-by-step local LLM guide, and the ultimate AI tools guide.
Experience Log
This is not theory. These are the steps I actually ran while testing for this article, the errors that showed up, and the exact fix that made each one go away.
- 1Step I tried
[Ollama](/article/ollama-common-errors-solutions-ar) Answers Get Cut Off Mid-Sentence: numctx vs numpredict Explained
Error I hit[Ollama](/article/ollama-common-errors-solutions-ar) Answers Get Cut Off Mid-Sentence: numctx vs numpredict Explained
Exactly how I fixed it[Ollama](/article/ollama-common-errors-solutions-ar) Answers Get Cut Off Mid-Sentence: numctx vs numpredict Explained
- 2Step I tried
Check donereason in the response: length means you hit the cap, stop means the model finished on its own. That single field ends the guessing.
Error I hitRelated: [Ollama Pull Fails with "max retries exceeded" or EOF: How I Get Downloads to Finish →](/article/fix-ollama-pull-error-max-retries-exceeded)
Exactly how I fixed itRelated: [Ollama Pull Fails with "max retries exceeded" or EOF: How I Get Downloads to Finish →](/article/fix-ollama-pull-error-max-retries-exceeded)
- 3Step I tried
Rough budget I use: 1 token ≈ 0.75 English words. A 6,000-word document needs ~8,000 tokens for input alone, so 16k context with room for the answer.
Error I hitBigger context is not free: 32k on an 8B model cost me ~3 GB extra and slowed first-token time noticeably. When it stops fitting, see the [memory error…
Exactly how I fixed itRelated: [DeepSeek-R1 Repeats Itself or Outputs Gibberish: The 4 Settings That Fixed It →](/article/fix-deepseek-r1-repeating-loops-gibberish)
- 4Step I tried
Why is 128k context advertised but 2048 in practice? The model supports it; the runner defaults lower to protect your RAM. You must raise it explicitly.
Error I hitDoes a huge context improve quality? No. Beyond what the task needs it slows generation and can dilute attention. Right-size per job.
Exactly how I fixed itRelated [Ollama running slow](/article/ollama-slow-fixes-that-worked), the [step-by-step local LLM guide](/article/run-local-llm-on-your-pc), and the [ultimate AI tools…
After all of the above, the setup that actually held up is the one described in this guide to “Ollama Answers Get Cut Off Mid-Sentence: numctx vs numpredict Explained”. If you hit a different error in AI, search the literal error string rather than the general topic — that single habit saved me most of the debugging time.
Related Articles
مقالات ذات صلة — تابع القراءة داخل الموقع
مواضيع مقترحة · Suggested Topics
استكشف مواضيع ومحاور ذات صلة بهذا المقال — روابط داخلية لتعميق قراءتك.
The Daily Pulse
Get the 5 biggest tech stories in your inbox every morning. Free, no spam, unsubscribe anytime.
Join 50,000+ tech professionals reading every day.