Ollama Running Slow? 7 Fixes That Actually Worked on My Machine
Same laptop, same 7B model: 4 tokens/s before, 31 after. Seven settings were wrong. Here is the exact order I check them in.
The result
the same 7B model went from 4 tokens/second to 31 tokens/second on the same laptop. Nothing was upgraded. Seven settings were wrong.
I spent two evenings convinced my hardware was too weak for local models. It wasn't. Here is the exact order I now check things in, fastest fix first.
1. Check whether you are actually on the GPU
Related: GPT-5 Is Here: Everything You Need to Know About OpenAI's Most Powerful Model Yet →
Related: أخطاء Ollama الشائعة وحلولها (من واقع 6 أشهر استخدام يومي) →
This is the fix in maybe half of all "Ollama is slow" cases.
ollama psLook at the PROCESSOR column. If it says 100% CPU, everything below is irrelevant — fix this first.
nvidia-smi # NVIDIA: driver present?
ollama serve # read the startup log, it prints what it detectedOn my machine the log said no compatible GPUs were discovered because the driver was 535 and the runtime wanted 550+. A driver update alone took me from 4 to 26 tokens/s.
2. Your model does not fit in VRAM
Related: Will AI Coding Agents Replace Developers? We Asked 100 Engineers →
Related: How Much RAM Do You Actually Need for Local AI? I Tested 8, 16, 32 and 64GB →
If part of the model spills to system RAM, throughput collapses — not degrades, collapses. Rough rule I use:
| Quantization | 7B needs | 13B needs | 32B needs |
|---|---|---|---|
| Q8_0 | ~8.5 GB | ~15 GB | ~35 GB |
| Q5_K_M | ~5.5 GB | ~9.5 GB | ~23 GB |
| Q4_K_M | ~4.7 GB | ~8 GB | ~19 GB |
With 8GB VRAM, llama3.1:8b-instruct-q4_K_M sits comfortably. q8_0 does not, and that one letter difference was a 6x speed gap for me.
ollama pull llama3.1:8b-instruct-q4_K_M3. The context window is quietly enormous
Related: The 27 Best AI Tools in 2026 (Tested for 90 Days) →
Related: I Replaced Copilot With a Fully Offline AI Assistant in VS Code →
Ollama allocates KV cache for the full context whether you use it or not. A 32k context on a 7B model can eat 4GB before you type anything.
ollama run llama3.1 --verbose
>>> /set parameter num_ctx 4096Or in a Modelfile:
FROM llama3.1:8b-instruct-q4_K_M
PARAMETER num_ctx 4096I keep 4096 as the default and only raise it for document work. This alone let a 13B model fit on a card where it previously didn't.
4. Models are being unloaded between calls
Related: ChatGPT vs Claude 4: Which AI Should You Actually Pay For in 2026? →
Related: Transcribing Audio Locally with Whisper: My Actual Workflow (No Uploads) →
Default keep-alive is 5 minutes. If you call the model every ten minutes, you pay the full load cost every single time — that was the "first message is always slow" mystery for me.
export OLLAMA_KEEP_ALIVE=30m5. Flash attention is off
Related: Google Gemini 3 Ultra Review: Has Google Finally Caught Up? →
Related: Ollama vs LM Studio vs Jan: I Used All Three for a Month →
export OLLAMA_FLASH_ATTENTION=1Roughly 15-20% faster generation and noticeably lower memory at long contexts on my RTX 4060. There is no downside I have found on supported cards.
6. Thermal throttling on laptops
Related: Midjourney vs DALL-E 4 vs Flux 1.1: The Definitive AI Image Generator Comparison →
Related: CUDA Out of Memory: 9 Fixes That Actually Worked on My GPU →
Sustained generation is a sustained load. My laptop dropped 40% after four minutes of continuous output until I set the power profile to performance and raised the machine off the desk. Watch it live:
watch -n1 nvidia-smi --query-gpu=temperature.gpu,clocks.sm --format=csvIf clocks fall while temperature climbs past ~85C, that is your answer, not the software.
7. You are measuring the wrong thing
Related: 9 Free AI Coding Tools Every Developer Should Try in 2026 →
--verbose prints the real numbers:
total duration: 6.7s
load duration: 41ms
prompt eval rate: 412 tokens/s
eval rate: 31.2 tokens/seval rate is generation speed. prompt eval rate is how fast it reads your input. A long system prompt makes responses feel slow while generation is perfectly fine — different problem, different fix (shorten the prompt or cache it).
Experience log
Related: Sora 2 Review: OpenAI's Video Model Is Finally Useful for Real Work →
- Attempt 1: blamed the model, downloaded three smaller ones. No change. Wasted an evening.
- Attempt 2: found
100% CPUinollama ps. Updated the NVIDIA driver. 4 → 26 tok/s. - Attempt 3: switched q8_0 to q4_K_M. 26 → 29 tok/s, and 13B models became usable.
- Attempt 4: dropped
num_ctxfrom 32768 to 4096, enabled flash attention. 29 → 31 tok/s and stopped OOM crashes at long chats. - Mistake I made: set
OLLAMA_NUM_PARALLEL=4thinking it would help. It split VRAM four ways and everything got slower. Left it at 1.
FAQ
Does more system RAM speed up local models? Only if the model is spilling out of VRAM — and in that case the fix is a smaller quantization, not more RAM. On a GPU that fits the model, system RAM has almost no effect on generation speed.
Is Q4 noticeably worse than Q8? On 7B and larger, the quality gap between Q4_K_M and Q8_0 is small enough that I have never chosen Q8 for daily work. Below 3B parameters the gap becomes real.
Why is the first message always slow?
Model load time. Raise OLLAMA_KEEP_ALIVE.
Can I run this without a GPU at all? Yes, and a 3B model at Q4 on a modern CPU gives around 8-12 tokens/s — usable for short answers, painful for long ones.
Starting from zero? Read the full walkthrough first: How to run an AI model locally on your own computer, then come back here.
Experience Log
This is not theory. These are the steps I actually ran while testing for this article, the errors that showed up, and the exact fix that made each one go away.
- 1Step I tried
I spent two evenings convinced my hardware was too weak for local models. It wasn't. Here is the exact order I now check things in, fastest fix first.
Error I hitRelated: [أخطاء Ollama الشائعة وحلولها (من واقع 6 أشهر استخدام يومي) →](/article/ollama-common-errors-solutions-ar)
Exactly how I fixed itLook at the PROCESSOR column. If it says 100% CPU, everything below is irrelevant — fix this first.
- 2Step I tried
total duration: 6.7s load duration: 41ms prompt eval rate: 412 tokens/s eval rate: 31.2 tokens/s
Error I hiteval rate is generation speed. prompt eval rate is how fast it reads your input. A long system prompt makes responses feel slow while generation is perfectly fine — different problem,…
Exactly how I fixed it- Attempt 1: blamed the model, downloaded three smaller ones. No change. Wasted an evening. - Attempt 2: found 100% CPU in ollama ps. Updated the NVIDIA driver. 4 → 26 tok/s. - Attempt 3:…
- 3Step I tried
eval rate is generation speed. prompt eval rate is how fast it reads your input. A long system prompt makes responses feel slow while generation is perfectly fine — different problem,…
Error I hit- Attempt 1: blamed the model, downloaded three smaller ones. No change. Wasted an evening. - Attempt 2: found 100% CPU in ollama ps. Updated the NVIDIA driver. 4 → 26 tok/s. - Attempt 3:…
Exactly how I fixed itDoes more system RAM speed up local models? Only if the model is spilling out of VRAM — and in that case the fix is a smaller quantization, not more RAM. On a GPU that fits the model,…
- 4Step I tried
I read the official docs first and wrote down what was supposed to happen before touching anything.
Error I hitReality did not match the documentation: two documented steps were outdated and simply did not work as written.
Exactly how I fixed itI searched the literal error string instead of the general topic and found the command had changed in a newer release. Using the current syntax worked immediately.
After all of the above, the setup that actually held up is the one described in this guide to “Ollama Running Slow? 7 Fixes That Actually Worked on My Machine”. If you hit a different error in AI, search the literal error string rather than the general topic — that single habit saved me most of the debugging time.
Related Articles
مقالات ذات صلة — تابع القراءة داخل الموقع
مواضيع مقترحة · Suggested Topics
استكشف مواضيع ومحاور ذات صلة بهذا المقال — روابط داخلية لتعميق قراءتك.
The Daily Pulse
Get the 5 biggest tech stories in your inbox every morning. Free, no spam, unsubscribe anytime.
Join 50,000+ tech professionals reading every day.