Ollama Running Slow? 7 Fixes That Actually Worked on My Machine

I spent two evenings convinced my hardware was too weak for local models. It wasn't. Here is the exact order I now check things in, fastest fix first.

1. Check whether you are actually on the GPU

Related: Midjourney vs DALL-E vs Flux: A Practical Image-Tool Comparison Framework →

Related: أخطاء Ollama الشائعة وحلولها (من واقع 6 أشهر استخدام يومي) →

This is the fix in maybe half of all "Ollama is slow" cases.

ollama ps

Look at the PROCESSOR column. If it says 100% CPU, everything below is irrelevant — fix this first.

nvidia-smi          # NVIDIA: driver present?
ollama serve        # read the startup log, it prints what it detected

On my machine the log said no compatible GPUs were discovered because the driver was 535 and the runtime wanted 550+. A driver update alone took me from 4 to 26 tokens/s.

2. Your model does not fit in VRAM

Related: 9 Free AI Coding Tools Every Developer Should Try in 2026 →

Related: How Much RAM Do You Actually Need for Local AI? I Tested 8, 16, 32 and 64GB →

If part of the model spills to system RAM, throughput collapses — not degrades, collapses. Rough rule I use:

With 8GB VRAM, llama3.1:8b-instruct-q4_K_M sits comfortably. q8_0 does not, and that one letter difference was a 6x speed gap for me.

ollama pull llama3.1:8b-instruct-q4_K_M

3. The context window is quietly enormous

Related: Sora 2 Review: OpenAI's Video Model Is Finally Useful for Real Work →

Related: I Replaced Copilot With a Fully Offline AI Assistant in VS Code →

Ollama allocates KV cache for the full context whether you use it or not. A 32k context on a 7B model can eat 4GB before you type anything.

ollama run llama3.1 --verbose
>>> /set parameter num_ctx 4096

Or in a Modelfile:

FROM llama3.1:8b-instruct-q4_K_M
PARAMETER num_ctx 4096

I keep 4096 as the default and only raise it for document work. This alone let a 13B model fit on a card where it previously didn't.

4. Models are being unloaded between calls

Related: How to Clone Your Voice with AI in 2026 (Free and Paid Options) →

Related: Transcribing Audio Locally with Whisper: My Actual Workflow (No Uploads) →

Default keep-alive is 5 minutes. If you call the model every ten minutes, you pay the full load cost every single time — that was the "first message is always slow" mystery for me.

export OLLAMA_KEEP_ALIVE=30m

5. Flash attention is off

Related: Ollama vs LM Studio vs Jan: I Used All Three for a Month →

export OLLAMA_FLASH_ATTENTION=1

Roughly 15-20% faster generation and noticeably lower memory at long contexts on my RTX 4060. There is no downside I have found on supported cards.

6. Thermal throttling on laptops

Related: CUDA Out of Memory: 9 Fixes That Actually Worked on My GPU →

Sustained generation is a sustained load. My laptop dropped 40% after four minutes of continuous output until I set the power profile to performance and raised the machine off the desk. Watch it live:

watch -n1 nvidia-smi --query-gpu=temperature.gpu,clocks.sm --format=csv

If clocks fall while temperature climbs past ~85C, that is your answer, not the software.

7. You are measuring the wrong thing

--verbose prints the real numbers:

total duration:       6.7s
load duration:        41ms
prompt eval rate:     412 tokens/s
eval rate:            31.2 tokens/s

eval rate is generation speed. prompt eval rate is how fast it reads your input. A long system prompt makes responses feel slow while generation is perfectly fine — different problem, different fix (shorten the prompt or cache it).

Experience log

• Attempt 1: blamed the model, downloaded three smaller ones. No change. Wasted an evening.
• Attempt 2: found 100% CPU in ollama ps. Updated the NVIDIA driver. 4 → 26 tok/s.
• Attempt 3: switched q8_0 to q4_K_M. 26 → 29 tok/s, and 13B models became usable.
• Attempt 4: dropped num_ctx from 32768 to 4096, enabled flash attention. 29 → 31 tok/s and stopped OOM crashes at long chats.
• Mistake I made: set OLLAMA_NUM_PARALLEL=4 thinking it would help. It split VRAM four ways and everything got slower. Left it at 1.

FAQ

Does more system RAM speed up local models? Only if the model is spilling out of VRAM — and in that case the fix is a smaller quantization, not more RAM. On a GPU that fits the model, system RAM has almost no effect on generation speed.

Is Q4 noticeably worse than Q8? On 7B and larger, the quality gap between Q4_K_M and Q8_0 is small enough that I have never chosen Q8 for daily work. Below 3B parameters the gap becomes real.

Why is the first message always slow? Model load time. Raise OLLAMA_KEEP_ALIVE.

Can I run this without a GPU at all? Yes, and a 3B model at Q4 on a modern CPU gives around 8-12 tokens/s — usable for short answers, painful for long ones.

Starting from zero? Read the full walkthrough first: How to run an AI model locally on your own computer, then come back here.