Why Ollama's First Response Is Slow (Cold Start Fix)

I noticed it the first week I moved my daily coding assistant off the cloud: the first question of the morning took about 40 seconds to start answering. Every question after that came back in two or three seconds. Same model, same prompt length, same machine. It felt broken, so I spent an afternoon measuring instead of guessing.

It was not broken. It was a cold start — Ollama unloading the model from memory and loading it again from disk. Once I understood the sequence, I got the worst case from 40 seconds down to about 3, and most of the fix was configuration rather than hardware.

What is actually happening during those 40 seconds

Related: Ollama Answers Get Cut Off Mid-Sentence: num_ctx vs num_predict Explained →

When you send a prompt, Ollama has to do several things before a single token appears:

1. Read the model weights off disk (for a 7B Q4 model that is roughly 4-5 GB).
2. Copy those weights into VRAM, or into system RAM if they will not fit.
3. Allocate the KV cache for your context window.
4. Run the prompt through as a prefill pass.
5. Start generating.

Steps 1 and 2 are the expensive ones, and they only run when the model is not already resident. Once it is loaded, your next prompt skips straight to step 4, which is why response two feels instant.

You can watch this directly. Run this in one terminal:

watch -n 1 ollama ps

Send a prompt and you will see the model appear with a size and an UNTIL column. Wait a few minutes without prompting and the row disappears — that is the unload. Prompt again and you have paid the full cold start.

Fix 1: stop the model from unloading (keep_alive)

Related: Ollama Error "model requires more system memory": How I Fixed It in 10 Minutes →

By default Ollama keeps a model in memory for five minutes after the last request, then frees it. On a workstation with spare RAM, that default is doing you no favours.

Set the lifetime globally with an environment variable:

OLLAMA_KEEP_ALIVE=24h

On Linux with systemd, put it in the service override so it survives reboots:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_KEEP_ALIVE=24h"

Then reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama

You can also set it per request, which is useful when a script wants a model warm only for the length of a job:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "hello",
  "keep_alive": "1h"
}'

Use -1 to keep a model resident indefinitely and 0 to unload the moment the request finishes. I use -1 on my desktop and 10m on the shared box at home so a model does not squat on VRAM other people need. There is more detail on the flag behaviour in our guide to keeping a model loaded with keep_alive.

Fix 2: preload the model before you need it

Related: DeepSeek-R1 Repeats Itself or Outputs Gibberish: The 4 Settings That Fixed It →

Keeping a model alive only helps after the first load. To pay the cost while you are still pouring coffee, send an empty prompt at login — Ollama treats it as a load request and returns immediately after the weights are resident:

curl -s http://localhost:11434/api/generate -d '{"model":"llama3.1:8b","keep_alive":-1}' > /dev/null

A small systemd user timer or a login script line is enough. On macOS I put the same curl in a launchd plist. The nice part is the load happens while the machine is otherwise idle, so by the time you open your editor the model is already sitting in memory.

If you use several models, preload the one you actually start with, not all of them. Loading three 8B models at boot just moves the pain to your swap file.

Fix 3: check whether it is disk, not memory

Related: Ollama Pull Fails with "max retries exceeded" or EOF: How I Get Downloads to Finish →

If your cold start is longer than about 15 seconds for a 7-8B model, the bottleneck is probably storage. Time a raw read of the blob:

du -sh ~/.ollama/models/blobs
sync; sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
time cat ~/.ollama/models/blobs/sha256-* > /dev/null

On my NVMe drive a 4.7 GB model reads in about 4 seconds. On an external USB drive I tested with, the same read took 38 seconds — and that number matched the cold start almost exactly. If your models live on a spinning disk, a network share, or an external enclosure, moving them is the single biggest win available.

Moving the directory is straightforward; set OLLAMA_MODELS to the new path and re-pull or copy the blobs across. We walk through it in moving the Ollama models directory.

Fix 4: shrink what has to be loaded

Related: Ollama "connection refused on 127.0.0.1:11434": The 5 Causes I Have Actually Hit →

Two settings change how much data has to move before the first token:

Quantisation. A Q4_K_M build of an 8B model is around 4.7 GB. The Q8 build of the same model is closer to 8.5 GB, so it takes roughly twice as long to load and uses twice the VRAM. Unless you have measured a quality difference on your own prompts, Q4_K_M is the sensible default. We ran that comparison in Q4 vs Q5 vs Q8 quantisation quality.

Context window. The KV cache is allocated at load time and scales with num_ctx. Jumping from 4096 to 32768 tokens on an 8B model added about 3.5 GB of allocation on my setup and roughly two seconds to every cold start. Set the context to what your work actually needs, not to the maximum the model advertises.

You can pin both in a Modelfile so you are not passing flags every time:

FROM llama3.1:8b
PARAMETER num_ctx 8192

ollama create work-llama -f Modelfile

What the numbers looked like on my machine

Related: DeepSeek-R1 Shows Its <think> Tags in the Output — Here Is How I Strip Them →

RTX 4060 Ti 16 GB, Ryzen 5 7600, models on NVMe, llama3.1:8b Q4_K_M:

The headline improvement was not a faster GPU. It was storage, then context size, then simply refusing to let the model unload.

When the slow first response is something else

A few cases look like a cold start but are not:

• Every response is slow, not just the first. That is usually CPU fallback rather than GPU inference. Check ollama ps — if the size column says 100% CPU, the model did not fit in VRAM. Our notes on Ollama not using the GPU under WSL2 cover the usual causes.
• Long pauses on long prompts only. That is prefill, and it scales with input length. Sending a 20,000-token document will always take a while before the first token, warm or cold.
• The first response after a model switch is slow. Expected. Loading model B evicts model A if VRAM is tight, and switching back reloads it. If you swap between two models constantly, either keep both resident or accept the reload; see running two local models on one GPU.
• Delays of a few seconds inside Docker on Windows. Filesystem passthrough is slow. Keep the model volume inside the WSL filesystem rather than on a bind mount from C:.

The setup I settled on

On my workstation the whole configuration is four lines:

OLLAMA_KEEP_ALIVE=-1
OLLAMA_MODELS=/mnt/nvme/ollama
OLLAMA_MAX_LOADED_MODELS=2
OLLAMA_FLASH_ATTENTION=1

Plus one preload curl in my login script. First prompt of the day: about a second. No hardware was purchased, and nothing about the model changed.

If you are still stuck above ten seconds warm, the problem has moved from loading to inference speed, and that is a different set of levers — batch size, offload layers, and context. Our broader write-up on Ollama slowness fixes that actually worked picks up where this one stops.