How to Run a Local LLM on 8GB of RAM (What Actually Works in 2026)
I tested seven small models on an 8GB laptop with no dedicated GPU. Three were genuinely usable, four were not. Here is the exact setup, the quantization that worked, and the errors I hit.
Short answer
yes, 8GB of RAM is enough for a local model — but only if you stay at or below a 4B-parameter model in 4-bit quantization, and only if you close your browser first. I got 12–18 tokens per second on a five-year-old ThinkPad with no GPU. Anything larger swapped to disk and became unusable.
I spent a weekend on this because I kept reading "you need 16GB minimum" and I did not believe it. Here is what actually happened.
The hardware I used
Related: Midjourney vs DALL-E vs Flux: A Practical Image-Tool Comparison Framework →
Related: Ollama vs LM Studio vs Jan: I Used All Three for a Month →
Nothing special, on purpose: Intel i5 (4 cores), 8GB DDR4, integrated graphics, Ubuntu 24.04, NVMe drive. If it works here, it works on most office laptops.
Step 1: install Ollama
Related: 9 Free AI Coding Tools Every Developer Should Try in 2026 →
Related: كيف تبني وكيل ذكاء اصطناعي يعمل على جهازك بدون إنترنت (Ollama + MCP) — جرّبته 3 أسابيع (2026) →
curl -fsSL https://ollama.com/install.sh | sh
ollama --versionThat is the whole installation on Linux and macOS. On Windows, download the installer instead — WSL adds a memory tax you cannot afford at 8GB.
Step 2: pick a model that actually fits
Related: Sora 2 Review: OpenAI's Video Model Is Finally Useful for Real Work →
Related: How to Run an AI Model Locally on Your Own Computer (Step-by-Step 2026 Guide) →
This is where most guides go wrong. The rule I use: model file size on disk must be under 55% of your total RAM. The runtime needs headroom for the context window, and your OS needs about 2GB to stay alive.
| Model | Quantized size | Fits in 8GB? | Speed I measured | Honest verdict |
|---|---|---|---|---|
| Qwen3 4B (Q4) | ~2.4 GB | Yes, comfortably | 16–18 tok/s | Best all-rounder here |
| Llama 3.2 3B (Q4) | ~2.0 GB | Yes | 18–22 tok/s | Fastest, weaker at reasoning |
| Phi-4 mini (Q4) | ~2.5 GB | Yes | 14–16 tok/s | Strong at maths and code |
| Gemma 3 4B (Q4) | ~2.9 GB | Tight | 11–13 tok/s | Fine, close the browser |
| Mistral 7B (Q4) | ~4.4 GB | Barely | 4–6 tok/s | Painful, constant swapping |
| Llama 3.1 8B (Q4) | ~4.9 GB | No | 1–3 tok/s | Do not bother |
| Any 13B model | 7 GB+ | No | Crashes | Do not bother |
Pull the winner:
ollama pull qwen3:4b
ollama run qwen3:4bStep 3: cap the context window
Related: How to Clone Your Voice with AI in 2026 (Free and Paid Options) →
Related: CUDA Out of Memory: 9 Fixes That Actually Worked on My GPU →
This is the single change that made the biggest difference and nobody mentions it. Ollama's default context allocation eats RAM you do not have. Drop it:
ollama run qwen3:4b --keepalive 5m
# inside the session:
/set parameter num_ctx 2048For a persistent setting, create a Modelfile:
FROM qwen3:4b
PARAMETER num_ctx 2048
PARAMETER num_thread 4ollama create qwen-light -f Modelfile
ollama run qwen-lightGoing from a 8192-token context to 2048 cut my memory use by roughly 1.3GB and stopped the swap thrashing entirely.
Experience log: the three errors I hit
Related: I Built a Chatbot for My Own Documents in One Evening (Free, Local, No API Key) →
1. Error: model requires more system memory (5.1 GiB) than is available (3.4 GiB)
Cause: Firefox with 20 tabs. Fix: close it, or add OLLAMA_MAX_LOADED_MODELS=1 to stop a previously loaded model staying resident. Check what is still loaded with ollama ps and unload with ollama stop <model>.
2. Generation started fast, then crawled after ~200 tokens.
Cause: the KV cache growing into swap. Fix: the num_ctx 2048 change above. If you need longer context, you need more RAM — there is no trick here.
3. Error: llama runner process has terminated: signal: killed
That is the Linux OOM killer, not Ollama. Confirm with dmesg | tail -20. Fix: smaller model or smaller context. Adding swap "fixes" the crash but the result is too slow to use.
What 8GB is genuinely good for
Related: كيف تؤتمت عملك بالذكاء الاصطناعي باستخدام n8n مجاناً (دليل عملي خطوة بخطوة 2026) →
After a week of daily use, honest scope:
- Rewriting and summarising text: excellent, no complaints.
- Short code snippets and regex: good with Phi-4 mini.
- Answering questions about a pasted 2–3 page document: fine.
- Long multi-file code refactoring: no. Use a hosted model.
- Anything needing 8k+ context: no.
Should you just use a cloud model instead?
If you have reliable internet and no privacy constraint, a hosted model will be better and cheaper than your time. Local makes sense when the data cannot leave your machine, when you are offline, or when you are making thousands of small calls where per-token pricing adds up. I use local for the first two reasons.
FAQ
Does a GPU with 4GB VRAM help?
Yes, more than extra RAM does. Offload with num_gpu layers; even partial offload roughly doubled my speed on a test machine with a GTX 1650.
Is Q4 quantization noticeably worse than the full model? On 3–4B models the gap is small for writing tasks and noticeable for maths. Q4_K_M is the sweet spot; Q3 and below degrade fast.
Can I run this on a Raspberry Pi 5 with 8GB? Yes, at about 4–6 tok/s with a 3B model. Usable for background jobs, frustrating for chat.
Does macOS handle 8GB better? Yes. Unified memory on Apple Silicon means the GPU shares the same pool, and I saw roughly 25 tok/s on an M1 Air with the same 4B model.
The takeaway: 8GB is not a blocker, it is a constraint that forces you to pick a 3–4B model and cap your context. Do both and it works.
Starting from zero? Read the full walkthrough first: How to run an AI model locally on your own computer, then come back here.
Related Articles
مقالات ذات صلة — تابع القراءة داخل الموقع
مواضيع مقترحة · Suggested Topics
استكشف مواضيع ومحاور ذات صلة بهذا المقال — روابط داخلية لتعميق قراءتك.
The Daily Pulse
Newsletter delivery is not connected yet. This form only saves your address in this browser; no email is sent.
Get concise, source-linked technology notes without the hype.