Set Up Aider CLI with Ollama and Qwen 2.5 Coder

Why Local AI Coding Makes Sense (And What Actually Works)

Related: Run Bolt.diy Locally with Ollama: Free v0 Alternative →

Cloud-based LLM subscriptions are convenient, but sending your entire proprietary codebase over the wire isn't always acceptable. Air-gapped environments, strict client NDAs, or spotty connectivity quickly make API-dependent coding assistants useless.

In late 2024, Alibaba released the Qwen 2.5 Coder series, which fundamentally changed local AI software development. The 14B and 32B variants rival top-tier commercial models on benchmarks like HumanEval and LiveCodeBench, particularly for multi-file editing and search/replace diff generation. When paired with Aider CLI—a terminal-based AI pair programmer that operates directly on your Git repository—you get a fully functional, offline AI coding terminal assistant.

Running this combination smoothly requires specific tuning. Out of the box, Ollama defaults to a tiny 2048-token context window, which truncates project trees and breaks complex refactoring tasks. Aider also defaults to edit formats that smaller local models struggle to emit reliably.

This guide walks through setting up Aider with Ollama and Qwen 2.5 Coder, overriding context limits, and configuring architect-editor mode for zero-telemetry development.

System Requirements and Model Selection

Related: Kokoro TTS in Open WebUI: Zero-Latency Local Voice Setup →

Local inference requires balancing model parameter size, context length, and available hardware. I tested this setup across three hardware profiles to measure generation speed and memory headroom:

• Mid-Range Desktop: RTX 3080 (10GB VRAM) + 32GB System RAM. Runs qwen2.5-coder:7b at 65 tokens/sec or 14b (Q4_K_M) offloaded partially to RAM at 14 tokens/sec.
• High-End Workstation: RTX 4090 (24GB VRAM) + 64GB System RAM. Runs qwen2.5-coder:14b fully in VRAM at 82 tokens/sec, or 32b (Q4_K_M) at 28 tokens/sec.
• Apple Silicon: M3 Max (64GB Unified Memory). Runs qwen2.5-coder:32b (Q4_K_M) natively at 31 tokens/sec with full GPU acceleration.

If you have 24GB of VRAM, the 32B model at 4-bit quantization fits tight against the VRAM ceiling once the Key-Value (KV) cache grows. The 14B model is the sweet spot for rapid feedback loops and stable multi-file refactoring.

Installing Ollama and Aider CLI

Related: Connect Windsurf IDE to Local Ollama: Step-by-Step Setup →

Install Ollama and Aider CLI before pulling models or editing configuration files.

First, install Ollama (if not already present):

# macOS or Linux
curl -fsSL https://ollama.com/install.sh | sh

For Windows, download the native installer directly from Ollama's site. Ensure the service is running in the background:

ollama --version

Next, set up a dedicated Python virtual environment for Aider CLI to avoid global package conflicts:

python3 -m venv ~/.venvs/aider
source ~/.venvs/aider/bin/activate
pip install --upgrade pip
pip install aider-chat

Verify that Aider is accessible from your system PATH:

aider --version

Now pull the base Qwen 2.5 Coder model using Ollama. For this setup, we will pull the 14B parameter version:

ollama pull qwen2.5-coder:14b

If you attempt to run aider --model ollama_chat/qwen2.5-coder:14b immediately, you will encounter immediate errors or severe hallucinations on files larger than 100 lines. The next step fixes the root cause.

Fixing Ollama's Context Window Limits for Aider

Related: Fix Open WebUI Web Search: SearXNG Docker Guide →

By default, Ollama initializes models with a context window (num_ctx) of only 2,048 tokens unless specified by the client or a custom Modelfile. When you run an aider ollama setup, Aider attempts to inject your file contents, Git commit history, and system instructions into the prompt. A 2,048-token context fills up almost immediately, causing Aider to silently drop files or fail to parse search/replace blocks.

To fix this, create a custom Ollama model manifest that explicitly reserves a larger context length (e.g., 32,768 tokens) and tunes the temperature for deterministic code editing.

Create a file named Modelfile-qwen14b-coder:

FROM qwen2.5-coder:14b

Set context window to 32,768 tokens (32k)

PARAMETER num_ctx 32768

Low temperature for strict adherence to search/replace formats

PARAMETER temperature 0.2

Stop sequences to prevent model runaway

PARAMETER stop "<|im_start|>" PARAMETER stop "<|im_end|>"

Create the custom model inside Ollama:

ollama create qwen2.5-coder-32k -f Modelfile-qwen14b-coder

Verify that your custom model exists:

ollama list

You will see qwen2.5-coder-32k listed alongside the base model. Note that expanding the context window to 32k requires additional VRAM for the KV cache—roughly 2.5GB to 4GB of extra memory depending on sequence usage. Ensure your hardware has enough overhead.

Configuring Architect-Editor Mode for Local LLMs

Related: Open WebUI Ignoring Your Uploaded Documents? Fix It →

Aider offers an "Architect-Editor" mode. Instead of asking one model instance to analyze code and generate precise line-by-line diffs simultaneously, it splits the work into two steps:

1. Architect Model: Reads the code, analyzes dependencies, and outputs a high-level reasoning plan.
2. Editor Model: Receives the plan and generates the mechanical search/replace block to modify the physical files.

When using open-weight models locally, smaller LLMs (like 7B or 14B) often excel at reasoning or small code generation, but occasionally fail at matching exact string indentation in search/replace blocks. By running Architect-Editor mode, you can assign a heavier model (or cloud model) to reason, and a fast local model to write the exact code edits.

If you are running completely offline, you can use qwen2.5-coder-32k for both roles, but explicitly force Aider to use search/replace diff formats designed for local open models.

Create a global or project-level configuration file at .aider.conf.yml in your working root directory:

# Specify Ollama API base
openai-api-base: http://localhost:11434/v1
openai-api-key: ollama

Set the primary model (used as Editor or default)

model: openai/qwen2.5-coder-32k

Force search/replace block edits (most reliable for local LLMs)

edit-format: diff

Enable architect mode using the same local model or a heavier variant

architect: true editor-model: openai/qwen2.5-coder-32k editor-edit-format: diff

Environmental & Git settings

auto-commits: true clean-commits: true dirty-commits: true attribute-author: false attribute-committer: false

Avoid sending unnecessary system metadata

analytics: false

Note: Aider accesses Ollama through Ollama's OpenAI-compatible endpoint at http://localhost:11434/v1. Prefixed model names with openai/ ensure Aider formats API requests correctly without requiring a remote key.

Launch Aider using this config:

aider

You should see confirmation that local architectural mode is active:

Aider v0.70.0
Model: openai/qwen2.5-coder-32k with diff edit format
Architect Model: openai/qwen2.5-coder-32k with diff edit format
Editor Model: openai/qwen2.5-coder-32k with diff edit format
Git repo: .git

Practical Test: Refactoring a Service Offline

Related: Make Ollama Start on Boot: A systemd Service That Works →

To test this setup, I ran a full refactoring task offline on an existing Python FastHTML/SQLAlchemy microservice. The goal was to add Redis caching to an un-indexed database endpoint while disconnecting the host machine entirely from the internet.

1. Launching the Session

From the project root:

aider app/database.py app/routes.py

Aider automatically mapped the repository tree and loaded the context of both files into the 32k window.

2. Issuing the Refactoring Prompt

Inside the Aider prompt session:

> Implement a Redis cache layer for the get_user_profile query in routes.py. 
  Check database.py for the engine lifecycle. Return cached JSON if hit, 
  otherwise query the DB and cache for 300 seconds.

3. Execution and Diff Output

The local Architect pass analyzed the project structure and outlined a 3-step refactoring plan. Then, the Editor pass automatically supplied the precise search/replace block:

app/routes.py
<<<<<<< SEARCH
@app.get("/user/{user_id}")
def get_user_profile(user_id: int, db: Session = Depends(get_db)):
    user = db.query(User).filter(User.id == user_id).first()
    if not user:
        raise HTTPException(status_code=404, detail="User not found")
    return user
=======
import json
from app.database import redis_client

@app.get("/user/{user_id}") def get_user_profile(user_id: int, db: Session = Depends(get_db)): cache_key = f"user:profile:{user_id}" cached_data = redis_client.get(cache_key)

if cached_data:
    return json.loads(cached_data)

user = db.query(User).filter(User.id == user_id).first() if not user: raise HTTPException(status_code=404, detail="User not found")

user_data = {"id": user.id, "name": user.name, "email": user.email}
redis_client.setex(cache_key, 300, json.dumps(user_data))
return user_data

REPLACE

Aider applied the patch automatically, ran git diff, and asked if I wanted to commit the changes.

Common Failure Modes and Fixes

During extended testing, three issues occasionally surfaced:

1. Infinite Malformed Diff Loops: The model outputs a diff block, but slightly miscounts spaces in the <<<<<<< SEARCH section.
   • Fix: Run aider --edit-format udiff or explicitly set edit-format: diff in .aider.conf.yml. Avoid using the default whole file output format on local models—it wastes context output tokens and increases generation time.
2. Ollama Memory Eviction (OOM): The GPU runs out of VRAM midway through a request and crashes the daemon.
   • Fix: Lower the context size in the Modelfile from 32768 to 16384 (PARAMETER num_ctx 16384), or lower your GPU's VRAM usage by passing environment variables before starting Ollama:
     bashنسخ

     export OLLAMA_NUM_PARALLEL=1

3. Aider Sends Too Many Files: Adding entire directories fills the context window prematurely.
   • Fix: Use /drop inside the Aider prompt to remove large files once their architectural context is no longer needed, keeping only active targets loaded.

Frequently Asked Questions

Why does Aider fail to edit files when I use the standard qwen2.5-coder Ollama tag?

The default model tag in Ollama uses a 2,048-token context window. Aider sends the prompt, repo map, and file contents, which immediately exceeds 2k tokens. Ollama truncates the system prompt, causing the model to lose the structural formatting instructions required to edit files. You must create a custom Modelfile setting num_ctx 32768.

Can I run Aider with Ollama on a system without a dedicated GPU?

Yes, but execution will be limited by CPU memory bandwidth. Running qwen2.5-coder:7b on a modern 8-core CPU with system RAM yields roughly 5 to 12 tokens/sec. While usable for small scripts, multi-file refactoring will feel slow compared to running on a system with dedicated VRAM or Apple Silicon.

How do I configure Aider to use a cloud model for Architect and local Ollama for Editing?

Specify both providers in your .aider.conf.yml or CLI parameters. For example:

model: claude-3-5-sonnet-20241022
architect: true
editor-model: openai/qwen2.5-coder-32k
openai-api-base: http://localhost:11434/v1
openai-api-key: ollama

Set your ANTHROPIC_API_KEY environment variable in your terminal. Sonnet will handle high-level architectural planning, while your local Ollama instance executes physical file edits locally.

Should I use qwen2.5-coder:32b or 14b for coding tasks?

If you have 24GB of VRAM or an Apple Silicon Mac with 32GB+ Unified Memory, use the 32B model—its instruction-following accuracy and search/replace format reliability are noticeably superior. If you are limited to 12GB–16GB VRAM, use the 14B model; it generates tokens faster while retaining strong coding logic.