I Replaced Copilot With a Fully Offline AI Assistant in VS Code

The reason was boring and practical: a client contract that forbids sending source code to third-party services. Copilot was out. This is what replaced it.

What you install

Related: GPT-5 Is Here: Everything You Need to Know About OpenAI's Most Powerful Model Yet →

Related: كيف تبني وكيل ذكاء اصطناعي يعمل على جهازك بدون إنترنت (Ollama + MCP) — جرّبته 3 أسابيع (2026) →

# 1. the runtime
curl -fsSL https://ollama.com/install.sh | sh    # macOS/Linux

2. two models: one for chat, one for completion

ollama pull qwen2.5-coder:7b ollama pull qwen2.5-coder:1.5b-base

Then install the Continue extension from the VS Code marketplace. That is the whole stack.

Two models matter here. Chat can be slow and smart; autocomplete must answer in under 300ms or you will turn it off within a day. The 1.5b base model is for completion only — do not use an instruct model there, it will try to explain your code instead of finishing the line.

The config that actually works

Related: Will AI Coding Agents Replace Developers? We Asked 100 Engineers →

Related: How to Run an AI Model Locally on Your Own Computer (Step-by-Step 2026 Guide) →

Open ~/.continue/config.json:

{
  "models": [
    {
      "title": "Qwen Coder 7B",
      "provider": "ollama",
      "model": "qwen2.5-coder:7b",
      "contextLength": 8192
    }
  ],
  "tabAutocompleteModel": {
    "title": "Autocomplete",
    "provider": "ollama",
    "model": "qwen2.5-coder:1.5b-base"
  },
  "tabAutocompleteOptions": {
    "debounceDelay": 350,
    "maxPromptTokens": 1024,
    "multilineCompletions": "auto"
  },
  "embeddingsProvider": {
    "provider": "ollama",
    "model": "nomic-embed-text"
  }
}

maxPromptTokens: 1024 is the setting people miss. The default sends far more context to the completion model, and on local hardware that is the difference between suggestions appearing as you type and appearing after you have already written the line.

How it compares in daily use

Related: The 27 Best AI Tools in 2026 (Tested for 90 Days) →

Related: Ollama Running Slow? 7 Fixes That Actually Worked on My Machine →

I am not going to pretend it matches Cursor on large refactors across ten files. It does not. For single-file work, writing tests, and explaining unfamiliar code, the difference is small enough that I stopped noticing.

Indexing your codebase

Related: ChatGPT vs Claude 4: Which AI Should You Actually Pay For in 2026? →

Related: How to Run a Local LLM on 8GB of RAM (What Actually Works in 2026) →

Continue builds a local embedding index so chat can reference your own files:

@codebase how does the auth middleware validate tokens?

On a 40k-line repo the first index took about 6 minutes and 300MB of disk. It is stored in ~/.continue/index, entirely local.

Experience log

Related: Google Gemini 3 Ultra Review: Has Google Finally Caught Up? →

Related: I Built a Chatbot for My Own Documents in One Evening (Free, Local, No API Key) →

• Day 1: used the 7B instruct model for autocomplete. Suggestions took 2-4 seconds and arrived as markdown code fences inside my file. Disabled it in an hour.
• Day 2: switched completion to the 1.5b base model. Latency dropped to ~250ms and suggestions became actual code. This was the fix.
• Week 1: context length at 32768 meant every chat message reloaded a huge KV cache. Dropped to 8192; chat became instant and I lost nothing I actually used.
• Week 3: @codebase returned garbage until I added nomic-embed-text as the embeddings provider. The default transformers.js embedder was much weaker on my code.
• Month 2: added .continueignore for node_modules, dist, and lockfiles. Index time went from 6 minutes to 90 seconds.

When I still reach for a hosted model

Related: Midjourney vs DALL-E 4 vs Flux 1.1: The Definitive AI Image Generator Comparison →

Genuinely: unfamiliar frameworks, anything involving very recent library versions, and long multi-file refactors. About 15% of my week. The other 85% runs offline, which is exactly the ratio that makes the contract work.

FAQ

Related: 9 Free AI Coding Tools Every Developer Should Try in 2026 →

Minimum hardware? 8GB VRAM runs both models comfortably. On 8GB unified memory (M1/M2 Air) use qwen2.5-coder:1.5b for chat too — it works, it is just noticeably less capable.

Does it work with JetBrains? Continue ships a JetBrains plugin with the same config file. I have only used it briefly; autocomplete behaved identically.

Can I use it alongside Copilot? Yes, but disable one of the two autocomplete providers or you get duplicate ghost text fighting for the same keystroke.

Which model would you pick today? qwen2.5-coder:7b for chat, its 1.5b base sibling for completion. I tested DeepSeek Coder and CodeLlama against it; Qwen was better at both in my testing, and not by a small margin.