Connect Windsurf IDE to Local Ollama: Step-by-Step Setup

Why I Ditched the Cloud for Local Qwen 2.5 Coder in Windsurf

Related: Fix Open WebUI Web Search: SearXNG Docker Guide →

Cloud AI coding assistants are fast until you hit rate limits, lose internet access on a flight, or get blocked by enterprise security policies against sending internal code to third-party APIs.

I set out to build a completely offline setup inside Windsurf IDE using local open-weight models. I tested this setup on two machines: an Apple M3 Max with 64GB of unified memory and a Linux workstation running a single RTX 4090 with 24GB of VRAM.

+-------------------------------------------------------------------+
|                           Windsurf IDE                            |
|  +---------------------------+   +-----------------------------+  |
|  |     Cascade / Agent       |   |         Inline Chat         |  |
|  +-------------+-------------+   +--------------+--------------+  |
+----------------|--------------------------------|-----------------+
                 | HTTP / OpenAI v1 API Spec      |
                 v (http://127.0.0.1:11434/v1)    v
+-------------------------------------------------------------------+
|                            Ollama Daemon                          |
|  +-------------------------------------------------------------+  |
|  | Models: qwen2.5-coder-32k (14B / 32B Quantized)              |  |
|  | Custom Parameters: num_ctx=32768, temperature=0.2           |  |
|  +-------------------------------------------------------------+  |
+-------------------------------------------------------------------+

The model of choice for this build is Qwen 2.5 Coder. The 14B variant running at Q4_K_M quantization delivers roughly 62 tokens per second on the RTX 4090 and 41 tokens per second on the M3 Max. Its code completion and refactoring accuracy rivals closed cloud models like Claude 3.5 Sonnet for standard TypeScript, Python, and Go tasks.

Connecting Windsurf IDE to a local Ollama instance sounds simple because Windsurf supports custom OpenAI-compatible API endpoints. In practice, default configurations lead to dropped connections, clipped context windows, and infinite loading loops in Cascade mode. Here is how to fix those bottlenecks and configure a seamless offline AI coding environment.

Step 1: Preparing Ollama for External IDE Connections

Related: Open WebUI Ignoring Your Uploaded Documents? Fix It →

By default, Ollama binds to 127.0.0.1:11434 and accepts requests primarily from local CLI sessions. When IDEs like Windsurf query Ollama through internal Node.js or web fetch handlers, strict Cross-Origin Resource Sharing (CORS) rules and host binding limitations can block communication.

First, pull the target model. We will use the 14-billion parameter instruct model of Qwen 2.5 Coder:

ollama pull qwen2.5-coder:14b-instruct-q4_K_M

If you have at least 32GB of VRAM or unified memory, you can pull the larger variant:

ollama pull qwen2.5-coder:32b-instruct-q4_K_M

Next, configure environment variables so Ollama handles incoming requests from Windsurf without dropping connections.

On macOS

If you run Ollama as a macOS application, quit the application completely from the menu bar. Launch it from your terminal with explicit CORS and host settings:

export OLLAMA_ORIGINS="*"
export OLLAMA_HOST="0.0.0.0:11434"
ollama serve

To make this permanent across system reboots, set the environment variables in your launchd agent or add export lines to your ~/.zshrc file if launching from terminal.

On Linux (systemd)

Edit the systemd service file for Ollama:

sudo systemctl edit ollama.service

Add the environment directives under the [Service] block:

[Service]
Environment="OLLAMA_ORIGINS=*"
Environment="OLLAMA_HOST=0.0.0.0:11434"

Save the file and restart the service:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Verifying the Endpoint

Test whether Ollama's OpenAI-compatible endpoint is running correctly by sending a cURL request to the local API compatibility layer:

curl http://127.0.0.1:11434/v1/models

You should receive a JSON response listing your pulled models formatted like this:

{
  "object": "list",
  "data": [
    {
      "id": "qwen2.5-coder:14b-instruct-q4_K_M",
      "object": "model",
      "created": 1735689600,
      "owned_by": "library"
    }
  ]
}

If this returns a valid response, the API listener is ready.

Step 2: Fixing the Default Context Window Bottleneck

Related: Make Ollama Start on Boot: A systemd Service That Works →

The biggest issue with running local models inside an advanced IDE like Windsurf is context length truncation.

By default, Ollama initializes models with a tiny context window of 2,048 tokens (num_ctx 2048). Windsurf constructs large prompts containing file trees, open tabs, and system instruction wrappers. When sent to a model limited to 2,048 tokens, the model loses sight of the original instructions, produces broken code, or hallucinates syntax mid-function.

Qwen 2.5 Coder natively supports up to 128k context, but setting 128k locally will consume extreme amounts of VRAM for the KV cache. A context window of 32,768 tokens (32k) is the sweet spot for local hardware.

We must create a customized Ollama model definition using a custom Modelfile.

Create a file named Modelfile-qwen-32k on your machine:

FROM qwen2.5-coder:14b-instruct-q4_K_M

Set the context length to 32k tokens

PARAMETER num_ctx 32768

Lower temperature slightly for deterministic, precise code output

PARAMETER temperature 0.2

Set context window shift parameters

PARAMETER repeat_penalty 1.1

Now, create the updated model in Ollama:

ollama create qwen2.5-coder-32k -f ./Modelfile-qwen-32k

Verify that the new 32k variant was built:

ollama list

You will see qwen2.5-coder-32k in the output. This model allocation reserves roughly 2.8 GB of extra VRAM for the 32k KV cache on top of the base weights, easily fitting within a 24GB or 64GB memory envelope.

VRAM Allocation Map (RTX 4090 24GB):
+---------------------------------------------------------------+
| Qwen 2.5 Coder 14B Weights (Q4_K_M): ~9.2 GB                  |
+---------------------------------------------------------------+
| KV Cache Allocation (32,768 Context Window): ~2.8 GB          |
+---------------------------------------------------------------+
| Windsurf Workspace Scratchpad & CUDA Overhead: ~1.5 GB       |
+---------------------------------------------------------------+
| Remaining Unallocated VRAM Headroom: ~10.5 GB                 |
+---------------------------------------------------------------+

Step 3: Configuring Custom OpenAI Endpoints in Windsurf

Related: Why Ollama's First Response Is Slow (Cold Start Fix) →

With Ollama configured and the context window extended, we can now hook it into Windsurf IDE.

1. Open Windsurf.
2. Click the Gear icon in the bottom-left corner and open Settings.
3. Search for Custom OpenAI or navigate to Advanced AI Settings.
4. Enable Custom OpenAI Provider.

Fill in the explicit connection parameters:

• Base URL: http://127.0.0.1:11434/v1
• API Key: ollama (Windsurf expects a string value in this field even though local Ollama doesn't require authentication; entering any string prevents client-side validation errors)
• Model Name: qwen2.5-coder-32k

Windsurf Settings Config:
--------------------------------------------------
[x] Enable Custom OpenAI Provider

Base URL: http://127.0.0.1:11434/v1 API Key: ollama Model Name: qwen2.5-coder-32k Override Model: qwen2.5-coder-32k

Important Node.js Networking Fix: Always use http://127.0.0.1:11434/v1 instead of http://localhost:11434/v1. Modern versions of Node.js embedded inside Electron applications resolve localhost to IPv6 (::1) first. If Ollama is listening on IPv4 (127.0.0.1), Windsurf will throw connection timeouts (ECONNREFUSED ::1:11434).

Save the settings and reload the Windsurf window (Cmd+R on macOS, Ctrl+R on Linux/Windows).

Open the Windsurf Cascade agent panel or the Chat panel. In the model selection dropdown, select your newly exposed model endpoint: qwen2.5-coder-32k.

Test the integration by typing a prompt in the chat:

"Write a Go function that reads a JSON payload from an HTTP request body, validates it against a struct, and handles errors with proper status codes."

If the model responds instantly and streams the code directly into the panel, your setup is complete.

Performance Benchmarks: Local Ollama vs Cloud APIs in Windsurf

Related: Running Two Local Models on One GPU Without Crashing →

To evaluate whether running Qwen 2.5 Coder locally makes sense compared to cloud models, I ran performance benchmarks across three configurations inside Windsurf workspace environments.

The test task involved refactoring a 450-line TypeScript file from class-based components to functional components using hooks, followed by running AST code syntax checks.

While Claude 3.5 Sonnet handles complex multi-file architectural refactoring with higher initial accuracy, Qwen 2.5 Coder 14B drastically wins on latency and code-generation speed.

For local edit-and-continue cycles, inline completions, and quick method implementations, the zero-latency experience of running locally on hardware makes coding in Windsurf feel significantly faster.

Troubleshooting Common Connection & Context Breakages

Related: Fix Roo Code Ollama Connection Errors in 5 Steps →

Even with the correct configuration, local setups can occasionally break. Here are the fixes for the issues I encountered during testing.

1. Windsurf Cascade Stuck in Infinite "Thinking" or Tool-Calling Loops

Windsurf's Cascade feature relies on structured system prompts and function calling (JSON tool schema) to edit files and execute terminal commands on your behalf.

Smaller local models (7B and some 14B quantizations) can fail to parse tool call formats cleanly, outputting plain markdown instead of valid JSON tool calls. This causes Cascade to hang waiting for structured data.

Fix:

• Lower the model temperature in your Modelfile to 0.1.
• If using Cascade agent mode, upgrade to the qwen2.5-coder:32b variant, which handles JSON function calling much better than smaller models.
• For standard inline code editing and sidebar chat, switch Windsurf from Cascade mode to standard Chat mode, which does not require strict tool-calling output schemas.

2. Docker Container Port Isolation Errors

If you run Ollama inside a Docker container rather than natively on your host system, Windsurf will fail to connect to 127.0.0.1:11434.

Fix: Ensure your Docker container explicitly maps port 11434 and listens on all network interfaces inside the container:

docker run -d \
  --gpu all \
  -v ollama_storage:/root/.ollama \
  -p 11434:11434 \
  -e OLLAMA_ORIGINS="*" \
  -e OLLAMA_HOST="0.0.0.0:11434" \
  --name ollama \
  ollama/ollama

3. Model Out of Memory (OOM) Crashes on Large Contexts

If your entire GPU freezes or Ollama exits abruptly during a deep codebase query, your GPU run out of available VRAM due to the expanding context window.

Fix: Check your model's context settings. Reduce num_ctx in your custom Modelfile from 32768 to 16384:

PARAMETER num_ctx 16384

Re-create the model with ollama create qwen2.5-coder-16k -f ./Modelfile-qwen-16k and update the base model string inside Windsurf settings.

4. FetchFailed Error / CORS Rejection

If Windsurf displays a red popup reading FetchFailed: Failed to fetch model list, Ollama is dropping requests originating from Electron's internal web views.

Fix: Verify that OLLAMA_ORIGINS is set to *. You can test if host origins are blocking access by curling the headers from your terminal:

curl -I -H "Origin: vscode-webview://windsurf" http://127.0.0.1:11434/v1/models

Look for Access-Control-Allow-Origin: * in the returned headers. If it is missing, restart your Ollama service with the correct environment variables set.

Frequently Asked Questions

Can I use Windsurf Cascade agent mode completely offline with Ollama?

Yes, but with caveats. Windsurf's standard Chat mode works completely offline with local models without issue. Cascade agent mode (which executes terminal commands and modifies multi-file trees automatically) works offline as well, provided you use a model capable of tool-calling schemas like qwen2.5-coder:32b. Smaller models may struggle with tool-use syntax and crash agentic loops.

Which Qwen 2.5 Coder parameter size is best for local Windsurf development?

For machines with 16GB to 24GB of VRAM (such as an RTX 4090, RTX 3090, or M-series Mac with 36GB unified memory), the 14b-instruct-q4_K_M model offers the ideal balance between output speed (~60 tokens/sec) and reasoning quality. If you have 64GB or more of unified memory, run the 32b-instruct-q4_K_M model for better multi-file refactoring accuracy.

Why does Windsurf show "Model not found" even though Ollama is running?

This usually occurs when the model name defined in Windsurf's settings does not match the exact model tag in Ollama. Run ollama list in your terminal, copy the string under the NAME column (e.g., qwen2.5-coder-32k), and paste that exact string into the Model Name field inside Windsurf settings.

How do I reduce VRAM consumption when running 32k context windows locally?

You can lower the VRAM usage of the KV cache by using context quantization if supported by your build, or by reducing the num_ctx parameter inside your custom Modelfile from 32768 to 16384 or 8192. Additionally, closing background applications that consume VRAM (such as web browsers or heavy GPU apps) will prevent out-of-memory errors.