Run OpenHands Locally with Ollama: Step-by-Step Guide

The Reality of Running OpenHands Offline

Related: Enable Ollama Parallel Requests Without OOM Crashes →

OpenHands (formerly OpenDevin) is one of the most capable open-source autonomous AI software engineering agents available. It can read your repository, write code, run terminal commands, execute unit tests, and fix bug tickets without human intervention.

When powered by cloud LLMs like Claude 3.5 Sonnet, OpenHands works surprisingly well. However, routing your entire code base through external API endpoints introduces privacy risks, API rate limits, and mounting token costs. Running a free autonomous AI software engineer local setup gives you complete control over your code privacy and execution environment.

+-------------------------------------------------------------+
|                      Host Machine                           |
|                                                             |
|  +---------------------+        +------------------------+  |
|  | OpenHands Container |        |  Ollama Service        |  |
|  | (Agent + Sandbox)   |        |  (0.0.0.0:11434)       |  |
|  |                     |        |                        |  |
|  |  [Event Loop]       |        |  +------------------+  |  |
|  |  [Tool Engine]  ----+------->|  | Qwen2.5-Coder-14B |  |  |
|  |  [Workspace Sync]   | HTTP   |  | (32k Context)    |  |  |
|  +---------------------+        |  +------------------+  |  |
|                                 +------------------------+  |
+-------------------------------------------------------------+

Running OpenHands 100% locally with Ollama presents unique technical hurdles. Small local models fail in specific, predictable ways: they drop system instructions, hallucinate invalid JSON tool calls, and run into context-window limits that trigger infinite edit loops.

To run an openhands local ollama setup that stays stable, you need precise network routing, proper context window scaling, and explicit configuration tweaks.

Hardware Requirements

Before running OpenHands locally, verify your host machine specs:

• Minimum: 16GB RAM, 8GB VRAM (GPU), 50GB free SSD storage. Suitable for 7B/8B models.
• Recommended: 32GB+ RAM, 16GB+ VRAM (e.g., RTX 3090/4090 or Apple Silicon M2/M3/M4 Pro/Max with Unified Memory). Suitable for 14B to 32B models.
• Storage: Fast NVMe storage is essential. Ollama weights and Docker container layers perform poorly on traditional spinning disks.

Preparing Ollama for OpenHands Orchestration

Related: Connect Claude Code CLI to Local Ollama Models →

By default, Ollama binds its API server to 127.0.0.1:11434. Because OpenHands executes inside an isolated Docker container, a default Ollama setup will reject incoming requests from the container bridge network.

Container (172.17.0.2) --[HTTP Request]--> Host (127.0.0.1:11434) = Connection Refused

You must explicitly configure Ollama to listen on all network interfaces (0.0.0.0) and pull a coding model capable of instruction following and tool usage.

Step 1: Bind Ollama to Network Interfaces

Configure your operating system environment variables to expose the host port to Docker containers.

On Linux (systemd):

Create or edit the service drop-in configuration:

sudo mkdir -p /etc/systemd/system/ollama.service.d
echo '[Service]' | sudo tee /etc/systemd/system/ollama.service.d/override.conf
echo 'Environment="OLLAMA_HOST=0.0.0.0"' | sudo tee -a /etc/systemd/system/ollama.service.d/override.conf
sudo systemctl daemon-reload
sudo systemctl restart ollama

On macOS:

Set the environment variable in your terminal before launching the desktop service, or run:

launchctl setenv OLLAMA_HOST "0.0.0.0"

Restart the Ollama desktop application afterward.

On Windows:

Open System Properties -> Environment Variables -> System Variables, add a new variable with Name OLLAMA_HOST and Value 0.0.0.0, then restart the Ollama taskbar app.

Test the binding from your terminal:

curl http://localhost:11434/api/tags

If you receive a JSON array of your installed models, host binding is working.

Step 2: Select and Prepare the Local LLM

OpenHands requires strong instruction-following capabilities. General-purpose models like Llama-3.1-8B often degrade quickly during multi-turn bash execution.

For the highest success rate, use Qwen2.5-Coder (14B or 32B parameters). It approaches Claude 3.5 Sonnet performance on basic code synthesis and handles tool execution formats reliably.

Run one of these commands based on your available VRAM:

# For 8GB - 12GB VRAM
ollama pull qwen2.5-coder:14b-instruct-q4_K_M

For 16GB - 24GB VRAM

ollama pull qwen2.5-coder:14b-instruct-fp16

For 24GB+ VRAM or 64GB Unified Memory

ollama pull qwen2.5-coder:32b-instruct-q4_K_M

Step 3: Extend the Default Ollama Context Window

This step is critical. Ollama's default context window is 2048 tokens, which is far too small for OpenHands. An open-source agent feeds its system prompt, file system directory trees, bash logs, and history back into the model on every single turn. At 2048 tokens, context fills up instantly, causing truncation errors and infinite token loops.

You must create a custom model file that forces a 32,768 (32k) token context length.

Create a file named Modelfile-openhands:

FROM qwen2.5-coder:14b-instruct-q4_K_M
PARAMETER num_ctx 32768
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"

Build the custom entry in Ollama:

ollama create qwen2.5-coder-32k -f Modelfile-openhands

Verify the new model exists:

ollama list

Launching OpenHands via Docker

Related: Set Up Aider CLI with Ollama and Qwen 2.5 Coder →

To run openhands locally, you launch the official OpenHands Docker container, bind it to your local workspace path, and point its internal LLM library (LiteLLM) at your host machine's Ollama instance.

Create a dedicated directory on your host system to store target projects:

mkdir -p ~/openhands-workspace
cd ~/openhands-workspace

Execute the Docker run command. Replace /path/to/your/workspace with your actual host workspace directory:

docker run -d \
    --name openhands-app \
    -e LLM_MODEL="ollama/qwen2.5-coder-32k" \
    -e LLM_BASE_URL="http://host.docker.internal:11434" \
    -e LLM_API_KEY="ollama" \
    -e LLM_NUM_CTX=32768 \
    -e WORKSPACE_BASE="/path/to/your/workspace" \
    -e LOG_ALL_EVENTS=true \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v "/path/to/your/workspace:/opt/workspace_base" \
    -v ~/.openhands-state:/.openhands-state \
    -p 3000:3000 \
    --add-host=host.docker.internal:host-gateway \
    docker.all-hands.dev/all-hands-ai/openhands:0.14

Breakdown of Critical Flag Configurations

• --add-host=host.docker.internal:host-gateway: Maps host.docker.internal inside Linux containers to the actual Docker host gateway IP address.
• -e LLM_MODEL="ollama/qwen2.5-coder-32k": Tells LiteLLM to treat the endpoint using Ollama's API schema rather than OpenAI's format.
• -v /var/run/docker.sock:/var/run/docker.sock: Allows OpenHands to launch its own sandbox runtime containers inside Docker to execute code safely.

Navigate to http://localhost:3000 in your web browser. You will see the OpenHands user interface.

+-----------------------------------------------------------------------+
|  OpenHands UI (http://localhost:3000)                                 |
|                                                                       |
|  +----------------------------+  +---------------------------------+  |
|  | Task Prompt                |  | Workspace Browser               |  |
|  | "Fix bug in main.py"       |  | /opt/workspace_base             |  |
|  +----------------------------+  |  ├── main.py                    |  |
|  | Agent Status: Active       |  |  └── tests/ test_main.py        |  |
|  +----------------------------+  +---------------------------------+  |
|  | Terminal Execution Logs    |  | LLM Status                      |  |
|  | $ python3 -m pytest        |  | Model: qwen2.5-coder-32k        |  |
|  | FAILED tests/test_main.py  |  | Context: 18.4k / 32.7k tokens   |  |
|  +----------------------------+  +---------------------------------+  |
+-----------------------------------------------------------------------+

Preventing Token Loop Crashes and Infinite Edits

Related: Run Bolt.diy Locally with Ollama: Free v0 Alternative →

When using an openhands local llm opendevin setup, the most common failure mode is an infinite token loop.

The agent runs a terminal command, receives an error, tries to edit a file, reads the file again, and repeats this cycle endlessly until it reaches maximum iterations or crashes the container.

[Agent Action: Read File] ---> [Model Generates Edit]
       ^                                 |
       |                                 v
[Token Context Overflow] <--- [Syntax Error / Loop Triggered]

This occurs due to three main causes:

1. The context window defaults to 2048 or 4096 tokens, causing historical actions to be dropped.
2. The model outputs native function calls that OpenHands' default parser fails to interpret.
3. The context buffer becomes polluted with full test logs and stdout traces.

To fix these issues permanently, apply custom runtime configurations via an explicit config.toml file.

Step 1: Create a Custom config.toml

Inside your host directory ~/.openhands-state, create or update config.toml:

[core]
workspace_base = "/opt/workspace_base"
max_iterations = 30
agent = "CodeActAgent"

[llm] model = "ollama/qwen2.5-coder-32k" base_url = "http://host.docker.internal:11434" api_key = "ollama" max_output_tokens = 4096 max_message_chars = 30000 num_ctx = 32768 temperature = 0.0 top_p = 0.95 native_tool_calling = false custom_llm_provider = "ollama"

[agent] enable_cmd_history = true memory_enabled = false

Why These Tweaks Stop Crashes

• native_tool_calling = false: Forces OpenHands to convert agent actions into standard JSON prompts rather than depending on Ollama's native function-calling parser, which often fails with smaller model weights.
• temperature = 0.0: Prevents creative variance during code editing actions, making tool responses predictable and deterministic.
• max_message_chars = 30000: Truncates massive log files or test output before sending them to the LLM context buffer.

Restart your container after applying changes:

docker restart openhands-app

Model Performance Benchmark: Local LLMs vs. Cloud Engineers

Related: Kokoro TTS in Open WebUI: Zero-Latency Local Voice Setup →

Running OpenHands fully locally involves clear trade-offs in speed, reliability, and accuracy compared to cloud endpoints.

The following benchmark details test runs across various local LLMs executing basic code modification, file editing, and test repairs inside the OpenHands runtime environment (testing done on an RTX 4090 24GB / 64GB System RAM host).

Analysis

• Qwen2.5-Coder-14B is the minimum viable model for local OpenHands tasks. It handles context well and rarely fails tool-calling execution when native_tool_calling = false is applied.
• Qwen2.5-Coder-32B offers the highest success rate among local models, handling complex refactoring across multiple files with minimal intervention.
• Llama-3.1-8B struggles with agent tool execution. It frequently fails to generate valid syntax for file edits, resulting in frequent token crash loops.

Troubleshooting Common OpenHands and Ollama Breakages

1. Connection Refused to host.docker.internal

2. Immediate Token Context Overflow on Second Turn

3. File Permission Errors in Docker Workspace

sudo chown -R 1000:1000 ~/openhands-workspace

4. Agent Infinite Loop: Repeats Command Without Modifying Code

Frequently Asked Questions

Can I run OpenHands locally with only 16GB of VRAM?

Yes. You can comfortably run Qwen2.5-Coder-14B at 4-bit quantization (q4_K_M), which requires roughly 11.2 GB of VRAM. This leaves enough memory for your operating system and host display while providing reliable tool-calling performance.

Why does OpenHands require mounting /var/run/docker.sock?

OpenHands executes code, installs dependencies, and runs unit tests inside a isolated container environment rather than running directly on your host machine. Access to the Docker socket allows OpenHands to spawn and manage these temporary sandbox runtimes safely.

Is Qwen2.5-Coder better than Llama 3.1 for local OpenHands setups?

Yes. Qwen2.5-Coder is significantly better tuned for software development tasks, tool calling, and structured output parsing. In practice, Llama 3.1 models tend to lose context quickly during multi-step bash execution, triggering token loop crashes.

How do I update OpenHands without losing local configuration settings?

Your local configurations and agent settings are saved in ~/.openhands-state on your host machine. To update OpenHands, stop the running container, pull the latest image, and re-run the container pointing to the same state directory:

docker stop openhands-app && docker rm openhands-app
docker pull docker.all-hands.dev/all-hands-ai/openhands:latest