Run CrewAI with Ollama: Fix Crashes & Local Setup Guide

Why Running CrewAI on Local Ollama Hits a Wall (And How to Fix It)

Related: Connect Avante.nvim to Local Ollama: 2026 Neovim Guide →

Connecting CrewAI to local open-weights models through Ollama promises zero API bills, total data privacy, and uncapped agent loops. But if you drop standard Ollama model strings like ollama/llama3.1 into a multi-agent workflow, your terminal will likely turn into a wall of red stack traces.

The core issue stems from how CrewAI was architected versus how local models handle inference. CrewAI relies heavily on JSON-formatted function calls, precise thought-action loops, and persistent scratchpad memory. Cloud endpoints like GPT-4o handle these structural guarantees natively. Smaller local models running through default Ollama configurations fail in three specific ways:

1. JSON Schema Violations: Models prepend conversational text like "Sure! Here is your response:" before outputting the expected tool payload. This breaks CrewAI's output parser and raises an OutputParserError.
2. The 2048-Token Window Limit: Ollama defaults to a context window of 2,048 tokens (num_ctx: 2048) unless explicitly overridden. After two agent turns, CrewAI fills this window with memory scratchpads, causing the model to hallucinate or drop tool syntax mid-execution.
3. Connection Drops under Parallel Execution: When multiple CrewAI agents trigger tasks simultaneously or poll local models concurrently, Ollama rejects requests with connection reset errors (crewai ollama connection error).

Fixing these issues does not require moving back to API keys. It requires fine-tuning Ollama’s inference parameters, overriding CrewAI’s model defaults using LiteLLM syntax, and tuning context allocation.

Step-by-Step Environment Setup for Offline CrewAI Execution

Related: LiteLLM Proxy Local Ollama Setup: Unified AI Gateway →

To use CrewAI offline with local models reliably, you must bypass global environment variables and define structured LLM objects inside your Python code.

1. Configure the Local Ollama Instance

First, verify that your local server runs Ollama 0.3.0 or higher. Open your terminal and verify the environment:

ollama --version

Pull the target models. For agentic workflows with function calling, Qwen 2.5 (14B or 7B) and Llama 3.1 (8B) provide the highest success rates for tool parsing:

ollama pull qwen2.5:14b
ollama pull llama3.1:8b

2. Prepare the Python Virtual Environment

Create an isolated directory and install the necessary package versions. Specific version locks are essential here because CrewAI's local model integration logic shifted substantially after version 0.70.0.

mkdir crewai-ollama-local
cd crewai-ollama-local
python3 -m venv venv
source venv/bin/activate

pip install crewai==0.86.0 crewai-tools==0.14.0 langchain-community==0.3.7

3. Initialize the CrewAI LLM Connector

Instead of passing raw strings to agents, instantiate the explicit LLM class provided by crewai. This passes execution flags straight through LiteLLM to Ollama's API layer.

import os
from crewai import Agent, Crew, Process, Task, LLM

Disable telemetry for total offline execution

os.environ["OTEL_SDK_DISABLED"] = "true" os.environ["CREWAI_TELEMETRY_OPT_OUT"] = "true"

Define local Ollama model instance

local_llm = LLM( model="ollama/qwen2.5:14b", base_url="http://localhost:11434", config=dict( temperature=0.2, num_ctx=16384, # Overrides Ollama's 2k default ) )

Setting temperature=0.2 reduces token randomness, which helps keep the agent aligned with the expected JSON structure for tool invocation.

Resolving JSON Output Parsing Crashes in Multi-Agent Loops

Related: Connect AnythingLLM to Local Ollama: 2026 Setup Guide →

When a local model crashes with an OutputParserError, it means CrewAI attempted to extract a Action: and Action Input: structural block from the raw text response, but the model included conversational filler, broken code blocks, or missing quotes.

Modelfile Overrides for Strict Formatting

The cleanest way to eliminate formatting crashes is to build a custom Modelfile inside Ollama that enforces clean system prompts at the engine level. Create a file named Modelfile.agent:

FROM qwen2.5:14b

Force larger system context window

PARAMETER num_ctx 16384

Control generation output constraints

PARAMETER stop "Thought:" PARAMETER stop "Observation:"

SYSTEM """ You are an execution engine participating in an agentic loop. You MUST respond strictly in the requested JSON format or structured Thought/Action block without introductory chat, code block backticks, or greetings. Never start responses with "Sure", "Here is", or "Okay". """

Compile this custom model into Ollama:

ollama create qwen2.5-agent -f Modelfile.agent

Update your Python script to point to ollama/qwen2.5-agent.

Python-Level Output Parsing Safety

Inside your CrewAI Python code, set explicit retries and agent execution limits to prevent infinite loops when a parsing error occurs:

researcher = Agent(
    role="Senior Data Analyst",
    goal="Extract raw metrics without adding commentary",
    backstory="You are an expert analyst focused purely on data validation.",
    verbose=True,
    allow_delegation=False,
    max_iter=3,  # Prevent endless loops on broken output
    max_retry_limit=2,  # Retries parsing on JSON failure
    llm=local_llm
)

Overcoming Context Window Limits in Local Workflows

Related: Run Dify Locally with Ollama: Step-by-Step Setup Guide →

A common point of failure during long agentic runs is silent context pruning. Ollama defaults to num_ctx=2048. In a standard CrewAI pipeline, the system prompt alone consumes 800 tokens. When an agent executes a tool and reads the output back into its memory scratchpad, it exceeds 2,048 tokens instantly. The model drops early instruction parameters and starts outputting garbage.

Memory Allocation Math for Local GPUs

To run local workflows reliably, you need to match your num_ctx setting against available System VRAM/RAM.

Total Model RAM = Base Model Weights + (num_ctx * Layer Overhead)

Here is how common local models behave when scaling context sizes:

Configuring Extended Contexts Safely

Always match your Python code's context definition with your hardware availability:

# Setup for a machine with 16GB VRAM (e.g., RTX 4090 or Apple M-Series 36GB)
heavy_context_llm = LLM(
    model="ollama/qwen2.5:14b",
    base_url="http://localhost:11434",
    config=dict(
        num_ctx=16384,
        num_thread=8  # Tune CPU threads if offloading layers
    )
)

If VRAM fills up, Ollama automatically offloads remaining transformer layers to system RAM. This prevents out-of-memory crashes, but generation speed will drop from 50+ tokens per second down to 4–6 tokens per second. Watch system RAM usage via nvidia-smi or macOS Activity Monitor to ensure you stay inside your dedicated GPU memory budget.

Fixing the Connection Error ("crewai ollama connection error")

Related: Zed Editor Ollama Setup: Fast Local AI Coding Guide →

If you see httpx.ConnectError: [Errno 111] Connection refused or crewai ollama connection error, your local server is refusing connections due to execution timeouts or concurrent request collisions.

1. Increase Ollama Keep-Alive and Concurrent Request Limits

Ollama unloads models from memory after 5 minutes of inactivity by default. When CrewAI runs an offline workflow with multiple sequential tasks, the model may unload during intermediate data processing, causing a connection failure on the next call.

Set these environment variables before launching the Ollama server:

On Linux / macOS:

export OLLAMA_KEEP_ALIVE="24h"
export OLLAMA_NUM_PARALLEL="2"
ollama serve

On Windows (PowerShell):

$env:OLLAMA_KEEP_ALIVE="24h"
$env:OLLAMA_NUM_PARALLEL="2"
ollama serve

2. Configure Timeout Options in CrewAI

Pass explicit timeout parameters down to LiteLLM inside your script:

local_llm = LLM(
    model="ollama/qwen2.5:14b",
    base_url="http://localhost:11434",
    timeout=300.0,  # 5-minute threshold for slow local generations
    config=dict(
        num_ctx=16384
    )
)

Production-Ready Local CrewAI Script (Walkthrough)

Related: Run LM Studio Headless on Linux: Full CLI Setup Guide →

Here is a functional, end-to-end script designed to execute completely offline. It creates a multi-agent system (Researcher + Technical Writer) using custom tools, explicitly avoids cloud telemetry, handles errors gracefully, and runs over local Ollama inference.

import os
from crewai import Agent, Crew, Process, Task, LLM
from crewai.tools import tool

1. Environment Controls for Complete Privacy

os.environ["OTEL_SDK_DISABLED"] = "true" os.environ["CREWAI_TELEMETRY_OPT_OUT"] = "true"

2. Local Model Configuration

local_llm = LLM( model="ollama/qwen2.5:14b", base_url="http://localhost:11434", timeout=240.0, config=dict( temperature=0.1, num_ctx=16384, ) )

3. Custom Local Tool Definition

@tool("Local System Log Reader") def read_local_logs(log_type: str) -> str: """Reads local system log types: system, network, or security.""" logs = { "system": "STATUS: OK | CPU_LOAD: 14% | MEM_USAGE: 8.2GB/32GB", "network": "STATUS: ALERT | DROPPED_PACKETS: 412 | INTERFACE: eth0", "security": "STATUS: OK | FAILED_LOGINS: 0 | SSH_STATE: ACTIVE" } return logs.get(log_type.lower(), "ERROR: Unknown log domain specified.")

4. Agent Definitions

diagnotician = Agent( role="System Diagnostics Engineer", goal="Analyze local system diagnostic telemetry and identify bottlenecks.", backstory="You process raw hardware metrics into clear system operational assessments.", tools=[read_local_logs], verbose=True, allow_delegation=False, max_iter=3, max_retry_limit=3, llm=local_llm )

report_writer = Agent( role="Technical Incident Documentation Specialist", goal="Convert raw diagnostics into markdown operational summaries.", backstory="You produce concise technical summaries for infrastructure teams.", verbose=True, allow_delegation=False, max_iter=3, max_retry_limit=3, llm=local_llm )

5. Task Definitions

analysis_task = Task( description="Query the 'network' system log and identify operational issues.", expected_output="Detailed breakdown of the network status metrics.", agent=diagnotician )

report_task = Task( description="Take network metrics and write an incident warning alert.", expected_output="Markdown formatted operational incident summary.", agent=report_writer )

6. Instantiate and Execute Crew

system_monitoring_crew = Crew( agents=[diagnotician, report_writer], tasks=[analysis_task, report_task], process=Process.sequential, verbose=True )

if name == "main": print("--- STARTING OFFLINE WORKFLOW ---") results = system_monitoring_crew.kickoff() print("\n--- WORKFLOW EXECUTION COMPLETE ---") print(results)

Output Validation and Log Analysis

When you execute this script, verbose=True prints the real-time thought loop to your console. Look for clean state changes like this:

[2026-03-30 10:14:22][INFO]: Agent System Diagnostics Engineer Action: Local System Log Reader
[2026-03-30 10:14:22][INFO]: Action Input: {"log_type": "network"}
[2026-03-30 10:14:23][INFO]: Tool Output: STATUS: ALERT | DROPPED_PACKETS: 412 | INTERFACE: eth0

If the agent formats the input as raw strings instead of standard JSON, lower the temperature parameter to 0.0 or switch from llama3.1 to qwen2.5:14b, which natively handles structured tool inputs more reliably.

Frequently Asked Questions

How do I fix the "crewai ollama connection error" or connection refused error?

This happens when Ollama unloads the model due to inactivity or drops requests under load. Set OLLAMA_KEEP_ALIVE="24h" in your environment before starting Ollama, increase the timeout parameter in your CrewAI LLM definition to 300.0, and set OLLAMA_NUM_PARALLEL="2" if using multi-agent loops.

Why does my local CrewAI agent get stuck in an infinite tool-calling loop?

Infinite loops occur when the local model output fails to match CrewAI's output parsing format, causing CrewAI to retry the prompt automatically. Set max_iter=3 and max_retry_limit=2 on your Agent objects, and lower your model's generation temperature (temperature=0.1).

Can I run CrewAI completely offline without sending any metadata to third parties?

Yes. CrewAI attempts to transmit anonymous telemetry by default. Set os.environ["OTEL_SDK_DISABLED"] = "true" and os.environ["CREWAI_TELEMETRY_OPT_OUT"] = "true" at the top of your script to block outbound network requests entirely.

Which local model size is the minimum requirement for reliable multi-agent workflows?

The minimum viable size is an 8B parameter model like Llama 3.1 8B or Qwen 2.5 7B. For reliable tool calling and low JSON crash rates, use Qwen 2.5 14B or larger with a custom context window of at least 8,192 tokens (num_ctx=8192).