Connect LibreChat to Ollama: Docker Setup Guide

The Architecture and Why Local Networking Breaks

Related: Ollama Flash Attention: Cut VRAM Usage & Boost Speed →

Running LibreChat inside Docker while hosting Ollama directly on your host machine's operating system provides the ideal balance between UI isolation and GPU acceleration. You get the clean containerized ecosystem of LibreChat with its MongoDB and MeiliSearch dependencies, while Ollama maintains raw access to your Metal APIs on macOS or CUDA/ROCm drivers on Linux and Windows.

This hybrid architecture breaks out of the box due to network boundary isolation. When LibreChat runs inside a Docker container, localhost or 127.0.0.1 refers to the container's isolated network loopback interface, not your host operating system. Attempting to point LibreChat to http://localhost:11434 yields an immediate ECONNREFUSED error.

Error: connect ECONNREFUSED 127.0.0.1:11434
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1157:16)

Resolving this requires configuring two network pathways. First, Ollama must listen on all local network interfaces (0.0.0.0) rather than restricting itself to its default local loopback binding (127.0.0.1). Second, Docker must be instructed to map a specific hostname, typically host.docker.internal, to the host gateway IP address.

On macOS and Windows Desktop installations, Docker Desktop manages host.docker.internal automatically. On Linux systems running native Docker Engine, this hostname does not exist unless explicitly defined in your docker-compose.override.yml file using the extra_hosts parameter.

Step 1: Configuring Ollama for External Network Requests

Related: Self-Host Perplexica with Local Ollama: Zero-Cost AI Search →

By default, the Ollama daemon binds exclusively to 127.0.0.1:11434. Requests originating from Docker container subnets like 172.17.0.0/16 or 172.18.0.0/16 are dropped before reaching the model runner. You must force Ollama to listen across all network bridges.

Linux (Systemd Service)

If you installed Ollama using the official Linux shell script, it runs as a systemd service. Modifying environment variables directly in your active shell will not affect the background daemon. You must override the service unit file.

Open the systemd service override editor:

sudo systemctl edit ollama.service

Add the following environment definitions under the [Service] block:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_ORIGINS=*"

Save the file and exit the editor. Reload systemd daemon configs and restart the Ollama service:

sudo systemctl daemon-reload
sudo systemctl restart ollama

macOS Settings

If running Ollama via the macOS menu bar app, quit the application completely. Open your terminal and set the environment variable persistently, or run it through terminal launching:

launchctl setenv OLLAMA_HOST "0.0.0.0:11434"
launchctl setenv OLLAMA_ORIGINS "*"

Restart the Ollama desktop application to inherit these environment changes.

Windows Configuration

Open System Properties, navigate to Environment Variables, and add a new System Variable:

• Variable name: OLLAMA_HOST
• Variable value: 0.0.0.0:11434

Add a second variable to prevent Cross-Origin Resource Sharing (CORS) blocks:

• Variable name: OLLAMA_ORIGINS
• Variable value: *

Restart Ollama from the Windows system tray.

Verifying Network Accessibility

Test whether your host machine is actively serving Ollama across local interfaces by running a curl request against your host machine's internal gateway IP, or simply testing localhost on port 11434:

curl http://127.0.0.1:11434/api/tags

If configured properly, this returns a JSON payload listing your pulled local models.

{"models":[{"name":"llama3.3:latest","modified_at":"2026-02-15T10:14:22.402Z","size":42244833280}]}

Step 2: Crafting the Docker Compose Override

Related: Run CrewAI with Ollama: Fix Crashes & Local Setup Guide →

Modifying the base docker-compose.yml supplied by the official LibreChat repository makes pulling upstream updates difficult. The cleanest approach uses Docker Compose's built-in override mechanism (docker-compose.override.yml).

Create a file named docker-compose.override.yml in the root directory of your LibreChat repository alongside docker-compose.yml.

version: '3.4'

services: api: extra_hosts: - "host.docker.internal:host-gateway" environment: - OLLAMA_BACKEND_URL=http://host.docker.internal:11434

This short block delivers two critical instructions:

1. The extra_hosts mapping injects an entry into the /etc/hosts file inside the api container, mapping host.docker.internal directly to the host system's bridge IP address (typically 172.17.0.1 on Linux).
2. The OLLAMA_BACKEND_URL environment variable provides LibreChat with the direct host connection route.

To apply this change without tearing down database volumes, run:

docker compose down
docker compose up -d

Confirm that the route is resolvable from inside the container by running an exec command directly into the running LibreChat API container:

docker compose exec api ping -c 2 host.docker.internal

If you receive packet responses from your host interface IP, your Docker container network bridge is properly constructed.

Step 3: Building a Rock-Solid librechat.yaml Config

Related: Connect Avante.nvim to Local Ollama: 2026 Neovim Guide →

LibreChat provides a dedicated configuration file (librechat.yaml) to handle custom endpoints, model parameters, context windows, and model filtering. Create or edit librechat.yaml in your project root.

Here is a battle-tested configuration that connects LibreChat to Ollama, enforces strict context limits to prevent VRAM out-of-memory crashes, and exposes custom parameters for advanced local models like Llama 3.3, Qwen 2.5 Coder, and DeepSeek R1.

version: 1.1.5

cache: true

endpoints: ollama: # Points to Ollama via host gateway baseURL: "http://host.docker.internal:11434"

# Enable fetching models dynamically from Ollama's /api/tags endpoint
fetch: true

# Title generation settings
titleConvo: true
titleModel: "qwen2.5-coder:7b"
titleMethod: "completion"

# Model list configuration
models:
  default:
    - "llama3.3:70b"
    - "qwen2.5-coder:32b"
    - "deepseek-r1:14b"
  fetch: true

Global overrides across all Ollama models

streamRate: 50

# Override custom parameters for specific models
modelDisplayLabel: "Local Ollama Engine"

Parameter Overrides

modelParameters:

• name: "llama3.3:70b" reset: true params: num_ctx: 32768 temperature: 0.6 top_p: 0.9 repeat_penalty: 1.1

• name: "qwen2.5-coder:32b" reset: true params: num_ctx: 16384 temperature: 0.2 top_p: 0.95

• name: "deepseek-r1:14b" reset: true params: num_ctx: 32768 temperature: 0.7

Key Parameter Definitions

• num_ctx: Controls the context window size allocated in VRAM. Ollama defaults to a tiny 2048 context unless explicitly overridden here or in a Modelfile. Bumping this to 32768 (32k) requires significantly more VRAM, so scale this number based on your available hardware.
• fetch: true: Automatically scans your local Ollama instance (ollama list) and adds newly pulled models directly to the LibreChat model selector UI without requiring a server restart.
• reset: true: Ensures that parameters specified in librechat.yaml override any defaults hardcoded into the original GGUF model files.

Restart the API container to re-parse librechat.yaml:

docker compose restart api

Comparing Connectivity Approaches: Host Native vs Containerized Ollama

Related: LiteLLM Proxy Local Ollama Setup: Unified AI Gateway →

Choosing how to run Ollama relative to LibreChat dictates your operational efficiency, inference speed, and maintenance complexity. Below is a breakdown based on empirical testing across macOS (M-series) and Linux CUDA systems.

Host-native Ollama combined with containerized LibreChat remains the best option for local development setups. It prevents unnecessary driver passthrough overhead while keeping web dependencies contained.

Troubleshooting Common Tripwires and Error Codes

Related: Connect AnythingLLM to Local Ollama: 2026 Setup Guide →

Even with clean configuration files, small system differences cause connection failures. Here is how to fix the four most frequent post-setup bugs.

1. connect EHOSTUNREACH host.docker.internal:11434

This error indicates that Docker cannot resolve the host IP, or your host firewall is blocking requests coming from the Docker network interface (br-xxxxxxx or docker0).

sudo ufw allow in on docker0 to any port 11434 proto tcp

If using Red Hat / Fedora Enterprise Linux with firewalld:

sudo firewall-cmd --zone=trusted --add-interface=docker0 --permanent
sudo firewall-cmd --reload

2. Models Load Extremely Slowly or Output Garbage Characters

This happens when LibreChat defaults to an incompatible context window size, causing Ollama to offload layers from VRAM to system RAM (swap paging).

nvidia-smi
# Or on macOS
powermetrics --samplers cpu_power,gpu_power

If VRAM spikes to 100% and text output drops below 5 tokens per second, reduce the num_ctx setting in your librechat.yaml file from 32768 down to 8192 or 16384.

3. Ollama Models Disappear After LibreChat Restarts

If you pulled a model via terminal (ollama pull deepseek-r1:14b), but LibreChat fails to list it in the frontend dropdown, the model cache in LibreChat is stale.

curl -X POST http://localhost:3080/api/models/refresh

4. CORS Errors in Web Console (Access-Control-Allow-Origin)

If the LibreChat interface renders model lists but fails during streaming completions with a generic network error in the browser console, Ollama rejected the request header.

Frequently Asked Questions

How do I enable thinking/reasoning outputs for DeepSeek R1 models?

DeepSeek R1 reasoning models output raw thinking steps encased in <think>...</think> tags. LibreChat natively supports rendering these blocks. Ensure you are running LibreChat v0.7.6 or newer, and set num_ctx to at least 16384 in your librechat.yaml file so the thinking tokens do not consume the entire output budget before answering.

Can I run LibreChat and Ollama on two completely separate computers on my LAN?

Yes. Replace host.docker.internal in your docker-compose.override.yml or librechat.yaml with the actual local IP address of your dedicated Ollama machine (e.g., http://192.168.1.150:11434). Ensure the host machine running Ollama has OLLAMA_HOST=0.0.0.0 set and its local OS firewall accepts incoming connections on TCP port 11434.

Why does LibreChat show "Model Not Found" even when ollama list shows it?

Ollama model tags are case-sensitive and must match full names including tag suffixes. For example, pointing LibreChat to llama3.3 instead of llama3.3:70b or llama3.3:latest can cause a 404 response. Run ollama list in your host terminal, copy the exact string in the NAME column, and paste it directly into your librechat.yaml model list.

How do I pass custom system prompts to Ollama models through LibreChat?

System prompts can be configured directly inside the LibreChat UI per conversation, saved as custom presets, or defined system-wide inside LibreChat's agent builder. If you prefer hardcoding prompts directly at the model level, create a custom Modelfile in Ollama on your host machine using the SYSTEM parameter and point LibreChat to that new custom model alias.