Run Dify Locally with Ollama: Step-by-Step Setup Guide

Q: Why can't I use `http://localhost:11434` as the Server URL inside Dify?

When Dify runs inside Docker containers, `localhost` resolves to the container's isolated local network interface rather than your host computer. Because Ollama runs directly on your host operating system to access GPU resources, using `localhost` forces the container to look for Ollama inside itself, causing a `Connection Refused` error. Using `host.docker.internal` or `172.17.0.1` routes the traffic out of the container back to the host machine.

Q: How do I configure local embedding models for RAG workflows in Dify?

Pull an embedding model like `nomic-embed-text` via Ollama (`ollama pull nomic-embed-text`). Then go to Dify Settings -> Model Provider -> Ollama -> Add Model. Set the model type to Text Embedding, enter `nomic-embed-text` as the model name, and supply your local server URL (`http://host.docker.internal:11434`). Once connected, you can select this embedding model in your Dify Knowledge Base settings.

Connect Dify to local Ollama without network errors. Fix host.docker.internal traps, configure Docker Compose, and launch local LLMs fast.

Tidqom Editorial

August 24, 2026 · 5 min read

Run Dify Locally with Ollama: Step-by-Step Setup Guide

The Reality of Self-Hosting Dify with Local Ollama

Pairing Dify’s workflow orchestration engine with Ollama’s local inference runtime gives you a complete, privacy-focused AI stack running entirely on your own hardware. You get visual agent building, vector database integrations, prompt evaluation, and complex RAG workflows without paying per-token API fees or leaking sensitive data to cloud endpoints.

However, setting up this stack often runs into networking issues. Dify runs inside a cluster of Docker containers (dify-api, dify-web, dify-worker), while Ollama usually runs as a native system process directly on your host operating system to access GPU acceleration (Apple Silicon Metal, CUDA, or ROCm).

When you try to connect Dify to local Ollama using http://localhost:11434, the connection fails immediately. Inside a Docker container, localhost refers to the container itself, not your host computer. Resolving this issue requires configuring loopback interfaces, environment variables, and Docker network routing.

terminal

+-------------------------------------------------------------------+
|                           HOST MACHINE                            |
|                                                                   |
|   +-----------------------------------------------------------+   |
|   |                       Ollama Process                      |   |
|   |                 Listens on 0.0.0.0:11434                  |   |
|   |            Accesses GPU / Metal / Apple Silicon           |   |
|   +-----------------------------------------------------------+   |
|                                 ^                                 |
|                                 | (Host Gateway / Bridge)         |
|                                 v                                 |
|   +-----------------------------------------------------------+   |
|   |                     Docker Engine                         |   |
|   |                                                           |   |
|   |   +------------------+             +------------------+   |   |
|   |   |   dify-api       |             |   dify-worker    |   |   |
|   |   |   Container      |             |   Container      |   |   |
|   |   +------------------+             +------------------+   |   |
|   +-----------------------------------------------------------+   |
+-------------------------------------------------------------------+

This guide covers the entire setup process. You will set up Ollama for network exposure, launch Dify via Docker Compose, fix host networking bottlenecks across macOS, Linux, and Windows WSL2, and register local models inside the Dify UI.

Hardware Baseline and System Requirements

Running a self-hosted Dify instance alongside local LLMs requires sufficient RAM and VRAM. Dify’s background services (PostgreSQL, Redis, Weaviate/Qdrant, API, Worker, and Web UI) consume roughly 3 GB to 4.5 GB of system RAM without active workload processing. Ollama’s resource consumption depends on the model size you load.

Minimum Hardware Profile (Small Models)

Target Models: Llama-3.2-3B, Qwen2.5-7B, DeepSeek-R1-Distill-Qwen-8B (4-bit quantization)
System RAM: 16 GB
VRAM / Unified Memory: 8 GB dedicated VRAM or Apple Silicon Unified Memory
Storage: 30 GB free NVMe SSD storage

Recommended Hardware Profile (Production / Complex RAG)

Target Models: Llama-3.3-70B (Q4), Qwen2.5-32B, Nomic-Embed-Text
System RAM: 32 GB to 64 GB
VRAM / Unified Memory: 16 GB to 24 GB+ dedicated VRAM (e.g., NVIDIA RTX 4090, Apple M2/M3/M4 Max)
Storage: 100 GB+ NVMe SSD storage

Software Dependencies

Ensure you have installed the following software versions before starting:

Ollama: v0.5.0 or newer
Docker Desktop / Engine: v24.0.0 or newer
Docker Compose: v2.20.0 or newer
Git: Installed and available in your system path

Step 1: Configuring Ollama for Docker Access

By default, Ollama binds its HTTP API to 127.0.0.1:11434. This local loopback binding drops incoming connections originating outside localhost, including traffic routed from Docker bridge networks.

To connect Dify to local Ollama, you must instruct Ollama to bind to 0.0.0.0 (all network interfaces) and explicitly allow Cross-Origin Resource Sharing (CORS) requests originating from Dify's containers.

Configuring Ollama on macOS

On macOS, Ollama typically runs as a background menu bar application. System-wide environment variables must be passed using launchctl.

Quit the Ollama application from the top menu bar.
Open your terminal and run the following commands to configure host binding and CORS:

bash

launchctl setenv OLLAMA_HOST "0.0.0.0"
launchctl setenv OLLAMA_ORIGINS "*"

Relaunch the Ollama application from your Applications folder.

To make these variables persist across system reboots, add them to your shell configuration file (~/.zshrc or ~/.bashrc):

bash

echo 'export OLLAMA_HOST="0.0.0.0"' >> ~/.zshrc
echo 'export OLLAMA_ORIGINS="*"' >> ~/.zshrc

Configuring Ollama on Linux (systemd)

If Ollama is installed as a system service on Linux, update its systemd configuration file.

Edit the systemd service override file:

bash

sudo systemctl edit ollama.service

Add the following lines under the [Service] block:

Advertisement — In Article

ini

[Service]
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_ORIGINS=*"

Save the file, reload systemd units, and restart the service:

bash

sudo systemctl daemon-reload
sudo systemctl restart ollama

Configuring Ollama on Windows (WSL2 / Native)

Quit Ollama from the Windows taskbar system tray.
Open Control Panel -> System and Security -> System -> Advanced system settings.
Click Environment Variables.
Under User variables, create two new entries:
- Variable: OLLAMA_HOST | Value: 0.0.0.0
- Variable: OLLAMA_ORIGINS | Value: *
Restart Ollama from the Windows Start menu.

Verifying Host Exposure

Verify that Ollama accepts non-loopback connections by querying its tags endpoint using your computer's local network IP or local hostname:

bash

curl http://127.0.0.1:11434/api/tags

If configured correctly, the server responds with an HTTP 200 status code and a JSON string listing your pulled models.

Pull a basic model to use for testing during configuration:

bash

ollama pull llama3.2
ollama pull nomic-embed-text

Step 2: Deploying Dify via Docker Compose

With Ollama configured to accept external traffic, clone the Dify repository and initialize the container suite.

Clone the official repository and move into the docker directory:

bash

git clone https://github.com/langgenius/dify.git
cd dify/docker

Create your environmental configuration file from the provided template:

bash

cp .env.example .env

Launch the container stack in detached mode:

bash

docker compose up -d

The system will download and run several services:

text

[+] Running 11/11
 ✔ Network dify_default          Created                               0.1s
 ✔ Container dify-db-1           Started                               0.5s
 ✔ Container dify-redis-1        Started                               0.5s
 ✔ Container dify-sandbox-1      Started                               0.4s
 ✔ Container dify-ssrf_proxy-1   Started                               0.4s
 ✔ Container dify-weaviate-1     Started                               0.5s
 ✔ Container dify-api-1          Started                               0.8s
 ✔ Container dify-worker-1       Started                               0.8s
 ✔ Container dify-web-1          Started                               0.9s
 ✔ Container dify-nginx-1        Started                               1.1s

Verify that all containers are healthy by checking their running status:

bash

docker compose ps

Once running, access the Dify web console by opening http://localhost in your browser. Complete the initial setup by creating your administrator account credentials.

terminal

+------------------------------------------------------------------------+
| Initial Administrator Setup                                            |
+------------------------------------------------------------------------+
| Email Address: admin@yourdomain.local                                 |
| Password:      ****************                                        |
| Set Username:  Admin                                                   |
+------------------------------------------------------------------------+

Step 3: Solved: The Docker Networking Traps

Connecting Dify inside Docker to local Ollama on the host requires selecting the right network route for your operating system. Using the wrong network target produces the following standard API error inside Dify:

text

ConnectError: Failed to establish a new connection: [Errno 111] Connection refused

Advertisement — In Article

Here is how to resolve this networking trap based on your platform architecture.

terminal

         +-------------------------------------------------------+
         |               OS & Networking Matrix                  |
         +-------------------------------------------------------+
         | Platform          | Base URL Destination              |
         +-------------------+-----------------------------------+
         | macOS (Desktop)   | http://host.docker.internal:11434 |
         | Windows (Desktop) | http://host.docker.internal:11434 |
         | Linux (Engine)    | http://172.17.0.1:11434           |
         | Linux (Bridge)    | http://host.docker.internal:11434*|
         +-------------------+-----------------------------------+
         * Requires extra_hosts entry in docker-compose.yaml

Scenario A: macOS and Windows Docker Desktop

Docker Desktop automatically manages an internal DNS entry that maps host.docker.internal to the host machine's internal loopback interface.

Target Base URL: http://host.docker.internal:11434

If you encounter connection drops on Docker Desktop, verify that host networking resolution is enabled in your configuration. Open Docker Desktop Settings -> General -> ensure "Use Rosetta for x86/amd64 emulation on Apple Silicon" (if applicable) and "Enable host.docker.internal" settings are active.

Scenario B: Linux Native Docker Engine

Native Linux Docker installations do not resolve host.docker.internal by default. You have two options to fix this issue:

Option 1: Use the Default Gateway Bridge IP

By default, Docker's docker0 network bridge interface assigns the IP address 172.17.0.1 to the host machine.

Target Base URL: http://172.17.0.1:11434

To verify your specific Docker host bridge IP on Linux, execute:

bash

ip addr show docker0 | grep inet

Look for the IP address in the output:

text

inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0

Option 2: Map host.docker.internal in Docker Compose (Recommended)

To maintain structural consistency across development platforms, map host.docker.internal explicitly inside your Dify Docker Compose setup.

Open dify/docker/docker-compose.yaml in your editor. Locate the api and worker service blocks and insert the extra_hosts mapping:

yaml

services:
  api:
    image: langgenius/dify-api:0.15.3
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      - OLLAMA_API_BASE_URL=http://host.docker.internal:11434

worker: image: langgenius/dify-api:0.15.3 extra_hosts: - "host.docker.internal:host-gateway"

terminal

Apply the updated YAML structure by running:

bash

docker compose up -d

Docker Networking Matrix Comparison

Operating System	Recommended Endpoint	Host Mapping Strategy	Required Ollama Config	Primary Failure Mechanism
macOS (Apple Silicon / Intel)	`http://host.docker.internal:11434`	Built-in Docker Desktop DNS	`OLLAMA_HOST=0.0.0.0`	Binding limited to `127.0.0.1`
Windows 11 (WSL2 Backend)	`http://host.docker.internal:11434`	Built-in Docker Desktop DNS	`OLLAMA_HOST=0.0.0.0`	Firewall blocking WSL bridge
Linux (Ubuntu/Debian Native)	`http://172.17.0.1:11434`	Direct Docker `docker0` IP	`OLLAMA_HOST=0.0.0.0`	`host.docker.internal` unmapped
Linux (Docker Compose Extra)	`http://host.docker.internal:11434`	`host-gateway` in compose.yaml	`OLLAMA_HOST=0.0.0.0`	Omission of `extra_hosts` keys

Step 4: Connecting Ollama to the Dify UI

Now that the system services and network routes are configured, you can register your local Ollama models in the Dify web dashboard.

Adding an LLM Model

Log into your local Dify interface (http://localhost).
Click your user avatar in the top-right corner and select Settings.
On the left navigation pane, select Model Provider.
Scroll down to the available providers and locate Ollama. Click Add Model or Configure.

terminal

+------------------------------------------------------------------------+
| Configure Ollama Model Provider                                        |
+------------------------------------------------------------------------+
| Model Type:           [ LLM                                        v ] |
| Model Name:           [ llama3.2                                     ] |
| Server URL:           [ http://host.docker.internal:11434           ] |
| Mode:                 [ Chat                                       v ] |
| Context Window:       [ 8192                                         ] |
| Max Token Limit:      [ 4096                                         ] |
+------------------------------------------------------------------------+
| [ Cancel ]                                             [ Save Setup ]  |
+------------------------------------------------------------------------+

Fill in the model configuration fields:
- Model Type: Select LLM.
- Model Name: Enter the exact model string pulled in Ollama (e.g., llama3.2 or deepseek-r1:14b).
- Server URL: Enter http://host.docker.internal:11434 (macOS/Windows/Linux with extra_hosts) or http://172.17.0.1:11434 (native Linux default bridge).
- Model Architecture: Select Chat.
- Context Window: Set to 8192 (or your model's maximum context length).
- Max Token Limit: Set to 4096.
Click Save. Dify will send a test payload to the Ollama endpoint. If the connection succeeds, the modal closes and displays a green status indicator.

Adding an Embedding Model

To build vector databases, power RAG applications, and ingest documents using your local machine, register a local text embedding model:

Advertisement — In Article

Under the Ollama provider configuration, click Add Model.
Fill in the embedding configuration parameters:
- Model Type: Select Text Embedding.
- Model Name: Enter nomic-embed-text.
- Server URL: Enter http://host.docker.internal:11434 (or http://172.17.0.1:11434).
Click Save.

bash

# Test command to confirm embedding compatibility via CLI
curl http://localhost:11434/api/embeddings -d '{
  "model": "nomic-embed-text",
  "prompt": "Testing local vector embeddings."
}'

Performance Benchmarks and Optimization

Running local models alongside Dify's background workers consumes substantial system resources. Adjusting system settings helps maximize token generation speed and keep memory usage stable.

terminal

+------------------------------------------------------------------------+
|                       INFERENCE THROUGHPUT BENCHMARK                   |
|                        (Tokens per Second - Higher is Better)          |
+------------------------------------------------------------------------+
| M3 Max (64GB)    | ██████████████████████████████ 48 t/s (llama3.2-8b)  |
| RTX 4090 (24GB)  | ████████████████████████████████████████ 68 t/s       |
| RTX 3080 (10GB)  | ██████████████ 22 t/s                              |
| CPU Only (i9)    | █ 3.5 t/s                                            |
+------------------------------------------------------------------------+

Concurrency Tuning (`OLLAMA_NUM_PARALLEL`)

By default, Ollama processes one request at a time. If Dify sends concurrent execution requests—such as extracting metadata while fetching context fragments—subsequent operations are queued.

To process multiple concurrent requests, set the parallel execution variable before launching Ollama:

bash

# macOS Terminal
launchctl setenv OLLAMA_NUM_PARALLEL "4"

Linux systemd (/etc/systemd/system/ollama.service.d/override.conf)

Environment="OLLAMA_NUM_PARALLEL=4"

terminal

Note: Increasing parallel streams scales VRAM consumption linearly based on context context sizes.

Memory Optimization (`OLLAMA_KEEP_ALIVE`)

To prevent Ollama from repeatedly loading and unloading models from VRAM when Dify makes intermittent API calls, set the background model retention window:

bash

# Keeps loaded model parameters in VRAM for 24 hours
launchctl setenv OLLAMA_KEEP_ALIVE "24h"

Dify Environment Adjustments

In heavy multi-agent workflows, long responses generated by local models might trigger background task timeouts in Dify's worker services.

Open dify/docker/.env and update the proxy and HTTP client timeout values:

env

# Increase internal request timeout limits (Values in Seconds)
CONSOLE_API_TIMEOUT=360
API_TIMEOUT=360
HTTP_REQUEST_TIMEOUT=360

Apply these settings by restarting your Docker containers:

bash

docker compose restart api worker

Troubleshooting Common Errors

Error 1: "Model not found" on Save

text

ProviderResponseError: Model llama3.2 not found, try pulling it first

Cause: The model name string entered in the Dify settings panel does not match the tag returned by ollama list.
Fix: Open your host terminal, run ollama list, and verify the exact name. If the tag shows llama3.2:latest, enter llama3.2:latest or pull llama3.2 without additional tags.

Error 2: "CORS origin blocked"

text

Access to fetch at 'http://host.docker.internal:11434' has been blocked by CORS policy

Cause: Ollama is refusing HTTP connections originating from Dify's internal web client headers.
Fix: Confirm OLLAMA_ORIGINS="*". Restart the underlying system daemon (sudo systemctl restart ollama or quit and reopen the macOS app).

Error 3: "Connection Refused (111)" on Linux

text

Failed to connect to host.docker.internal port 11434: Connection refused

Cause: The Linux host lacks host resolution entries inside the isolated container namespace.
Fix: Use http://172.17.0.1:11434 directly or add extra_hosts: - "host.docker.internal:host-gateway" under both api and worker service blocks in your docker-compose.yaml file.

Frequently Asked Questions

Why can't I use `http://localhost:11434` as the Server URL inside Dify?

When Dify runs inside Docker containers, localhost resolves to the container's isolated local network interface rather than your host computer. Because Ollama runs directly on your host operating system to access GPU resources, using localhost forces the container to look for Ollama inside itself, causing a Connection Refused error. Using host.docker.internal or 172.17.0.1 routes the traffic out of the container back to the host machine.

Do I need to run Ollama inside Docker alongside Dify?

It is usually better to run Ollama directly on your host operating system instead of inside a Docker container. Running Ollama natively gives it direct access to host GPU drivers, such as Apple Silicon Metal, NVIDIA CUDA, or AMD ROCm, without requiring complex GPU passthrough configurations in Docker. Native installation delivers higher token throughput and lower latency.

How do I configure local embedding models for RAG workflows in Dify?

Pull an embedding model like nomic-embed-text via Ollama (ollama pull nomic-embed-text). Then go to Dify Settings -> Model Provider -> Ollama -> Add Model. Set the model type to Text Embedding, enter nomic-embed-text as the model name, and supply your local server URL (http://host.docker.internal:11434). Once connected, you can select this embedding model in your Dify Knowledge Base settings.

How do I fix high latency or sluggish execution when processing prompts?

Slow performance is usually caused by insufficient hardware resources or improper setup. Ensure your host machine has enough free RAM and VRAM to load the model without relying on system swap storage. To prevent performance drops from model reload cycles, set OLLAMA_KEEP_ALIVE=24h. Additionally, match your model selection to your hardware—use 3B to 8B parameter models for systems with 8 GB to 16 GB of VRAM, and reserve 32B+ parameter models for hardware configurations with 24 GB+ of VRAM.

مواضيع مقترحة · Suggested Topics

استكشف مواضيع ومحاور ذات صلة بهذا المقال — روابط داخلية لتعميق قراءتك.

Topics

#Connect Dify To Local Ollama #Dify Ollama Docker Setup #Dify Host Docker Internal Ollama #Self Host Dify Local Llm #Dify Docker Compose Ollama #Local Llm Orchestration

Explore

More in AI Tools →All articles Trending now

The Daily Pulse

Newsletter delivery is not connected yet. This form only saves your address in this browser; no email is sent.

Get concise, source-linked technology notes without the hype.

Run Dify Locally with Ollama: Step-by-Step Setup Guide

The Reality of Self-Hosting Dify with Local Ollama

Hardware Baseline and System Requirements

Minimum Hardware Profile (Small Models)

Recommended Hardware Profile (Production / Complex RAG)

Software Dependencies

Step 1: Configuring Ollama for Docker Access

Configuring Ollama on macOS

Configuring Ollama on Linux (systemd)

Configuring Ollama on Windows (WSL2 / Native)

Verifying Host Exposure

Step 2: Deploying Dify via Docker Compose

Step 3: Solved: The Docker Networking Traps

Scenario A: macOS and Windows Docker Desktop

Scenario B: Linux Native Docker Engine

Option 1: Use the Default Gateway Bridge IP

Option 2: Map host.docker.internal in Docker Compose (Recommended)

Docker Networking Matrix Comparison

Step 4: Connecting Ollama to the Dify UI

Adding an LLM Model

Adding an Embedding Model

Performance Benchmarks and Optimization

Concurrency Tuning (OLLAMA_NUM_PARALLEL)

Linux systemd (/etc/systemd/system/ollama.service.d/override.conf)

Memory Optimization (OLLAMA_KEEP_ALIVE)

Dify Environment Adjustments

Troubleshooting Common Errors

Error 1: "Model not found" on Save

Error 2: "CORS origin blocked"

Error 3: "Connection Refused (111)" on Linux

Frequently Asked Questions

Why can't I use http://localhost:11434 as the Server URL inside Dify?

Do I need to run Ollama inside Docker alongside Dify?

How do I configure local embedding models for RAG workflows in Dify?

How do I fix high latency or sluggish execution when processing prompts?

مواضيع مقترحة · Suggested Topics

The Daily Pulse

Concurrency Tuning (`OLLAMA_NUM_PARALLEL`)

Memory Optimization (`OLLAMA_KEEP_ALIVE`)

Why can't I use `http://localhost:11434` as the Server URL inside Dify?