Kokoro TTS in Open WebUI: Zero-Latency Local Voice Setup

The Local Speech Shift: Why Kokoro-82M Outperforms Legacy Engines

Related: Connect Windsurf IDE to Local Ollama: Step-by-Step Setup →

Cloud-based Text-to-Speech (TTS) services like ElevenLabs or OpenAI Audio deliver high-quality audio, but they introduce unacceptable latency and recurring costs for local LLM workflows. In an interactive chat session, waiting 1,200 to 2,500 milliseconds just for the cloud audio generation round-trip kills the flow of conversation.

Historically, self-hosted alternatives forced a tough compromise:

• Piper TTS: Incredibly lightweight and fast on CPU, but the speech synthesis sounds noticeably robotic and flat.
• XTTS v2: Expressive and rich, but requires a massive GPU memory footprint (over 2GB VRAM reserved exclusively for audio), runs slowly on CPU, and exhibits frequent hallucinatory artifacts on edge cases.
• Bark: Exceptional quality and expressive non-speech sounds, but far too slow for real-time text-to-speech interaction.

Kokoro-82M changes this landscape entirely. With only 82 million parameters, Kokoro delivers natural, highly human-like speech quality that rivals models 10 to 20 times its size. Because of its lightweight architecture, it achieves an astounding Real-Time Factor (RTF) of under 0.05 on a modest GPU and sub-0.20 on a standard modern CPU. Time-to-First-Byte (TTFB) drops to under 150 milliseconds when properly configured.

When paired with Open WebUI, Kokoro-82M allows you to run a completely self-hosted, air-gapped voice assistant that responds almost instantly. The key to making this work smoothly is exposing Kokoro via an OpenAI-compatible audio API endpoint and configuring Docker networking correctly so Open WebUI can route requests without network timeouts.

Container Architecture and Docker Networking Pitfalls

Related: Fix Open WebUI Web Search: SearXNG Docker Guide →

The single most common obstacle when setting up local text to speech in Open WebUI is container networking. Open WebUI natively supports OpenAI-formatted speech APIs (/v1/audio/speech). To bridge Kokoro-82M with Open WebUI, we run a dedicated API wrapper container that exposes this exact OpenAI endpoint.

However, if both Open WebUI and the Kokoro API container run inside isolated Docker instances on the same host, using http://localhost:8880 or http://127.0.0.1:8880 as your API base URL inside Open WebUI will fail. Inside a Docker container, localhost resolves to the container's own network namespace, not your host machine.

[ Browser Client ] 
        │
        ▼ (Port 3000)
┌────────────────────────────────────────────────────────┐
│ Docker Host                                            │
│                                                        │
│  ┌──────────────────┐   open-webui-net   ┌──────────┐ │
│  │  Open WebUI      │ ─────────────────> │ Kokoro   │ │
│  │  Container       │  http://kokoro:8880│ Container│ │
│  └──────────────────┘                    └──────────┘ │
└────────────────────────────────────────────────────────┘

To resolve this reliably, you have three options:

1. Custom Docker Network (Recommended): Place both Open WebUI and Kokoro on the same custom Docker bridge network. Open WebUI can then talk directly to the Kokoro container using its service name (e.g., http://kokoro-tts:8880/v1).
2. Host Gateway IP: Use http://host.docker.internal:8880/v1 in Open WebUI, provided you pass --add-host=host.docker.internal:host-gateway in your docker run command or set extra hosts in Docker Compose.
3. Host Networking Mode: Run containers using --net=host. This avoids virtual network bridges entirely, but breaks isolation and exposes all container ports directly onto your host interface.

Using a custom Docker bridge network is the cleanest, most secure, and most resilient solution across OS restarts.

Deploying the Kokoro OpenAI-Compatible API Server

Related: Open WebUI Ignoring Your Uploaded Documents? Fix It →

To expose Kokoro-82M as an OpenAI-compatible speech endpoint, we will use the community-maintained kokoro-fastapi image. This image packages Kokoro, ONNX/PyTorch runtimes, and a FastAPI server exposing /v1/audio/speech, /v1/models, and /v1/audio/voices.

Docker Compose Setup (Recommended)

Create a directory on your host named open-webui-voice and save the following docker-compose.yml file. This configures both Open WebUI and the Kokoro TTS engine on a shared virtual network.

version: "3.8"

networks: voice-network: driver: bridge

services: kokoro-tts: image: ghcr.io/remsky/kokoro-fastapi-gpu:v0.0.5 container_name: kokoro-tts restart: unless-stopped ports: - "8880:8880" environment: - PORT=8880 - USE_ONNX=true - FORWARDED_ALLOW_IPS=* deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu] networks: - voice-network

open-webui: image: ghcr.io/open-webui/open-webui:main container_name: open-webui restart: unless-stopped ports: - "3000:8080" volumes: - open-webui-data:/app/backend/data environment: - OPENAI_API_BASE_URL=http://kokoro-tts:8880/v1 networks: - voice-network

volumes: open-webui-data:

Note on CPU-only hosts: If you do not have an NVIDIA GPU, change the image tag to ghcr.io/remsky/kokoro-fastapi-cpu:v0.0.5 and remove the deploy.resources section. The CPU version uses ONNX Runtime with multi-threading, which still yields impressive latency under 200ms for short responses.

Starting the Services and Verifying the API

Run the stack in detached mode:

docker compose up -d

Check the Kokoro startup logs to verify that the weights have loaded cleanly and the fast API routes are live:

docker logs -f kokoro-tts

You should see output indicating that Kokoro loaded voice files (such as af_bella, am_adam, bf_emma) and started the server on port 8880.

To verify that the API is functioning properly before touching Open WebUI, execute a direct curl request from your terminal:

curl -X POST "http://localhost:8880/v1/audio/speech" \
     -H "Content-Type: application/json" \
     -d '{
       "model": "kokoro",
       "input": "Kokoro local text to speech integration is now active.",
       "voice": "af_bella",
       "response_format": "mp3",
       "speed": 1.0
     }' \
     --output test_voice.mp3

If a valid test_voice.mp3 file is saved and plays clearly, your Kokoro container is functioning correctly and ready to connect to Open WebUI.

Configuring Open WebUI Audio Settings

Related: Make Ollama Start on Boot: A systemd Service That Works →

With both containers running on the voice-network network bridge, you can now complete the open webui tts engine config.

Step 1: Access Audio Settings

1. Open your browser and navigate to http://localhost:3000 (or host IP).
2. Log in with your Open WebUI administrator account.
3. Click on your profile icon in the bottom-left corner and select Admin Panel.
4. Navigate to Settings -> Audio.

┌───────────────────────────────────────────────────────────────┐
│ Admin Panel > Settings > Audio                               │
├───────────────────────────────────────────────────────────────┤
│ Text-to-Speech Settings                                       │
│                                                               │
│ TTS Engine          : [ OpenAI                              ] │
│ API Base URL        : [ http://kokoro-tts:8880/v1           ] │
│ API Key             : [ not-needed                          ] │
│ TTS Model           : [ kokoro                              ] │
│ TTS Voice           : [ af_bella                            ] │
│                                                               │
└───────────────────────────────────────────────────────────────┘

Step 2: Apply the Key Settings

Configure the fields as follows:

• TTS Engine: Select OpenAI. (Do not select Web API or ElevenLabs; Kokoro mimics the OpenAI Speech API specs).
• API Base URL: Enter http://kokoro-tts:8880/v1.
  • Crucial: If Open WebUI is running inside Docker on the same network bridge, use the container name http://kokoro-tts:8880/v1.
  • If Open WebUI was installed directly via Python pip on the host OS while Kokoro runs in Docker, use http://127.0.0.1:8880/v1.
• API Key: Type not-needed or sk-dummy. The field cannot be left blank by Open WebUI's frontend validator, but Kokoro-FastAPI ignores auth tokens by default.
• TTS Model: Enter kokoro.
• TTS Voice: Enter your desired voice identifier. Kokoro uses specific naming conventions for its high-quality voice packs:
  • af_bella (American Female - Warm/Natural)
  • af_sky (American Female - Conversational)
  • am_adam (American Male - Clear)
  • am_michael (American Male - Deep)
  • bf_emma (British Female)
  • bm_george (British Male)

Click Save at the bottom of the page.

Step 3: Enable Auto-Playback and Speech Generation

Navigate to your personal user Settings -> Audio (or keep administrative default settings enabled):

1. Toggle Auto-Play Response to ON if you want Open WebUI to automatically speak LLM replies as they finish generating.
2. Under Set Voice, select the default voice (e.g., af_bella).
3. Click the Read Aloud speaker icon on any previous message in your chat workspace to test execution.

Optimizing Latency, Audio Formats, and Troubleshooting

Related: Why Ollama's First Response Is Slow (Cold Start Fix) →

Even with an efficient model like Kokoro-82M, misconfigurations in audio streaming, format encoding, or sentence parsing can introduce unnecessary delays or sound glitches.

1. Optimize Audio Response Format (MP3 vs WAV vs OPUS)

By default, Open WebUI requests mp3 format from OpenAI-compatible audio backends. While MP3 is broadly supported, encoding raw PCM audio into MP3 inside the FastAPI container adds a small CPU computational overhead.

If you observe higher-than-expected latency:

• In Kokoro-FastAPI environment variables, verify USE_ONNX=true. The ONNX Runtime performs execution optimizations on both CPU and GPU execution providers.
• If network bandwidth within your local LAN/host is not a constraint, test requesting uncompressed wav or opus in custom client requests. MP3 remains the safest baseline for browser compatibility across Safari, Chrome, and Firefox.

2. Fixing Truncated Audio or Sentence Boundary Cuts

When streaming responses from an LLM, Open WebUI sends text chunks to the TTS backend as the generation progresses. If the sentence splitting regex in Open WebUI breaks on uncommon punctuation or code blocks, the Kokoro server might receive empty strings or raw Markdown syntax.

To fix audio artifacts caused by raw Markdown output:

• In Admin Panel -> Settings -> Audio, ensure Format Text for TTS is enabled. This option strips Markdown headers (#), bold formatting (**), backticks, and code blocks before sending text payloads to Kokoro.

3. Debugging Connection and Timeout Errors

If you click the speaker icon in Open WebUI and see a generic error notification or no audio plays, check the following common failure points:

Issue A: Failed to fetch or Connection Refused

This indicates Open WebUI cannot route HTTP requests to the specified URL.

• Fix: Verify your network setup by entering the Open WebUI container shell and pinging the Kokoro container directly:
  bashنسخ

  docker exec -it open-webui curl http://kokoro-tts:8880/v1/models

  If this times out, your containers are not sharing the same Docker network bridge.

Issue B: 404 Not Found on /v1/audio/speech

This occurs when the API base URL has an incorrect path suffix appended.

• Fix: Make sure the URL configured in Open WebUI ends with /v1. Do not include /audio/speech in the API Base URL input field. Open WebUI automatically appends /audio/speech to the base endpoint path.

Issue C: High CPU Usage and Audio Stuttering

If speech generation sounds distorted or stutters during playback, the CPU thread allocation for ONNX runtime might be contending with the LLM inference engine (e.g., Ollama or vLLM).

• Fix: Limit the CPU core count assigned to the Kokoro container by setting thread limits in your docker launch flags:
  yamlنسخ

  environment:
    - NUM_THREADS=4

Frequently Asked Questions

Can I run Kokoro TTS with Open WebUI entirely on a CPU?

Yes. Kokoro-82M is extremely lightweight. When running via ONNX Runtime on a modern 6-core CPU, speech generation latency typically stays under 200 milliseconds, producing real-time speech synthesis without requiring a dedicated NVIDIA GPU.

How do I add custom or cloned voices to Kokoro in Open WebUI?

Kokoro uses specific pre-computed voice embedding files (.pt or .bin). To use additional voices, place your custom voice matrix files into the voices directory of your kokoro-fastapi deployment, then reference the filename (without extension) in Open WebUI's TTS Voice settings field.

Why is there no sound when using Open WebUI over HTTPS?

Modern web browsers block unencrypted mixed content. If you access Open WebUI via HTTPS (e.g., https://chat.yourdomain.com), your browser may block audio elements fetched from an unencrypted http:// local IP address. Ensure your reverse proxy (such as Nginx, Traefik, or Caddy) handles SSL termination for both Open WebUI and your Kokoro API endpoint.

How does Kokoro compare to Piper TTS in real-world usage?

While Piper TTS consumes slightly fewer system resources, its voice output sounds visibly synthesized and monotone. Kokoro-82M produces significantly more natural cadence, realistic inflection, and clearer pronunciation, making it far better suited for long-form reading and interactive voice conversations in Open WebUI.