Open WebUI Ignoring Your Uploaded Documents? Fix It

I uploaded a 60-page product manual to Open WebUI, typed a question about page 41, and got a polite answer explaining that the assistant had no access to any documents. The file was right there in the sidebar with a green checkmark. That gap — upload succeeded, retrieval silently did nothing — took me longer to debug than it should have, mostly because there is no error message anywhere.

Here are the seven causes I worked through, in the order worth checking.

1. You never referenced the document in the prompt

Related: Why Ollama's First Response Is Slow (Cold Start Fix) →

This catches almost everyone once. Uploading a file to your workspace does not automatically inject it into every chat. In Open WebUI you have to bring it in explicitly:

• Type # in the message box and pick the document or the collection.
• Or attach it with the + button inside that specific conversation.

If the document is not tagged into the message, the model genuinely never sees it. You will know it worked when a citations block appears under the response. No citations means no retrieval happened — that single signal saves a lot of guessing.

2. The embedding model was never downloaded

Related: CUDA Out of Memory: 9 Fixes That Actually Worked on My GPU →

Retrieval needs an embedding model, and it is a different model from your chat model. Open the admin panel, go to Settings, then Documents, and look at the embedding model field.

If it is set to a default that does not exist on your Ollama instance, indexing fails quietly. Pull one first:

ollama pull nomic-embed-text

Then verify the server can see it:

curl -s http://localhost:11434/api/tags | grep embed

Set that exact tag in the Documents settings and re-upload the file. Changing the embedding model does not re-index existing documents — you must reprocess them, which is the mistake I made twice. There is a comparison of the usual options in the best local embedding models.

3. Open WebUI cannot reach Ollama for embeddings

Related: Make Ollama Start on Boot: A systemd Service That Works →

The chat can work while embeddings fail if you are running in Docker and the container resolves the chat endpoint but the embedding call times out. Test from inside the container, not from your host:

docker exec -it open-webui curl -s http://host.docker.internal:11434/api/tags

On Linux, host.docker.internal needs an explicit host mapping:

extra_hosts:
  - "host.docker.internal:host-gateway"

If that curl fails, retrieval will never work no matter what you set in the UI. Our write-up on Open WebUI in Docker not connecting to Ollama goes through the networking cases in detail.

4. The PDF has no extractable text

Related: Ollama Filled My Disk: How I Moved the Models Directory Safely →

This one is invisible from the interface. A scanned document is a stack of images; the extractor pulls out zero characters, stores an empty set of chunks, and reports success.

Check quickly on your host:

pdftotext manual.pdf - | wc -w

If that returns something near zero, you need OCR before upload. Open WebUI can be pointed at Apache Tika or you can pre-process with ocrmypdf:

ocrmypdf --skip-text manual.pdf manual-ocr.pdf

Upload the OCR'd version and re-ask. In my case the original manual was a scan of a printed booklet, which explained a lot.

5. Chunk size and overlap are wrong for your content

Related: Ollama Not Using My NVIDIA GPU in WSL2: The Fix That Finally Worked →

Defaults are usually 1500 characters with 100 characters of overlap. That works for prose and works badly for structured documents — tables, spec sheets, and code get sliced through the middle, so no single chunk contains a complete answer.

What I use now, under Settings then Documents:

Smaller chunks with more overlap cost more storage and slightly slower indexing, and they noticeably improve recall on documents where the answer is one line in a table. Re-index after changing these; the old chunks stay as they were.

6. Top-K is too low, or the relevance threshold is too high

Related: Ollama Pull Fails with "max retries exceeded" or EOF: How I Get Downloads to Finish →

Two retrieval settings decide what actually reaches the model:

• Top K — how many chunks get pulled in. The default of 3 is stingy for a long document. I run 5 to 8.
• Relevance threshold — chunks scoring below it are discarded. If this is set above roughly 0.5 with a compact embedding model, you can filter out every result and end up sending nothing.

Set the threshold to 0 while you are debugging. If answers suddenly become correct, the threshold was the problem and you can raise it gradually until quality drops.

7. The context window is too small to hold the chunks

Retrieval can succeed and still be useless. If you pull 8 chunks of 1200 characters, that is roughly 2,400 tokens of context before your question, the system prompt, and the answer. On a model running with num_ctx of 2048, the retrieved text gets truncated away — often silently.

Check what the model is actually configured for and raise it:

FROM llama3.1:8b
PARAMETER num_ctx 8192

ollama create rag-llama -f Modelfile

Then select rag-llama in Open WebUI. If you see answers that quote the first chunk correctly and ignore everything else, this is almost always the cause. The trade-off is memory, which I covered in running two local models on one GPU, and truncation behaviour more generally in Ollama answers being cut off.

A quick end-to-end test

When something changes, I run the same five-step check rather than trusting the interface:

1. Upload a small plain text file containing one unusual sentence — I use a made-up part number.
2. Confirm the document status shows as processed in the admin Documents list.
3. Start a new chat, type #, select the document.
4. Ask for that exact part number.
5. Confirm a citations block appears with the file name.

If step 5 fails, retrieval is broken. If step 5 succeeds but the answer is wrong, retrieval is fine and the problem is chunking, top-K, or the model. Separating those two failure modes is most of the work.

What finally fixed mine

Two things, in combination: the embedding model in settings pointed at a tag that was not pulled, so nothing was ever indexed; and once that was fixed, the default chunk size of 1500 was cutting the manual's tables in half. Setting nomic-embed-text, chunk size 800, overlap 200, top-K 6, and threshold 0 gave correct answers with citations on the first attempt.

None of it was visible as an error. If you are building this stack from scratch rather than repairing it, building a RAG chatbot on your own documents covers the setup in order, and the Ollama plus Open WebUI Docker Compose stack has a configuration that starts out correct.