Self-Host Perplexica with Local Ollama: Zero-Cost AI Search

Why Build a 100% Local AI Search Engine?

Related: Run CrewAI with Ollama: Fix Crashes & Local Setup Guide →

Cloud-based AI search engines like Perplexity provide fast, cited answers from across the web. However, every query you send routes through third-party servers, costs API credits or subscription fees, and locks you into whatever model provider the company chooses to route your requests through.

Combining Perplexica, Ollama, and SearXNG gives you a completely self-hosted, open-source engine that replicates Perplexity's core feature set: deep web crawling, source citation, focus modes (Academic, Writing, YouTube, Reddit), and conversational follow-ups.

Running this stack locally gives you three concrete advantages:

1. Zero Recurring Costs: Once your hardware is running, you pay $0 per query. There are no API rate limits or monthly subscriptions.
2. Total Privacy: Search queries and context windows never leave your local network. Your internal documents and personal searches remain strictly on your machine.
3. Model Autonomy: You can pair any open-weights LLM (such as Qwen 2.5, Llama 3.1, or Mistral) with any embedding model, matching your local VRAM constraints.

Here is how a local Perplexica stack compares to standard search engines and commercial cloud AI search tools:

Hardware Requirements and Model Selection

Related: Connect Avante.nvim to Local Ollama: 2026 Neovim Guide →

Running an AI search pipeline requires running three components simultaneously: SearXNG (web scraping), Ollama (LLM text synthesis and embedding processing), and Perplexica (frontend UI and backend orchestration).

Recommended Hardware Specs

• GPU: NVIDIA RTX 3060 (12GB) / RTX 4070 (12GB) or Apple Silicon (M1/M2/M3 with 16GB+ Unified Memory).
• System RAM: 16GB minimum (32GB recommended if running SearXNG and Ollama on the same host).
• Storage: 20GB free space on an NVMe SSD for Docker images and GGUF model weights.

Model Selection for Ollama

For optimal performance with Perplexica local Ollama setups, you need two distinct models running in Ollama: an LLM for synthesis and an Embedding model for vectorizing search snippets.

Recommended LLM Models

• Qwen 2.5 (7B / 14B): Exceptional instruction-following capabilities, structured JSON output handling, and strong citation capabilities.
• Llama 3.1 (8B): Fast inference speeds, reliable summary output, low memory footprint (~4.7GB VRAM at q4_k_m quant).

Recommended Embedding Models

• nomic-embed-text: High context window (8192 tokens), low memory usage (~280MB), excellent retrieval metrics.
• bge-large-en-v1.5: Strong dense retrieval performance, standard 1024-dimension embeddings.

Step 1: Preparing Host Ollama and Pulling Models

Related: LiteLLM Proxy Local Ollama Setup: Unified AI Gateway →

You can run Ollama directly on your host operating system or inside a Docker container. Running Ollama natively on the host typically offers easier GPU passthrough and slightly lower latency on Linux and macOS hosts.

Step 1.1: Configure Ollama to Listen on All Interfaces

By default, Ollama binds only to 127.0.0.1:11434. Because Perplexica will run inside a Docker container, it cannot reach your host's loopback interface unless Ollama listens on all interfaces (0.0.0.0).

On Linux (systemd)

Edit the systemd service file:

sudo systemctl edit ollama.service

Add the following environment configuration under the [Service] section:

[Service]
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_ORIGINS=*"

Save the file and restart the service:

sudo systemctl daemon-reload
sudo systemctl restart ollama

On macOS

Open your terminal and run:

launchctl setenv OLLAMA_HOST "0.0.0.0"
launchctl setenv OLLAMA_ORIGINS "*"

Restart the Ollama desktop app.

Step 1.2: Pull the LLM and Embedding Models

Execute the following commands in your host terminal to pull the model weights required for your setup:

# Pull the text generation model
ollama pull qwen2.5:7b

Pull the text embedding model

ollama pull nomic-embed-text

Verify that both models are installed and accessible:

ollama list

You should see output similar to this:

NAME                    ID              SIZE    MODIFIED
qwen2.5:7b              843d13b1901c    4.7 GB  10 minutes ago
nomic-embed-text:latest 0a1021456974    274 MB  5 minutes ago

Step 2: Setting Up SearXNG for Perplexica

Related: Connect AnythingLLM to Local Ollama: 2026 Setup Guide →

Perplexica relies on SearXNG as its underlying meta-search engine. SearXNG aggregates web results from Google, Bing, DuckDuckGo, and dozens of other search providers without user tracking.

To allow Perplexica to parse web data efficiently, SearXNG must be configured to return search responses in JSON format.

Step 2.1: Create Directory Layout

Create a dedicated directory on your host machine to store configuration files and container states:

mkdir -p ~/perplexica-docker/searxng
cd ~/perplexica-docker

Step 2.2: Create searxng/settings.yml

Create a file named settings.yml inside the searxng directory:

nano searxng/settings.yml

Paste the following minimal, functional configuration:

use_default_settings: true

general: debug: false instance_name: "Local Perplexica Search"

search: safe_search: 0 autocomplete: "" default_lang: "en" formats: - html - json

server: port: 8080 bind_address: "0.0.0.0" secret_key: "generate_a_random_secret_key_here_change_me" limiter: false

engines:

• name: google engine: google shortcut: g disabled: false

• name: duckduckgo engine: duckduckgo shortcut: ddg disabled: false

• name: wikipedia engine: wikipedia shortcut: wp disabled: false

Critical Note: The formats: array MUST contain - json. If this key is missing or set only to HTML, Perplexica will throw fatal JSON parsing errors whenever it attempts a web search.

Step 3: Deploying Perplexica via Docker Compose

Related: Run Dify Locally with Ollama: Step-by-Step Setup Guide →

Now that Ollama is serving models on port 11434 and SearXNG is configured, you can launch the complete stack using Docker Compose.

Step 3.1: Clone the Perplexica Repository

Clone the official repository into your working directory:

git clone https://github.com/ItzCrazyKaty/Perplexica.git
cd Perplexica

Step 3.2: Configure config.toml

Perplexica uses a config.toml file to route traffic to backend LLMs, embedding models, and SearXNG instances. Copy the example configuration and open it for editing:

cp sample.config.toml config.toml
nano config.toml

Update your config.toml settings to match local Ollama endpoints. Modify the file to reflect these values:

[GENERAL]
PORT = 3001
SIMILARITY_MEASURE = "cosine"

[API_KEYS] OPENAI = "" GROQ = "" ANTHROPIC = ""

[API_ENDPOINTS] SEARXNG = "http://searxng:8080" OLLAMA = "http://host.docker.internal:11434"

Networking Strategy: We use http://host.docker.internal:11434 as the Ollama URL. This tells the Perplexica backend container to route out of the Docker bridge network directly into the host machine's native Ollama service.

Step 3.3: Write the docker-compose.yml File

In the main Perplexica folder, replace or create the docker-compose.yml file:

version: '3.8'

networks: perplexica-network: driver: bridge

services: searxng: image: searxng/searxng:latest container_name: perplexica-searxng volumes: - ./searxng/settings.yml:/etc/searxng/settings.yml:ro ports: - "8080:8080" networks: - perplexica-network restart: unless-stopped

perplexica-backend: build: context: . dockerfile: backend.Dockerfile container_name: perplexica-backend ports: - "3000:3000" volumes: - ./config.toml:/usr/src/app/config.toml extra_hosts: - "host.docker.internal:host-gateway" networks: - perplexica-network depends_on: - searxng restart: unless-stopped

perplexica-frontend: build: context: . dockerfile: app.Dockerfile container_name: perplexica-frontend ports: - "3001:3000" networks: - perplexica-network depends_on: - perplexica-backend restart: unless-stopped

Step 3.4: Build and Start Containers

Build the images and spin up the containers in detached mode:

docker compose up -d --build

Monitor the logs to verify everything initializes cleanly:

docker compose logs -f

Open your web browser and navigate to http://localhost:3001. You should be greeted by the Perplexica UI.

Step 4: Configuring Perplexica Embedding Models in the Web UI

Related: Zed Editor Ollama Setup: Fast Local AI Coding Guide →

Once the interface opens, you must select your models inside the Perplexica Settings page to complete the setup.

1. Open http://localhost:3001 in your web browser.
2. Click the Settings gear icon in the lower-left corner.
3. Under Model Provider, select Ollama.
4. Set the Ollama API URL to http://host.docker.internal:11434.
5. Under Chat Model, select your pulled text model (e.g., qwen2.5:7b or llama3.1:8b).
6. Under Embedding Model Provider, select Ollama.
7. Under Embedding Model, choose nomic-embed-text:latest.
8. Click Save Changes.

Step 5: Fixing Common Docker Network Connection Errors

Integrating containers with host services frequently triggers networking issues. Below are concrete fixes for the most common bugs encountered during a local Perplexica installation.

Error 1: connect ECONNREFUSED 127.0.0.1:11434 or host.docker.internal Resolution Failure

Symptom

The backend logs (docker compose logs perplexica-backend) show connection refused errors when attempting to reach Ollama.

Root Cause

Containerized applications inside a Docker network treat 127.0.0.1 as their own container internal loopback, not the host machine running Ollama.

Fix

1. Verify extra_hosts exists in your docker-compose.yml under perplexica-backend:
   yamlنسخ

   extra_hosts:
     - "host.docker.internal:host-gateway"

2. If running on pure Linux without standard Docker Desktop mapping support, replace host.docker.internal in your config.toml with the default Docker gateway IP address:
   tomlنسخ

   OLLAMA = "http://172.17.0.1:11434"

3. Test connectivity directly from inside the running Perplexica container:
   bashنسخ

   docker exec -it perplexica-backend curl http://host.docker.internal:11434/api/tags

   If it returns a JSON payload listing your models, the network link is functional.

Error 2: SearXNG Error: Format "json" is not supported

Symptom

Perplexica returns generic search failure messages, and backend logs report empty search results or HTML rendering errors.

Root Cause

SearXNG defaults to serving search queries as standard HTML pages and blocks raw API JSON output unless explicitly enabled in its YAML config.

Fix

1. Confirm that searxng/settings.yml contains:
   yamlنسخ

   search:
     formats:
       - html
       - json

2. Validate the SearXNG JSON endpoint using curl from your terminal:
   bashنسخ

   curl -X POST "http://localhost:8080/search" -d "q=test&format=json"

   If this outputs raw JSON search results, restart the backend container:
   bashنسخ

   docker compose restart perplexica-backend

Error 3: Embedding Model Vector Dimension Mismatch

Symptom

When submitting a search query, the backend crashes with a vector dimension dimension error: Error: Vector dimension mismatch. Expected 768, got 1024 (or similar numbers).

Root Cause

Perplexica's internal vector index caches embeddings from a previously selected embedding model. Swapping from an integrated provider (like OpenAI's 1536-dim text-embedding-3-small) to Ollama's nomic-embed-text (768-dim) causes schema conflicts.

Fix

Clear Perplexica's internal cache database by restarting containers and resetting local storage volumes, or ensure your initial config.toml matches your chosen local embedding model before running queries:

docker compose down
docker volume prune -f
docker compose up -d

Step 6: Benchmarks, Tuning, and Performance Optimization

Once your pipeline is online, fine-tuning your parameters will keep token latency down and answer relevance high.

Recommended Quantization Levels

To balance speed and accuracy when running local LLMs alongside search workflows:

• q4_k_m (4-bit medium quantization): Best balance of memory usage and answer quality. Provides roughly 35 to 55 tokens per second on mid-range GPUs.
• q8_0 (8-bit quantization): Useful if you encounter hallucination issues with source citations, but requires ~40% more VRAM.

Performance Benchmark

Here are practical execution numbers collected from a test rig running an NVIDIA RTX 4070 (12GB VRAM), 32GB System RAM, and Ubuntu 22.04 LTS:

• LLM Model: qwen2.5:7b-instruct-q4_k_m
• Embedding Model: nomic-embed-text:latest
• SearXNG Query Time: ~320ms (fetching across 3 search engines)
• Embedding Retrieval Latency: ~45ms
• Time-to-First-Token (TTFT): 850ms
• Generation Speed: ~48 tokens/sec

Tuning Perplexica Response Quality

To optimize local inference performance:

1. Reduce Source Fetch Limit: Inside Perplexica's web UI, lower the max search results limit from 10 down to 5. This reduces vector processing time by half, significantly speeding up synthesis on lower-tier hardware.
2. Use Specialized Focus Modes:
   • Web Search: Best for general news, recent events, and technical documentation.
   • Writing Assistant: Disables SearXNG web retrieval entirely. Runs exclusively as an offline LLM instance for fast text editing and code generation.
   • Academic: Targets Google Scholar and ArXiv via SearXNG for research papers.

Frequently Asked Questions

Can I run Perplexica without an active internet connection?

Perplexica requires an active internet connection to execute web searches via SearXNG. However, if you switch to the Writing Assistant mode inside the UI, web scraping is bypassed, allowing you to use your local Ollama models fully offline.

Why is Perplexica slow compared to native Ollama CLI responses?

When you prompt Ollama directly in your terminal, it processes only your query. When you run a query through Perplexica, the system executes multiple steps: it generates search queries via the LLM, queries SearXNG, parses and scrapes top web pages, runs embedding generation on those text chunks, stores them temporarily in memory, retrieves relevant contexts, and then streams the final answer. Hardware with fast NVMe drives and dedicated VRAM minimizes this overhead.

How do I update Perplexica containers to the latest version?

Pull the latest code from the GitHub repository, then rebuild the Docker containers:

cd Perplexica
git pull origin master
docker compose down
docker compose up -d --build

Can I run this stack on a Raspberry Pi 5 or low-power mini PC?

You can run SearXNG and the Perplexica web services on low-power devices. However, running local LLMs like qwen2.5:7b requires substantial compute. On an N100 mini-PC or Raspberry Pi 5, CPU generation speeds will drop to 1-3 tokens per second. For low-spec hardware, consider using smaller 0.5B or 1.5B parameters models (qwen2.5:1.5b), or route Perplexica's backend to a separate local desktop PC running Ollama on your local network.