Give Your Local AI Private Web Search With SearXNG

I love running a local model. I can ask it anything, it never phones home, and it costs me nothing per question. But it has one embarrassing blind spot: it cannot see today. Ask my Ollama model who won a match last night or what version of some tool just shipped, and it either shrugs or — worse — invents a confident, wrong answer. A model that can’t see the present is only half a tool.

The fix is to give it eyes: a search engine it can query, fetch pages from, and cite. And because this is a homelab and I care about not leaking every question I ask to a search company, I want that engine to be mine. This post wires Open WebUI — the chat front-end most of us put on top of Ollama — to a self-hosted SearXNG metasearch engine, so answers come back with citations and nothing sensitive ever leaves my network but the search itself. This is the first stop in the Self-Host the Apps You Actually Use series, and it’s my favorite because the payoff is so immediate.

Here’s the whole pipeline we’re about to build:

Why a metasearch engine, and why self-host it

A metasearch engine doesn’t crawl the web itself. It takes your query, forwards it to a pile of existing search engines at once, merges what they return, and hands you a clean, ad-free, tracker-free list. SearXNG is the best-known open-source one, and it’s a joy to run at home for two reasons.

First, no API keys. Every “give your LLM web search” tutorial that reaches for a commercial search API asks you to paste a billing-attached key and start metering your curiosity. SearXNG needs none — it speaks to public engines the same way your browser does.

Second, no query leakage. When your model searches through SearXNG, the questions fan out to public engines but never accumulate into a profile at a single company tied to your account. For a machine whose whole selling point is privacy, sending every question to a cloud search API would rather defeat the purpose.

The catch — and it’s the whole reason this post exists — is that the wiring between Open WebUI and SearXNG has a few silent failure modes that make everything look connected while quietly doing nothing. We’ll deploy SearXNG first, then walk the wiring carefully.

Deploy SearXNG

I run SearXNG natively (no Docker) in a small Proxmox LXC container — it’s a lightweight Python app and doesn’t need a container-in-a-container. The official step-by-step install is the source of truth; here’s the shape of it with the two homelab-specific settings that matter.

# as root, create the service user and directories
sudo useradd --system --home-dir /usr/local/searxng --shell /bin/bash searxng

# as the searxng user, clone and install into a venv
git clone https://github.com/searxng/searxng /usr/local/searxng/searxng-src
python3 -m venv /usr/local/searxng/searx-pyenv
. /usr/local/searxng/searx-pyenv/bin/activate
pip install --use-pep517 --no-build-isolation -e /usr/local/searxng/searxng-src

use_default_settings: true

search:
formats:
  - html
  - json          # REQUIRED, or Open WebUI gets empty/403 results

server:
limiter: false    # LAN-only instance: skip the Redis/Valkey rate-limiter
public_instance: false
secret_key: "CHANGE_ME"   # generate a random value inside the container

granian --interface wsgi --host 0.0.0.0 --port 8888 --access-log searx.webapp:app

curl -s -o /dev/null -w '%{http_code}
' "http://127.0.0.1:8888/search?q=proxmox&format=json"
# expect: 200

Wire Open WebUI to it — and the silent traps

This is where the hour goes, so let’s go slowly. On paper it’s three settings. In practice, three separate things can make the globe icon in your chat do absolutely nothing while reporting no error at all.

The official Open WebUI SearXNG docs give you the three settings:

ENABLE_WEB_SEARCH=True
WEB_SEARCH_ENGINE=searxng
SEARXNG_QUERY_URL=http://10.0.0.20:8888/search?q=<query>

The literal string <query> in that URL is a placeholder Open WebUI fills in — leave it exactly as written. Now the three traps, in the order they bit me.

Trap 1: in recent versions, the database beats the environment

Newer Open WebUI (I was on the 0.10 line) stores settings in a database table, and on an existing install those rows win over environment variables. You can set ENABLE_WEB_SEARCH in your service file all day; if a config row already exists, your variable is politely ignored. The reliable fix is to set it in the running app’s Admin Panel → Settings → Web Search, which writes the database directly. Set the engine to searxng and paste the query URL there.

Trap 2: the model needs the right calling mode, or search silently no-ops

This is the subtle one. In the version I ran, Open WebUI only runs its forced web-search step for a model whose calling mode is set to legacy function calling. The default (native) path instead expects the model to decide on its own to call a search tool — which small local models essentially never do. And a freshly pulled Ollama model has no per-model config entry at all, so the setting is simply absent, and search is skipped for everyone with no error.

I only found this by reading the backend source for the exact version I was running, which is the honest lesson here: when a feature silently does nothing, read what the running code actually checks. The fix is to give each chat model a config entry setting its function-calling mode to legacy, via the model editor.

Trap 3: leave “search query generation” off for small models

Open WebUI has an option to let a model rewrite your prompt into a search query before searching. It sounds smart. With small local models it’s a disaster: they emit malformed or empty queries, and Open WebUI then silently skips the search and answers from stale memory — sometimes with a confident, wrong result. I re-enabled it deliberately once just to confirm, and got zero sources and a hallucinated answer. Leave search-query generation off; the raw prompt is a perfectly good search query.

The speed reality (so you’re not surprised)

When it works, the first thing you’ll notice is that web-search answers are slow on CPU-only hardware. That’s expected, and it’s mostly not the model’s fault. Turning your prompt into an answer breaks into: search (fast), fetch several web pages and embed their text (slow), then generate the reply (medium).

On my GPU-less nodes, a small qwen2.5:3b model gave plain chat replies in roughly 15–30 seconds but took around 80 seconds with web search — because fetching and embedding the pages dominates, not the thinking. A 7B model on the same CPU was simply too slow to enjoy. Two honest takeaways: use a small model for interactive web search on CPU, and know that a GPU is the real fix if you want it snappy. See Run an Ollama Cluster in LXC for how the inference side is laid out.

Don’t let it fail silently

There’s a nasty property to this setup: if SearXNG goes down, Open WebUI’s citations just quietly stop — the chat still answers, just without sources, and you might not notice for weeks. That’s exactly the kind of silent failure worth a monitoring check. I point a simple black-box HTTP probe at the SearXNG URL and alert if it stops returning 200, so a dead search engine pages me instead of degrading in silence. The companion playbook includes the deploy steps end to end.

What’s next

You’ve now got a local model that can see today, with citations, and without leaking a single question to a search vendor. Next in the series, we point that same local-AI muscle at your bookmarks: Karakeep auto-tags every link you save using a local model, so your bookmark graveyard becomes something you can actually find things in.

Related posts:

• Self-Hosted Bookmarks That Tag Themselves With Local AI — the next post in this series: point the same local model at your saved links.
• Paperless-ngx: A Searchable Archive for Every Document — same self-hosting instinct, applied to every PDF you own.
• Watch Any Web Page for Changes With changedetection.io — let a watcher poll pages so you don’t have to search for the same update twice.
• Run an Ollama Cluster in Proxmox LXC — the local inference layer this post assumes you already have.
• Open WebUI: Advanced Setup — the chat front-end we wired SearXNG into, configured properly.
• What Is Prometheus? — the monitoring that catches a silently-dead search engine.
• Create Your First Proxmox LXC Container — where SearXNG lives in this build.