Jarian Cottingham 645d8820f1 fix: repair middleware wiring, input validation, and resource lifecycle
- Wrap RateLimitMiddleware in Middleware() - bare class broke all requests (500)
- Clamp/validate num param - ?num=abc no longer returns 500
- Query length cap (500), result counts clamped to 1..20
- Wire lifespan into Starlette so shared httpx client closes on shutdown
- Purge stale per-IP rate limit entries to bound memory growth
- URL-encode queries in the Playwright scraper path
- Lazy lib imports so the package works without optional playwright
- Add pyproject.toml (activates ruff/pytest/bandit in CI), README, LICENSE
- Remove committed __pycache__, add .gitignore/.dockerignore
- Fix SearXNG healthcheck path (/health -> /healthz) and compose docs
2026-08-20 23:27:27 +00:00
2026-07-05 22:04:57 +00:00
2026-07-03 01:06:24 +00:00
2026-07-03 01:06:24 +00:00

MCP Search Servers

Model Context Protocol (MCP) servers that expose Google and DuckDuckGo search as tools for LLM clients (Claude Desktop, Cursor, VS Code, ...), backed by a self-hosted SearXNG metasearch instance.

SearXNG aggregates results from Google, DuckDuckGo, Brave, Wikipedia, and more with its own request handling, so a single SearXNG container serves both MCP servers — no browser fingerprinting, no API keys, fully self-hosted.

Services

Service Port MCP tool Description
google-mcp 3001 google_search Google results via SearXNG
duckduckgo-mcp 3002 duckduckgo_search DuckDuckGo results via SearXNG
searxng Internal metasearch engine (not published)

Each server exposes:

  • /sse + /messages/ — MCP SSE transport
  • /search?q=...&num=N — plain HTTP JSON endpoint
  • /health — liveness probe

Quick Start

cp .env.example .env        # fill in SEARXNG_SECRET
docker compose up -d --build

Generate a SearXNG secret:

python3 -c "import secrets; print(secrets.token_hex(32))"

Verify:

curl http://localhost:3001/health
curl "http://localhost:3001/search?q=python&num=3"

Then point your LLM client at http://localhost:3001/sse (Google) and/or http://localhost:3002/sse (DuckDuckGo). Client configuration examples for Claude Desktop, Cursor/Windsurf, and VS Code are in USAGE.md.

Tool Contract

{
  "name": "google_search",
  "arguments": { "query": "Python programming language", "num_results": 5 }
}

Returns numbered results with title, URL, and snippet (max 20 results, queries capped at 500 characters).

Configuration

Variable Default Description
SEARXNG_SECRET — (required) SearXNG session/CSRF secret
RATE_LIMIT_SECONDS 5 / 3 Minimum seconds between search requests
MCP_PORT 3001 / 3002 Port the MCP server listens on
SEARXNG_URL http://searxng:8080 URL of the SearXNG instance

Security Model

  • Internal network — SearXNG is on a internal: true Docker network; only the two MCP containers can reach it. It is not published to the host.
  • Per-IP rate limiting — HTTP endpoints allow 30 requests/minute per IP (/health exempt); excess requests get 429.
  • Request pacing — each server also enforces a minimum interval between upstream SearXNG queries to avoid throttling.
  • No auth on /search and /sse — the servers assume they are bound to a trusted network. Expose them publicly only behind an auth proxy.
  • Input validation — query length capped, result counts clamped to 1..20, malformed parameters return 400 instead of crashing.

Optional: Playwright Scraping Fallback

lib/google_search.py and lib/duckduckgo_search.py contain a legacy Playwright-based scraping path (single shared browser, stealth user agent, HTML parsing with multiple fallback strategies). It is not used by the production servers — Google aggressively blocks headless browsers and SearXNG is the working path. If you want to run the scraper directly:

pip install playwright
playwright install chromium

Development

python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

ruff check .
pytest tests/ -v

# End-to-end test (requires running containers)
python3 integration_test.py

Project Structure

├── docker-compose.yml        # All 3 services
├── searxng-settings.yml      # SearXNG engine config
├── google-mcp/
│   ├── Dockerfile
│   └── server.py             # Google MCP server (SearXNG-backed)
├── duckduckgo-mcp/
│   ├── Dockerfile
│   └── server.py             # DuckDuckGo MCP server (SearXNG-backed)
├── lib/
│   ├── playwright_manager.py # Single-instance browser manager (optional path)
│   ├── rate_limiter.py       # Async minimum-interval rate limiter
│   ├── google_search.py      # Playwright Google scraper (optional path)
│   └── duckduckgo_search.py  # Playwright DuckDuckGo scraper (optional path)
├── tests/
│   ├── test_rate_limiter.py
│   └── test_search_parsing.py
├── integration_test.py       # End-to-end MCP SSE test
└── test_client.py            # Manual stdio/SSE smoke client

License

MIT

Description
Self-hosted Google + DuckDuckGo search as MCP tools for LLM clients, backed by SearXNG. No API keys, no browser fingerprinting.
Readme 112 KiB
Languages
Python 98.6%
Dockerfile 1.4%