- Wrap RateLimitMiddleware in Middleware() - bare class broke all requests (500) - Clamp/validate num param - ?num=abc no longer returns 500 - Query length cap (500), result counts clamped to 1..20 - Wire lifespan into Starlette so shared httpx client closes on shutdown - Purge stale per-IP rate limit entries to bound memory growth - URL-encode queries in the Playwright scraper path - Lazy lib imports so the package works without optional playwright - Add pyproject.toml (activates ruff/pytest/bandit in CI), README, LICENSE - Remove committed __pycache__, add .gitignore/.dockerignore - Fix SearXNG healthcheck path (/health -> /healthz) and compose docs
5.6 KiB
MCP Search Services
MCP (Model Context Protocol) servers that expose Google and DuckDuckGo search as tools for LLMs.
Quick Start
# From the repository root
docker compose up -d --build
This starts 3 containers:
| Service | Port | Purpose |
|---|---|---|
google-mcp |
3001 | MCP server with google_search tool |
duckduckgo-mcp |
3002 | MCP server with duckduckgo_search tool |
searxng |
8080 | Internal metasearch engine |
Architecture
LLM/MCP Client
│
├── http://host:3001/sse ──→ google-mcp ──→ searxng ──→ Google
└── http://host:3002/sse ──→ duckduckgo-mcp ──→ searxng ──→ DuckDuckGo
SearXNG handles all the actual search work behind the scenes. It aggregates from Google, Brave, Startpage, Wikipedia, and other engines with rotating user agents, so there's no single browser fingerprint to detect.
Connecting an LLM Client
Claude Desktop
Add to claude_desktop_config.json:
{
"mcpServers": {
"google-search": {
"command": "npx",
"args": ["-y", "@anthropic-ai/mcp-remote", "http://localhost:3001/sse"]
},
"duckduckgo-search": {
"command": "npx",
"args": ["-y", "@anthropic-ai/mcp-remote", "http://localhost:3002/sse"]
}
}
}
Cursor / Windsurf
In your IDE's MCP settings, add:
Google Search
- Type: SSE
- URL:
http://localhost:3001/sse
DuckDuckGo Search
- Type: SSE
- URL:
http://localhost:3002/sse
VS Code with MCP extension
Install the MCP extension, then add to workspace settings:
{
"mcp": {
"servers": {
"google-search": {
"transport": {
"type": "sse",
"url": "http://localhost:3001/sse"
}
},
"duckduckgo-search": {
"transport": {
"type": "sse",
"url": "http://localhost:3002/sse"
}
}
}
}
}
Remote hosting
If your machine has a public IP or domain, replace localhost with your host:
http://your-server-ip:3001/sse
http://your-server-ip:3002/sse
Available Tools
google_search
Searches via Google engine through SearXNG.
{
"name": "google_search",
"arguments": {
"query": "Python programming language",
"num_results": 5
}
}
duckduckgo_search
Searches via DuckDuckGo engine through SearXNG.
{
"name": "duckduckgo_search",
"arguments": {
"query": "Python programming language",
"num_results": 5
}
}
Both tools return formatted text with numbered results containing title, URL, and snippet.
Configuration
| Variable | Default | Description |
|---|---|---|
RATE_LIMIT_SECONDS |
5 (Google) / 3 (DDG) | Minimum seconds between search requests |
MCP_PORT |
3001 / 3002 | Port the MCP server listens on |
SEARXNG_URL |
http://searxng:8080 |
URL of the SearXNG instance |
To change rate limits, edit docker-compose.yml and restart.
Management
# Start all services
docker compose up -d
# Stop all services
docker compose down
# View logs
docker compose logs -f google-mcp
docker compose logs -f duckduckgo-mcp
# Rebuild after code changes
docker compose up -d --build
# Run integration test
python3 integration_test.py
Testing
# Unit tests
PYTHONPATH=. python3 -m pytest tests/ -v
# Full integration test (requires running containers)
python3 integration_test.py
Troubleshooting
No results returned — SearXNG may be blocked on your network. Check which engines are working (SearXNG is on an internal Docker network, so query it from inside its container):
docker compose exec searxng python -c \
"import urllib.request; print(urllib.request.urlopen('http://localhost:8080/search?q=test&format=json').read()[:500])"
Services not starting — Check for port conflicts:
docker compose ps
docker compose logs
Rate limiting too aggressive — Lower RATE_LIMIT_SECONDS in docker-compose.yml.
Want more search engines? — Edit searxng-settings.yml to enable/disable engines, then restart SearXNG:
docker compose restart searxng
Project Structure
google-mcp/
├── docker-compose.yml # All 3 services
├── searxng-settings.yml # SearXNG engine config
├── google-mcp/
│ ├── Dockerfile
│ └── server.py # Google MCP server
├── duckduckgo-mcp/
│ ├── Dockerfile
│ └── server.py # DuckDuckGo MCP server
├── lib/
│ ├── playwright_manager.py # (legacy, not used)
│ ├── rate_limiter.py # Per-service rate limiter
│ ├── google_search.py # (legacy, not used)
│ └── duckduckgo_search.py # (legacy, not used)
├── tests/
│ ├── test_rate_limiter.py
│ └── test_search_parsing.py
├── integration_test.py # End-to-end MCP test
└── requirements.txt
Notes
- No Playwright in production — The original design used Playwright headless browsers, but Google actively blocks automated browsers. SearXNG is the working solution: it's a self-hosted metasearch aggregator that handles browser rotation internally.
- CloakBrowser / playwright-cli — If you need custom browser automation beyond search, those tools would pair well with this setup. SearXNG already solves the search detection problem.
- Rate limiting — Each MCP server has its own rate limiter to avoid overwhelming SearXNG or the upstream engines.
- SearXNG as single point — Both MCP servers share one SearXNG instance. This is fine for moderate usage. For heavy load, run multiple SearXNG instances behind a load balancer.