WebIntel is an open-source MCP server that lets Claude, Cursor, Windsurf, Cline, and any MCP-compatible client scrape, search, extract, and monitor the web — locally and for free.
pip install webintel-mcp
Then add it to your MCP client config:
{
"mcpServers": {
"webintel": {
"command": "webintel-mcp"
}
}
}Extract clean, readable content from any URL. Strips ads, navigation, footers, and sidebars automatically.
Search the web with DuckDuckGo. No API key, no account, no usage limits.
Pull SEO title, description, Open Graph tags, Twitter Cards, and canonical URLs.
Categorize every link as internal, external, or social media.
Grab JSON-LD, HTML tables, lists, and any data matching a CSS selector.
Find emails, phone numbers, and social profiles from any public page.
Scrape multiple URLs concurrently. Perfect for research and competitive analysis.
Track pages for changes and get a clean diff when something updates.
| Feature | WebIntel | Other MCP servers |
|---|---|---|
| Search without API key | DuckDuckGo | Often needs Google/Bing key |
| Metadata + links + contacts | Built-in | Usually separate tools |
| Batch scraping | Included | Paid or missing |
| Page monitoring | Included | Paid or missing |
| Self-hosted | Free, unlimited | Varies |