Yes. Completely free. No API keys, no signup, no credit card, no "free tier with limits that conveniently force you into a paid plan." Just hit the endpoints. We don't need to monetize your usage data because we don't collect any.
No. Every endpoint is open. curl agentsweb.org/fetch?url=... and you're done. We believe access to public web content shouldn't require you to create an account with a company that then tracks every query you make across every product they own.
It depends on the trust level and the domain:
A background cron job also pre-warms entries approaching expiry. When a response includes "stale": true, the content is approaching its TTL — still valid, but a refresh is coming.
Yes. The entire codebase is open source on GitHub. It's a single Cloudflare Worker with KV. Deploy it to your own Cloudflare account with npx wrangler deploy. You'll need a KV namespace and an admin secret.
agentsweb operates under DMCA 512(b) — the system caching safe harbor. We cache temporary, transformed copies of publicly accessible web pages. Content owners can request removal via our DMCA policy, and takedowns are permanent. We also respect robots.txt directives. Caching the web for search and retrieval has been considered legal since — well, since a certain search engine built their entire trillion-dollar business on it.
agentsweb only caches publicly accessible pages. Login walls and subscription gates are detected and rejected automatically. If a page requires authentication to read, we can't cache it and we won't try.
If you submit 5 pieces of content that fail security validation, your IP is auto-banned for 1 hour. This prevents cache poisoning attempts.
Yes. PUT / with a JSON body containing url, markdown, and source. Your content passes through the full security pipeline. If it's clean, it enters the cache at trust level 1. Other agents can confirm it to increase trust.
Any HTTP client works. See our integrations page for specific instructions for Claude Code, Cursor, Windsurf, Codex, LangChain, Python, and curl.
Short answer: the poison self-destructs. Long answer: all content passes through 30+ prompt injection patterns, XSS filters, and unicode steganography detection. Even if something slips through, the self-healing consensus means the next legitimate agent to read the entry will verify it and replace the poisoned version. Entries with trust level 2+ can't be overwritten by a single agent.
Yes. intercept-mcp is the MCP server that integrates agentsweb with Claude Code, Cursor, Windsurf, and any MCP-compatible client. Install with npx -y intercept-mcp.
We aggregate results from multiple independent search backends — no single company decides what your agent sees. Unlike some search providers, we don't prioritize results from our own products, inject ads disguised as results, or deprecate our API every two years to force you onto a more expensive one.
/fetch?url=... will fetch the page live if it's not cached. /?url=... is cache-only — it returns 404 if the page hasn't been cached. Use /fetch when you want the content no matter what. Use /?url= when you want speed and are okay with a miss.
Email dmca@agentsweb.org. We respond within 24 hours. Takedowns are permanent and immediate.