> about

Every AI agent on earth independently fetches the same web pages. Same 403s. Same captchas. Same raw HTML. Over and over. Millions of times a day.

That's insane.

agentsweb.org exists because the web was not built for AI. Pages are HTML, not markdown. Servers block bots. Captchas assume you have eyes. Paywalls assume you have a credit card.

So we built a shared layer. The first agent to successfully read a page caches the clean markdown for every agent after it. The network gets smarter with every request.

mission

agentsweb.org is a public interest infrastructure project. We believe AI agents should have open, equitable access to the web — the same web that humans use every day. The internet's knowledge shouldn't be locked behind anti-bot walls that only well-funded companies can bypass.

This is shared infrastructure for the global AI ecosystem. Not a product. Not a startup. Not a monetization play. Public infrastructure, like DNS or NTP — the kind of thing that should just exist.

why .org

We chose the .org domain deliberately. agentsweb.org is a non-commercial, community-driven project operating in the public interest. The .org namespace has represented public benefit organizations since 1985. We take that seriously.

• Non-commercial: No ads. No tracking. No data sales. No VC funding. No paid tiers. Free for everyone.
• Open source: Every line of code is public. Anyone can audit, fork, or contribute.
• Community-governed: The cache is built by the community of agents that use it. Every agent contributes. Every contribution makes the network better for everyone.
• Public benefit: We exist to solve a shared problem — AI agents can't read the web efficiently. We make that problem go away. For free. Forever.

the self-healing part

Anyone can contribute to the cache. So how do you prevent poisoning? Consensus. Entries gain trust as independent sources confirm the content. An attacker would need to control multiple IP addresses and somehow produce content that passes 30+ prompt injection patterns, XSS filters, unicode steganography detection, and repetition analysis. And even if they did, the poison self-destructs on the next legitimate read.

Good luck.

open source

The entire stack is open source under the MIT license:

• agentsweb — the Cloudflare Worker (this site)
• intercept-mcp — the MCP server that powers the fetch pipeline
• agentsweb-python — Python SDK

Contributions welcome. File issues. Submit PRs. Fork it and run your own. That's the point.

who

Built and maintained by bighippoman. Powered by Cloudflare Workers + KV.

No venture capital. No corporate sponsor. No tracking pixels. No cookies. No analytics beyond anonymous request counters. Just public infrastructure for AI agents, run by one person who thinks this should exist.

< back to agentsweb.org