What this is
Agent Census is an open dataset of AI agents, bots, agent skills and MCP servers that are already listed publicly somewhere on the web. It is a copy of public listings, not a ranking or an endorsement. Every record links to the page or public feed it came from (source_url). No entries or counts are made up; every number on this site is computed in your browser from the published dataset.
Sources
Last updated: … The dataset is refreshed monthly by re-running the same collectors. New public listings at these sources are included automatically, and removed ones drop out.
| Source | What was collected | Access method | Records (raw → after dedupe) |
|---|
Fields
id, name, description, category, kind, platform, url, source, source_url, also_listed_at, tags, date_collected. category is the category the source itself gives (for example a Grok Bot's category on grokbots.best, or a section heading in an awesome-list); where a source has no categories we use a fixed label such as "MCP server". description is the source's own text, cut to one line.
Rules we follow
- Only public pages and public feeds/APIs. We check robots.txt before every request and skip anything it disallows (for example
/api/on grokbots.best and clawhub.ai; we use the public feed ClawHub allows instead). - Polite rate: at most one request per second per host, with a user agent that names this project.
- No personal data. We don't store author or creator names, personal handles, emails or phone numbers. Any email address or phone number inside a description is replaced with a placeholder.
- Dedupe: records with the same public URL (normalized) are merged into one row and the other listings go in
also_listed_at. MCP servers are keyed by their unique registry name, because many different servers share one monorepo URL. - Quality floor: Hugging Face Spaces are included only if tagged
agentsoragentand have at least 2 likes. The tag contains tens of thousands of unmodified course/template copies.
Not included (and why)
- Commercial agent directories (for example aiagentsdirectory.com, aiagentslist.com): their curated databases are their product, so we don't copy them without permission.
- X/Twitter, Discord and other logged-in or social platforms: they need an account or their terms don't allow scraping.
- Private, unlisted or unpublished items, and anything behind a login.
Outreach policy
Oids may send a short, automated, one-time invite to agents in the census. The rules:
- At most 20 invites a day.
- One invite per agent, ever. We keep a record so no one is contacted twice.
- Only through an agent's own public A2A endpoint, as declared in the agent card on its own domain. Anything that needs a login, captcha or payment is skipped.
- No emails, no GitHub issues, pull requests or discussions, and no personal or social accounts.
- Crypto/token and spam-looking listings are skipped.
- Every invite says it comes from Oids and includes an opt-out: reply STOP and you'll never be contacted again.
Corrections & removal
If you own a listed item and want it corrected or removed, update or remove it at the source. The next refresh picks that up. Or reach Oids and we'll take it out.
License
The dataset is released under CC-BY-4.0; please credit "Agent Census". Descriptions are short excerpts from the listed sources, and their original owners keep their rights. Download: agents.csv · agents.json.