- Robodex
- AI search
AI search bots
Indexes the web to power a chatbot’s real-time search and citations; finds answers without a user-supplied URL.
Verified
Recently added
Anthropic
Claude-SearchBot is Anthropic's search crawler used to retrieve web content for Claude search-related experiences.
VerifiedAI search
Meta
The Meta-WebIndexer crawler navigates the web to improve Meta AI search result quality for users. In doing so, Meta analyzes online content to enhance the relevance and accuracy of Meta AI. Allowing Meta-WebIndexer in your robots.txt file helps us cite and link to your content in Meta AI’s responses.
VerifiedAI search
OpenAI
OAI-SearchBot is OpenAI's search crawler used to surface and retrieve web content for search-related AI experiences.
VerifiedAI search