AI agents
What is ClaudeBot, and what does it read on your site?
ClaudeBot is the web crawler operated by Anthropic to support Claude. It reads public web pages and identifies itself with the user-agent token "ClaudeBot". Like other reputable AI crawlers, it respects robots.txt, so you control which paths it may access.
See your site's llms.txt in seconds — free
Generate my llms.txtHow does ClaudeBot affect what Claude says about you?
When someone asks Claude about your business, Claude may draw on pages ClaudeBot has read. Clear, structured content makes it more likely Claude describes you accurately. An llms.txt file gives ClaudeBot a concise, correct summary of what you do and what matters.
FAQ
How do I block or allow ClaudeBot?
Add a robots.txt rule for User-agent: ClaudeBot to disallow paths, or allow it to let Claude understand your site. AgentFront also logs ClaudeBot visits, so you can see exactly when and what it reads.