Free analyzer
Robots.txt and llms.txt analyzer
Check whether your robots.txt and llms.txt files give crawlers and AI agents clear, useful discovery instructions.
Analyze discovery files
Paste a domain or URL. The tool checks /robots.txt and /llms.txt for crawler access, sitemap hints, Markdown structure, and useful links.
Paste a homepage URL or domain. The tool checks robots.txt and llms.txt at the site root.
Crawler discovery
Confirm crawlers can find your sitemap and are not accidentally blocked.
Agent browsing
Check whether llms.txt has a clear Markdown summary and useful links for AI agents.
Before Search Console validation
Use this before requesting validation or submitting new pages for indexing.
How to use the result
- robots.txt access rules
- Sitemap directives
- llms.txt headings and links
Use this result as a discovery-file sanity check. It does not replace content quality, rendered HTML checks, or Search Console validation.
If discovery files are missing or unclear
Add a simple robots.txt with a sitemap reference and an llms.txt file with clear Markdown links to important pages.