Public URL check
Check robots.txt
Check robots.txt by URL: see whether the file is available, what status code it returns, whether sitemap.xml responds and whether public indexability signals look risky.
robots.txt check by URL
Enter a website or page URL and URLCompass looks for robots.txt on the domain, checks the status code, file availability and basic search-engine signals.
What matters in robots.txt
The report shows robots.txt, sitemap.xml, canonical, robots meta and noindex signals so you can see whether a page may be blocked from search.
Sitemap and directives
URLCompass checks whether sitemap.xml responds and whether the basic technical files are available after a launch, migration or SEO change.
Clear limits
The check does not replace Google Search Console, server logs or a manual review of every crawler rule, but it helps reveal obvious robots.txt availability issues.
Example result
What the report can show before you open it
The exact values depend on the URL, but the report highlights practical signals instead of a raw technical dump.
robots.txt status, robots meta and basic signals that can block indexing.
Canonical, robots meta, noindex, robots.txt, sitemap.xml, hreflang and HTML language.
sitemap.xml status, sitemap index and child sitemaps when available.
The report groups output into critical issues, recommended fixes and useful context.
Report sections
The scan keeps the result readable
robots.txt
File presence, status code and availability of the standard /robots.txt location.
Indexability
Robots meta, noindex, canonical and other public signals that can affect search visibility.
Sitemap
sitemap.xml availability and basic sitemap response near robots.txt.
Server response
Status code, final URL, redirects, response time and HTML size of the checked page.
FAQ
Common questions
Does URLCompass need access to the website admin area?
No. The report is built from public signals available by URL: HTML, headers, DNS, TLS, robots.txt and sitemap.xml.
Can it show exact traffic or private analytics?
No. Exact traffic requires analytics access or paid data sources. URLCompass intentionally separates public checks from private metrics.
What should I do after the report?
Start with failed checks and warnings: indexability, HTTPS/TLS, security headers, page speed, missing metadata and weak DNS/mail protection.
Which URLs can be checked?
Use public http or https pages. Localhost, private IP addresses and closed admin pages are blocked to keep the checker safe.