Test the rule
before the release.
Paste a robots.txt file, choose a crawler and test a path. Processing stays in your browser.
Rule result
BlockedGooglebot
/private/reportThe longest matching rule is disallow: /private/
This local parser covers standard user-agent, allow, disallow, wildcard and end-anchor rules. It does not fetch a live file, test HTTP access or determine index eligibility.
Crawling is not indexing.
A robots rule controls crawler access. It does not remove an already known URL from a search index, and blocking a URL can prevent a crawler from seeing a noindex directive on that page.
Always test the deployed file, HTTP status and rendered directives as separate checks before changing production crawl controls.