Cloudflare setup
Use Cloudflare DNS, redirects, and Workers without creating loops or exposing private routes to crawler rendering.
On this page
Separate DNS from request handling
Cloudflare DNS decides where a hostname resolves. Redirect Rules and Workers can change how requests are handled. Start by deciding which hostname is canonical and which system is the origin for the visitor-facing site.
If you use a Worker to route crawler traffic, keep it focused on public HTML GET requests. Pass normal visitors through to the origin and skip assets, APIs, authentication, dashboard, checkout, and other private paths.
Avoid origin loops
The Worker must not fetch the same public hostname in a way that invokes itself again. Use a separate origin hostname, a direct provider URL, or a clearly configured bypass. Test redirects carefully after each DNS or Worker change.
Keep credentials server-side
Store render-service credentials as Worker secrets. Do not embed them in JavaScript sent to the browser, in URL parameters, or in repository files.
Verify the production hostname
Use the exact public hostname with both normal and crawler-style requests. Confirm the final URL, status, cache behavior, canonical URL, and readable content. If Cloudflare features such as bot protection or access rules are enabled, make sure they do not block the rendering service from fetching the public origin.
Robots.txt can guide crawlers, but it does not protect private content. Keep access control on the origin and application routes.
Instructions for your connected site
Open account setup for domain-specific values, diagnostics and copy-ready configuration. These stay private to your workspace.