Robots.txt generator

Set who can crawl your site and what they can reach — choose a preset or add per-bot rules, a Crawl-delay and your Sitemap. Copy or download a valid robots.txt. Nothing is uploaded.

Quick start

Crawler rules

Sitemap URL (optional)

Host (optional)

Your robots.txt Place this file at the root of your site, reachable at /robots.txt.

Built entirely in your browser. No upload, no sign-up — your rules stay on your device.

How it works

STEP 1

Pick a starting point

Choose Allow all, Block all, or Custom — then fine-tune the rules below.

STEP 2

Set per-bot rules

Name a User-agent (or * for all) and list the paths to Allow or Disallow. Add a Crawl-delay or your Sitemap if you want.

STEP 3

Copy or download

Grab a valid robots.txt and put it at your site root, reachable at /robots.txt. Nothing is uploaded.