Convert · On this device

robots and sitemap

Build robots.txt and sitemap.xml from the host and paths you type.

  1. கோப்பு அல்லது ஒட்டு
  2. முன்னோட்டம்
  3. பதிவிறக்கு
1User-agent: *
2Disallow: /admin/
3Disallow: /drafts/
4Allow: /
6Sitemap: https://example.com/sitemap.xml

robots.txt guide

Build robots and sitemap. Files update on the left. This page does not submit them.

Disallow rulesPaths crawlers should skip.

  • Disallow: /admin/ — 不让爬 /admin/
  • Disallow: /drafts/ — 不让爬 /drafts/

Put the files at your own origin root. This site does not fetch those URLs.

Sitemap 3 条 · 主机 https://example.com
Ad 728×90 · mobile 320×50
How it works, what you get, limits

என்ன செய்கிறது

Build robots.txt and sitemap.xml from the host and paths you type.

எப்படி

Enter host and one path per line, download both files, put them at your site root yourself.

என்ன கிடைக்கும்

Two text files. This does not submit them to search engines or check that URLs load.

விருப்பங்கள் மற்றும் ஒப்பீடு

Host and paths.

Generator only.

வரம்புகள்

No Search Console submit, no crawl of your URLs.

FAQ

Will you submit to Google?

No. Put the files at your own site root.

Disallow: / ?

That blocks crawling the whole site. Check the preview first.