Robots.txt Generator
Tell crawlers what to fetch and what to skip. Avoid accidental indexing mistakes.
User-agent: *
Allow: /
A word of caution
Disallow: / blocks the entire site from search engines — usually a mistake. Most sites only need to block /admin, /cart, and search results pages.