SEO-001
robots.txt is missing
LowDefault severity · SEO / Quality · SEO, quality and performance
Last reviewed
What it checks
Whether /robots.txt returns a plain-text file.
Why it matters
Crawlers get no guidance or sitemap location, and some hosts serve an HTML page there instead.
Recommended fix
Add a robots.txt that allows crawling and points to your sitemap.
How to verify
Open /robots.txt on the live site and confirm it returns plain text with a Sitemap line.
Example fix prompt
Generic prompt (your report adds the evidence)
Please fix this production issue in my app.
Issue SEO-001: robots.txt is missing
Site: https://your-app.example
First confirm the issue in the code or live config. If it is not present, say so and change nothing.
Task: Add a robots.txt served at /robots.txt that allows all crawlers on public pages, disallows private areas such as /dashboard and /api, and includes a `Sitemap:` line with the absolute sitemap URL.
Constraints: make the smallest change that fixes this, keep the current design and behaviour, never paste secret values into code or chat, and tell me which files you changed.
Done when: Open /robots.txt on the live site and confirm it returns plain text with a Sitemap line.