The fix, in one snippet
# list only what you would want indexed
Post.objects.filter(status='published', noindex=False)Illustrative values. Change the paths, hostnames and options to match your own site before using it.
The sections below explain what is tested, why it matters and the alternatives.
What we test
We take a sample of <loc> entries from the sitemap and compare them with what the crawl actually found. A URL fails when it returned a 4xx or 5xx status, or when the page it resolves to carries a noindex directive in its meta robots tag or X-Robots-Tag header.
Only URLs that appear in both the sitemap and the crawl are judged, so this is a spot check rather than a full sitemap validation. A clean result means the overlap was consistent, not that every one of several thousand URLs is live.
Why it matters
A sitemap full of dead URLs trains the crawler to distrust it. Google's documentation is explicit that sitemaps containing large numbers of errors are processed less often, which delays discovery of the URLs that are fine.
The contradiction of a noindex page listed in a sitemap is worse, because it sends two opposite instructions: the sitemap asks for indexing, the page refuses it. Search Console reports these as "Submitted URL marked noindex" and the page is dropped, but the crawl budget was spent anyway.
Both problems usually come from the same cause: the sitemap is generated from a different source than the site, and the two have drifted.
How to fix it
Generate the sitemap from live, canonical, indexable URLs only. In practice that means filtering at generation time:
# Django, as an example of the rule rather than the framework
class PostSitemap(Sitemap):
def items(self):
return Post.objects.filter(status="published", noindex=False)
def lastmod(self, obj):
return obj.updated_atRemove redirected URLs: list the destination, not the source. Remove URLs that canonicalise elsewhere. If a page is deliberately noindex, it does not belong in the sitemap at all.
After a rebuild, check Search Console's sitemap report for the discovered-versus-indexed gap; a large gap usually points at duplicate or thin pages rather than at a sitemap problem.
Where this fits
Sitemap URLs are live and indexable is check 8 of 10 that the seo & site health audit runs under indexing and crawlability, ordered the way they are worth fixing. That ordering is the point: Fixing this one while the check above it still fails buys less than it looks like.
Fix this one first
Above it in the same category sits Redirect chains (medium), where a URL that reaches its destination through two or more redirects costs every visitor and every crawler extra round trips. An attacker who has that does not need this, so it is the better use of the same hour.
What fixing this still leaves open
Sitemap URLs are live and indexable closes one route in. Immediately below it: XML sitemap published and declared, where an XML sitemap lists the URLs you want crawled and when they last changed; robots.txt present and usable, where robots.txt is the first file a crawler asks for.
Found in the same scan
The seo & site health audit reports this alongside checks from other categories that are at least as serious, including Broken internal links, where an internal link that returns an error is a dead end for the reader and a wasted request for the crawler, and Duplicate body content, where two pages with identical body text are one page at two addresses, and search engines will pick one of them for you. A single run of seo & site health audit answers all of them at once.
The first few lines
You are a senior web engineer. I ran a security and SEO scanner against my site and it reported the finding below. Fix it properly rather than suppressing the symptom.
Finding: Sitemap URLs are live and indexable (medium severity)
Scanner check id: seo-sitemap-accuracyThe rest of this prompt names the pages and line numbers we found the problem on, the configuration to change, and the constraints a good answer has to respect. It is free, it just needs an account so the work is not scraped wholesale.
Sign in with GoogleSigning in is free and takes one click. We store your email address and nothing else.