The fix, in one snippet
curl -sI https://example.com/ | grep -i x-robots-tag
curl -s https://example.com/ | grep -i 'name="robots"'Illustrative values. Change the paths, hostnames and options to match your own site before using it.
The sections below explain what is tested, why it matters and the alternatives.
What we test
For every crawled page we read the <meta name="robots"> tag and the X-Robots-Tag response header, and record any page carrying noindex. The check reports the count and lists the pages with the directive we found and where it came from.
A small number of noindex pages is normal and healthy: thank-you pages, internal search results, filtered views, staging content. The finding is raised as a failure only when a large share of the crawl is excluded, because at that point it is more likely to be a mistake than a decision.
Why it matters
Accidental noindex is one of the most common and most expensive SEO faults, and it is nearly invisible: the pages load normally for visitors, nothing looks broken, and traffic simply stops. The usual cause is a staging configuration reaching production, or a CMS-level "discourage search engines" checkbox left on after launch.
It compounds with time. Google revisits and drops the page, then stops recrawling it because it is not indexed, so restoring the tag does not restore the ranking overnight.
The X-Robots-Tag variant is especially easy to miss, because it is invisible in the page source and often applied at the server or CDN level to a whole path.
How to fix it
Decide the rule per page type, then apply it from one place in your templates so it cannot be set twice. To index a page, the simplest correct answer is to send nothing at all.
<!-- indexable: no robots meta tag is needed -->
<meta name="robots" content="index, follow">
<!-- deliberately excluded -->
<meta name="robots" content="noindex, follow">Check the header as well as the tag: curl -sI https://example.com/ | grep -i x-robots-tag. In WordPress, confirm Settings > Reading > "Discourage search engines" is unchecked. Then use the URL Inspection tool in Search Console to request reindexing of the pages that were wrongly excluded.
Where this fits
Pages excluded with noindex is check 2 of 10 that the seo & site health audit runs under indexing and crawlability, ordered the way they are worth fixing. That ordering is the point: Fixing this one while the check above it still fails buys less than it looks like.
Fix this one first
Above it in the same category sits Crawl errors (4xx / 5xx) (high), where uRLs reached during the crawl that returned a client or server error. An attacker who has that does not need this, so it is the better use of the same hour.
What fixing this still leaves open
Pages excluded with noindex closes one route in. Immediately below it: Redirect loops, where a URL that redirects back to something already in its own chain never resolves; Canonical URL missing, where a canonical link tells search engines which URL is the real one when the same content is reachable at several addresses; Canonical points to another URL, where a canonical pointing at a different URL asks search engines to index that URL instead of this one.
Found in the same scan
The seo & site health audit reports this alongside checks from other categories that are at least as serious, including Broken internal links, where an internal link that returns an error is a dead end for the reader and a wasted request for the crawler, and Duplicate body content, where two pages with identical body text are one page at two addresses, and search engines will pick one of them for you. A single run of seo & site health audit answers all of them at once.
The first few lines
You are a senior web engineer. I ran a security and SEO scanner against my site and it reported the finding below. Fix it properly rather than suppressing the symptom.
Finding: Pages excluded with noindex (high severity)
Scanner check id: seo-noindex-pagesThe rest of this prompt names the pages and line numbers we found the problem on, the configuration to change, and the constraints a good answer has to respect. It is free, it just needs an account so the work is not scraped wholesale.
Sign in with GoogleSigning in is free and takes one click. We store your email address and nothing else.