The fix, in one snippet
# robots.txt
Allow: /members/
<!-- on the page -->
<meta name="robots" content="noindex, follow">Illustrative values. Change the paths, hostnames and options to match your own site before using it.
The sections below explain what is tested, why it matters and the alternatives.
What we test
While crawling, every internal link is tested against the site's own robots.txt before it is followed. URLs that are linked from a crawled page but disallowed are collected and reported here. We use the same matching rules as a search engine: the longest matching Allow or Disallow path wins, and our own user-agent group is checked alongside *.
Why it matters
Blocking a URL in robots.txt stops it being crawled, not indexed. If other pages link to it, the URL can still appear in search results as a bare link with no title or snippet, because the engine was never allowed to read the page it is listing.
Link equity flowing into a blocked URL is also dead weight: the link is followed as far as the rule and then stops, so the authority passed through it reaches nothing.
Most often this is a leftover rule. A path was blocked years ago during a migration, the section came back, and the rule stayed. The navigation and the crawl rules now disagree about what the site is.
How to fix it
Decide which of the two statements is the true one, then make the other match.
If the page should be found, remove the Disallow line. If it should not, remove the internal links to it as well, or mark them so they are not presented as navigation. For pages that must exist but must not be indexed, allow crawling and serve noindex instead, which is the only combination that reliably removes a URL from the index:
# robots.txt - allow the crawl so the directive can be read
User-agent: *
Allow: /members/
# and on the page itself
<meta name="robots" content="noindex, follow">Verify with the robots.txt Tester in Search Console, which shows exactly which rule matched a URL.
Where this fits
Linked pages blocked by robots.txt is check 6 of 10 that the seo & site health audit runs under indexing and crawlability, ordered the way they are worth fixing. That ordering is the point: Fixing this one while the check above it still fails buys less than it looks like.
Fix this one first
Above it in the same category sits Canonical points to another URL (medium), where a canonical pointing at a different URL asks search engines to index that URL instead of this one. An attacker who has that does not need this, so it is the better use of the same hour.
What fixing this still leaves open
Linked pages blocked by robots.txt closes one route in. Immediately below it: Redirect chains, where a URL that reaches its destination through two or more redirects costs every visitor and every crawler extra round trips; Sitemap URLs are live and indexable, where every URL in a sitemap is a claim that the page exists and should be indexed; XML sitemap published and declared, where an XML sitemap lists the URLs you want crawled and when they last changed.
Found in the same scan
The seo & site health audit reports this alongside checks from other categories that are at least as serious, including Broken internal links, where an internal link that returns an error is a dead end for the reader and a wasted request for the crawler, and Duplicate body content, where two pages with identical body text are one page at two addresses, and search engines will pick one of them for you. A single run of seo & site health audit answers all of them at once.
The first few lines
You are a senior web engineer. I ran a security and SEO scanner against my site and it reported the finding below. Fix it properly rather than suppressing the symptom.
Finding: Linked pages blocked by robots.txt (medium severity)
Scanner check id: seo-blocked-by-robots
What this check tests: A page you link to internally but disallow in robots.txt sends contradictory signals and wastes the link.The rest of this prompt names the pages and line numbers we found the problem on, the configuration to change, and the constraints a good answer has to respect. It is free, it just needs an account so the work is not scraped wholesale.
Sign in with GoogleSigning in is free and takes one click. We store your email address and nothing else.