Noindex checker
Find every noindex that applies to a URL: meta robots and googlebot tags, the X-Robots-Tag header, and the robots.txt rule that decides whether Google can see them at all.
Up to 10 checks a minute. Pages are fetched from IndexCyborg's servers; private and local addresses are refused.
Where noindex can hide
Google documents two ways to set noindex, and the checker reads both:
- Meta tags in the
<head>:<meta name=“robots” content=“noindex”>for every crawler, orname=“googlebot”for Google only. Tags for other crawlers (bingbot and so on) are listed too. - The X-Robots-Tag header, for all crawlers or prefixed with a crawler name. It's set by the server or CDN, not the page, which makes it the usual suspect when a page looks clean.
none counts as noindex too: it means noindex, nofollow. The noindex tag checker reports the final URL after any redirect, since that's the page Google evaluates. The meta robots tag and X-Robots-Tag cover every directive.
noindex and robots.txt don't mix
For noindex to work, Google has to crawl the page and read it. If robots.txt blocks the URL, Google never sees the tag, and the URL can still be indexed from links, without its content. The checker flags that combination. Google also doesn't support a noindex line inside robots.txt itself. Robots.txt vs noindex covers when to use which.
In Search Console
Pages Google left out because of the directive appear in the Page indexing report as “Excluded by ‘noindex’ tag” (Google's help page calls it “URL marked ‘noindex’”). If that list holds pages you want indexed, remove the directive, then ask for a recrawl. See Excluded by ‘noindex’ tag and the noindex tag guide.
How IndexCyborg handles this
Every new and changed page in your sitemap is checked for noindex in meta robots and in the X-Robots-Tag header. A page that has one gets a critical issue with the directive as evidence, and the issue resolves itself once a later check finds it gone. noindex pages are never picked for Request indexing. See the technical SEO checklist for the other checks.
Frequently asked questions
- How do I check if a page has a noindex tag?
- Enter its URL. The checker reads every robots meta tag in the page's head and the X-Robots-Tag response header, shows which crawler each one targets, and marks the ones that contain noindex or none.
- Can noindex be set without a meta tag?
- Yes, with the X-Robots-Tag HTTP header, which also works for PDFs and other files. It can target all crawlers (X-Robots-Tag: noindex) or one (X-Robots-Tag: googlebot: noindex). It doesn't show in the page source, which is why it's easy to miss.
- Why is my noindex page still in Google?
- Either Google hasn't recrawled it since you added the tag, or robots.txt blocks the page, so Google never sees the noindex. Allow crawling, then use URL Inspection in Search Console to ask for a recrawl.