Skip to content
Application notes · 6 min read

“Crawled - currently not indexed” and the other things Google tells you

The URL Inspection API gives Google's verdict on a page in Google's own words. A field guide to the states you'll see most, and what to do about each.

For every URL you ask about, Google answers with a short phrase: Submitted and indexed, Crawled - currently not indexed, Page with redirect. Those phrases are the most precise signal you get from Google about a page, and they are easy to misread. Here is what the common ones mean.

Where the wording comes from

Search Console's URL Inspection API returns an indexStatusResult for each URL. Its fields include a verdict, the coverageState phrase, robotsTxtState, indexingState, pageFetchState, lastCrawlTime and the canonical URL Google picked next to the one you declared. The API allows 2,000 inspections a day and 600 a minute per property.

IndexCyborg stores coverageState exactly as Google returns it and files the URL under a status bucket so you can filter. The bucket helps you sort. The verbatim phrase is the evidence.

Indexed

Submitted and indexed and Indexed, not submitted in sitemap both mean Google has the page in its index. The second one is a hint that your sitemap is missing a URL Google found anyway.

Discovered - currently not indexed

Google knows the URL exists but hasn't crawled it yet. Google's own explanation is that it wanted to crawl the page but expected that to overload the site, so it rescheduled. On small sites this usually clears by itself. On large ones it can mean Google isn't convinced the pages are worth the crawl.

What helps: link to the page from pages that are already indexed, keep it in the sitemap, keep the server fast, and request indexing for the pages that matter most.

Crawled - currently not indexed

Google fetched the page and chose not to index it, for now. Google says it may or may not be indexed later, and that there's no need to resubmit it. This is not an error on your side and not a penalty. It is a judgement about the page.

What helps: make the page clearly different from similar pages, merge thin pages into stronger ones, and add internal links. If you changed the page substantially, one Request indexing is reasonable. Repeating it is not.

Excluded on purpose, or by accident

  • Excluded by ‘noindex’ tag: the page asks not to be indexed, through a meta robots tag or an X-Robots-Tag header. Fine if deliberate. If not, IndexCyborg's technical check has already flagged it as an issue.
  • Blocked by robots.txt: Googlebot isn't allowed to fetch the page. A blocked page can still appear in results without a description if other pages link to it.
  • Page with redirect: the URL redirects, and the target is what gets indexed. Take redirected URLs out of your sitemap.
  • Alternate page with proper canonical tag, Duplicate without user-selected canonical and Duplicate, Google chose different canonical than user: Google treats the page as a copy of another one and indexes that one instead. Compare the canonical you declared with the one Google picked.

Errors and unknowns

  • Not found (404), Soft 404 and Server error (5xx): Google couldn't get a normal page. Fix the response, or remove the URL from the sitemap if it's gone.
  • URL is unknown to Google: Google has never seen the URL. Make sure it is linked and in a sitemap Google reads.

Two rules IndexCyborg follows

A page Google hasn't been asked about is Not checked. It is never counted as indexed, and never as a problem. And coverage is always stated against the pages actually inspected: “182 of 214 inspected”, not a bare 85% that silently assumes the uninspected ones are fine.