Blocked Due to Access Forbidden (403) in Search Console: How to Fix It

A Google 403 Forbidden error in Search Console means Googlebot asked your server for a page and got a 403 status back. The Page indexing report lists those URLs under Blocked due to access forbidden (403), and Google won't index them. Googlebot never sends credentials, so the cause is something that refuses it anyway: a firewall, a bot-protection rule, a rate limit, a country block or a login wall.

If a Google page in your browser shows “403. That's an error.” instead, Google refused your own request and your site isn't involved; the next section separates the two. The Search Console status is one of the reasons in the page indexing report, and its neighbor Blocked due to unauthorized request (401) is the login case.

Which Google 403 you're looking at

• In Search Console: Blocked due to access forbidden (403) in the Page indexing report, or a failed Page fetch in URL Inspection. Your server refused Google. This article covers this case.
• On a Google page in your browser: “403. That's an error.” means Google's server refused your browser, which concerns your access to that Google product, not a site you run. For one of its own programs, Google traces it to being signed in to several Google accounts in the same browser profile, and suggests a separate profile for each account.

What Blocked due to access forbidden (403) means

Google's Page indexing report help says HTTP 403 means the user agent provided credentials but wasn't granted access, that Googlebot never provides credentials, “so your server is returning this error incorrectly”, and that the page won't be indexed. MDN's definition is broader: the server understood the request and refused it, and unlike with a 401, authenticating makes no difference.

Google's page on HTTP status codes adds two consequences. Google doesn't use the content of URLs that return a 4xx code, and an indexed URL that starts returning one is removed from the index. And 4xx codes other than 429 have no effect on crawl rate: Google says not to use 401 or 403 to limit it. A 403 doesn't slow Googlebot down. It only loses the page.

The URL Inspection API reports the failure as pageFetchState: ACCESS_FORBIDDEN (UrlInspectionResult reference); the 401 is ACCESS_DENIED. A robots.txt rule never causes this reason: a disallowed URL is reported as Blocked by robots.txt, and if robots.txt itself answers 403, Google treats it as if no robots.txt existed.

Where the 403 comes from

Bot protection and firewall rules

Start with the security layer: something decided Googlebot looks like a bot, which it is. On Cloudflare, the Block action returns a 403 for most security features. Managed bot settings are built to leave verified crawlers alone: Cloudflare says verified bots have historically been excluded from its default bot configurations, and Vercel says Attack Mode lets search engine crawlers through. Rules you write yourself get no such exemption unless you add it. These catch Googlebot:

• user-agent patterns such as bot|crawler|spider, which match Googlebot's own user agent;
• rules that refuse requests without an Accept-Language header, which Googlebot doesn't send;
• IP, ASN or data-center blocklists broad enough to cover Google's address ranges;
• fake-Googlebot checks that accept one hostname pattern. Google lists two for its common crawlers, crawl-***-***-***-***.googlebot.com and geo-crawl-***-***-***-***.geo.googlebot.com.

Cloudflare's Bot Fight Mode can't be bypassed or skipped with WAF custom rules or Page Rules, though an IP Access rule that matches first keeps it from triggering. If it acts on verified Google IPs, turn it off or move to Super Bot Fight Mode (Pro plans and up), which supports Skip rules.

Rate limits that answer 403

A rate limiter that answers bursts with 403 turns a busy crawl day into lost pages. To make Googlebot back off, Google's guidance on reducing the crawl rate is 500, 503 or 429 for a couple of hours or 1–2 days, no longer: URLs that return them for days may be dropped. Server error (5xx) covers that side.

Country blocks

Google says Googlebot's default IP addresses appear to be based in the USA, though it also crawls from other countries. A rule that admits only one country outside the US refuses its US-based crawls. If the block is there to cut abuse, exempt verified bots, for example in a Cloudflare Block rule:

(ip.src.country ne "IT" and not cf.client.bot)

cf.client.bot is true for known good bots as Cloudflare classifies them, Google's and others. If the block is a legal or licensing one, Google's advice is to treat Googlebot like any user from the country it appears to come from: block it where your users are blocked, allow it where the content is available.

Login walls and private areas

A page that answers anonymous visitors with 403 produces exactly this reason. For private pages that's correct: keep them out of your sitemaps and internal links. To get one indexed, Google's help says to admit non-signed-in users, or to allow Googlebot without authentication after verifying its identity. If Googlebot gets past a wall your users must pass, paywalled content markup (isAccessibleForFree) helps Google tell that apart from cloaking.

Rules on your own server

Below the CDN, look at .htaccess and Nginx rules, IP blocks in the hosting panel, WordPress security plugins and file permissions. The usual pattern is a scraper rule that matches more than intended:

# Blocks Googlebot too: its user agent contains "bot"
if ($http_user_agent ~* "(bot|crawler|spider)") {
    return 403;
}

How to confirm Googlebot is the one being blocked

Before you change a rule, find out which rule refused Google and whether the refused requests were Google's.

1. Inspect an affected URL

Open the reason in the report, pick an example and inspect it: Page fetch and Last crawl show what Googlebot got and when. Then run Test live URL, knowing that Google's URL Inspection help says the live test only confirms whether Google-InspectionTool can access the page. That crawler has its own user agent, without the word “bot”, so a pattern like bot|crawler|spider lets the live test pass while Googlebot keeps getting refused. A failed live test shows no HTML or screenshot either: Google only adds that response data when the result is URL is available to Google. More in the URL Inspection tool.

2. Find the refused requests, at the edge first

A request your CDN blocks never reaches your origin logs. On Cloudflare, Security Events (Analytics, Events tab, as of October 2026) lists mitigated requests, filterable by path, user agent, IP, country, action and the service that acted, and keeps 31 days. For rules on your own server, use the access log. In the combined log format, field 1 is the client IP and field 9 the status:

awk '$9 == 403' /var/log/nginx/access.log | grep -i googlebot \
  | awk '{print $1}' | sort | uniq -c | sort -rn | head

3. Verify the IP addresses

Google warns that Googlebot's user agent is often spoofed, so a user agent in a log line proves nothing. Per Verifying Googlebot, a reverse DNS lookup must end in googlebot.com, google.com or googleusercontent.com, and a forward lookup of that name must return the same IP:

$ host 66.249.66.1
1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com.
$ host crawl-66-249-66-1.googlebot.com
crawl-66-249-66-1.googlebot.com has address 66.249.66.1

For many IPs, match them against common-crawlers.json, Google's CIDR ranges for Googlebot and its other common crawlers. Google regenerates it often (its creationTime read October 30, 2026 when this was written), so fetch it on a schedule instead of pasting ranges into config. If the refused IPs verify, your rule is blocking Googlebot. If they don't, the rule is stopping impostors; keep looking for refused requests that do verify. More in how to verify Googlebot.

4. Don't trust a fake Googlebot from your laptop

Sending Googlebot Desktop's user agent (from Google's crawler list; W.X.Y.Z is the Chrome version) only tests rules that read the user agent:

curl -s -o /dev/null -D - \
  -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36" \
  https://example.com/pricing | grep -iE "^HTTP|cf-mitigated"

A 403 means a user-agent rule exists, or a fake-bot check caught you, as it should. A 200 says nothing about rules that check IPs or verified-bot status. A cf-mitigated: challenge header means Cloudflare served a challenge instead of your page. Site-wide, the Crawl stats report (Settings, root-level properties only) counts 403s under Other client error (4XX); a jump there can date the rule.

How to fix a Google 403 and get the page back

1. Decide whether the URL should be public. If not, the 403 is fine. Take the URL out of your sitemaps and internal links.
2. Change the rule, not the page. Exempt verified bots from country and bot rules, answer rate limits with 429 or 503, and answer login-only pages with 401. Don't allowlist the string Googlebot: anyone can send it.
3. Run the live test again. Page fetch should read Successful. If the rule was site-wide, test URLs from several sections.
4. Ask Google to come back. Request indexing for the URLs that matter most. There's a daily quota, and repeat requests won't get a URL crawled faster; crawling can take a few days to a few weeks. For the whole group, click Validate fix, which typically takes up to about two weeks (Validate fix in Search Console).

How IndexCyborg flags 403s

IndexCyborg's technical checks fetch the URLs in your sitemaps, raise Page returns HTTP 403 as a critical issue with the status as evidence, and resolve it once the page stops returning it. They send IndexCyborg's own user agent (IndexCyborgBot), so they're a second opinion: if both IndexCyborg and Google get a 403, the rule refuses non-browser clients in general; if only Google does, look at rules keyed on Google's user agent, IP ranges or crawl country. With Search Console connected, the URL Inspection result is stored word for word, Google's fetch state included (page_fetch_state), and GET /v1/urls/{id} or the MCP tool get_indexing_status returns it next to the latest technical check.

IndexCyborg never queues Request indexing for a page whose last technical check didn't return 200, even one you ask for by name: it comes back as not_requestable. After the fix, queue a fresh check (POST /v1/sites/{id}/check, or check_technical_seo with queue_fresh_check) and, once it returns 200, queue the URL with request_google_indexing. The Chrome extension clicks Request indexing in your own Search Console session, at most 10 per property per rolling 24 hours. That puts the URL in Google's crawl queue; it doesn't make Google index it. More on the features page.

Frequently asked questions