Key takeaways

  • Google's own guide to debugging traffic drops lists algorithm updates, technical errors, security threats, manual actions, seasonality, and migrations. AI is not on that list, so start there.
  • One comparison settles most arguments: if impressions AND clicks fell, you have a ranking or technical problem; if impressions held but clicks fell, a SERP feature like an AI Overview is eating the click.
  • Some lost traffic has not left, it has been re-bucketed. Roughly 35 to 70 percent of real AI-referral sessions land in Direct in GA4 because they arrive with no referrer.
  • AI is real but smaller than it feels. Similarweb estimates AI platforms sent about 1.13 billion referral visits in June 2025 against Google's 191 billion, so for most sites lost search clicks dwarf gained AI referrals.
  • Verify AI crawlers before you trust the user-agent string, using the published lists at openai.com/gptbot.json and Anthropic's crawler docs. And Google-Extended cannot cause a Search drop, because it is only a robots.txt token.

If your traffic is down and your first instinct is to blame AI, pause. AI Overviews and AI crawlers are a real force on publisher traffic, but a decline is more often caused by something ordinary and verifiable: a Google algorithm update, a crawl or indexing error, a security flag, a manual action, plain seasonality, a botched migration, or a reporting glitch. None of those are AI, and every one of them shows up differently in your data. The honest method is to check the boring causes first, using two things you already have: Search Console (impressions versus clicks) and a year-on-year comparison. Only once those are ruled out should AI move up your list of suspects. This guide walks through exactly how to do that.

Why is my website traffic down?

The most useful place to start is not a hot take about AI, it is Google's own documentation. Google Search Central publishes a formal guide to debugging Search traffic drops, and it lists the causes it considers most likely. In rough order those are: algorithmic ranking updates, technical problems (server downtime, robots.txt errors, 404s, a misplaced noindex tag), security threats such as malware or phishing, spam-policy manual actions, seasonality and changing interests, site migrations, and reporting glitches in the tools themselves.

Read that list again and notice what is missing. AI is not on it. Google does not name AI Overviews, ChatGPT, or crawlers as a primary cause of a Search-traffic drop. That does not mean AI has no effect on clicks, it clearly does, and we will get to the numbers. It means that when a publisher opens their analytics, sees a decline, and immediately concludes that AI killed their traffic, they are skipping past the more probable, more fixable explanations. A misplaced noindex tag, or a core update that moved you from position 4 to position 20, will cost you far more traffic, far faster, than any AI summary.

The discipline here is simple. Treat AI as a diagnosis of exclusion, not a first guess. Work through the verifiable causes, each of which leaves a distinct fingerprint in your data, before you settle on the one that cannot be fully proven from your own logs.

Is my traffic drop caused by AI or something else?

There is a single diagnostic that separates most causes, and it comes straight from Google's guidance: compare impressions against clicks in Search Console.

What you see in Search ConsoleWhat it most likely meansWhere to look next
Impressions and clicks both fall togetherAn algorithmic or technical loss. You are ranking for fewer things, or being shown less often.Ranking-updates page, Page Indexing report, Crawl Stats, Manual Actions, Security Issues
Impressions hold steady, clicks fallA SERP feature (such as an AI Overview) or a weak title or snippet is stealing the click. You still rank, people just stop clicking through.AI Overview presence on your top queries, title and meta-description quality

This one comparison does most of the work. If both lines drop, AI is not your problem, ranking or indexing is. If impressions are flat but clicks are sliding, then a SERP feature eating the click becomes a genuine candidate, and an AI Overview is the obvious one to check. It is worth being precise about scale here too. A position moving from 2 to 4 is normal churn and not worth a panic. A position moving from 4 to 29 is a real ranking problem that a core update or a technical fault will explain far better than AI.

When you suspect a ranking cause, the fastest confirmation is a date match. Google ran multiple broad core updates through 2025, each bringing ranking volatility. If your drop lines up with a documented update date, that is a ranking story, not an AI story, and it should be checked against Google's ranking-updates page.

How do I rule out seasonality and algorithm updates?

Seasonality is the cause publishers most often forget and most often mistake for something sinister. Google names it explicitly, and the method to test it is straightforward. Do not rely on a 28-day window against the previous 28 days, because that comparison is blind to annual cycles. Instead, compare the last 90 days against the same period last year, and cross-check demand with Google Trends.

The logic is clean. If demand for your topic fell across the whole industry, that shows up in Trends as an industry-wide decline, and it is seasonality, not your fault. If demand held steady everywhere but your traffic alone fell, that isolation points at a site-specific technical or ranking problem you can actually fix. A worked example: a gardening publisher sees traffic halve from July to November. Panic says AI Overviews. Google Trends says searches for how to prune roses collapse every autumn regardless of site. Year-on-year says this November matches last November almost exactly. That is seasonality, and the correct response is to do nothing except wait for spring.

For algorithm updates, the test is the date-line-up described above, applied deliberately. Pull the exact day your traffic broke, then set it against the published update dates. If they coincide, you have your answer, and the remedy is content and quality work aligned to what the update rewarded, not crawler blocking. The value of doing this properly is that it stops you from spending a fortnight editing robots.txt when the real event was a core update you could have spotted in five minutes.

CheckMethodRules out
SeasonalityLast 90 days vs same period last year, plus Google Trends for the topicNormal annual demand cycles
Algorithm updateMatch the drop date to Google's published ranking-updates pageCore-update ranking volatility
Technical lossPage Indexing report, Crawl Stats, robots.txt, noindex auditCrawl and indexing faults
PenaltiesManual Actions and Security Issues reportsManual actions and malware flags

What data proves whether AI is the cause?

AI is a real factor, and the click-loss numbers are large enough to take seriously. The Pew Research Center analysed 68,879 Google searches and found that when a Google AI summary was present, users clicked a traditional search result in only 8 percent of visits, versus 15 percent without a summary, close to half. Clicks on the links inside the summary itself happened in just 1 percent of visits. The same study found users ended their browsing session entirely after 26 percent of pages that showed an AI summary, against 16 percent without one, so summaries measurably increase zero-click behaviour. Around 18 percent of the searches tracked produced an AI summary at all.

Ahrefs puts a sharper number on the ranking cost. Across 300,000 keywords, it found that the presence of an AI Overview correlates with a 58 percent lower average click-through rate for the top-ranking page, as of its December 2025 update, up from a 34.5 percent reduction it measured in April 2025. So the effect is real and it is growing.

But scale matters, and this is where the proportion goes wrong in most people's heads. Similarweb data, reported by TechCrunch, estimates AI platforms generated over 1.13 billion referral visits in June 2025 against 191 billion referrals from Google Search. AI referrals grew 357 percent year on year, but they still equal roughly 0.6 percent of Google's search referrals. For most sites, the clicks lost to Google's own AI Overviews dwarf the visits gained from AI chatbots. So the proof of an AI cause is specific: impressions steady, clicks down, and an AI Overview verifiably present on the queries that lost the clicks. If impressions also fell, the AI story does not hold, no matter how the timing feels.

There is a second, quieter form of proof, and it points the other way. Some of your apparent drop is a measurement artefact. Roughly 35 to 70 percent of real AI-referral sessions arrive with no referrer header and get dumped into Direct in GA4 rather than shown as AI traffic. That traffic has not fallen, it has been re-bucketed. As of 7 June 2026, GA4 rolled out a native AI Assistant default channel covering ChatGPT, Gemini, Copilot, Grok, and DeepSeek, which helps, but it classifies on the referrer alone, so referrer-less sessions still land in Direct, and Perplexity is not in the definition and still lands in Referral. Before you mourn lost sessions, check whether Direct quietly rose by a similar amount over the same window. To close the gap, go to Admin, then Data settings, then Channel groups, and add a custom regex rule for AI tools above the Referral rule.

How do I confirm what is actually crawling my site?

"AI is scraping me" is a different claim from "my human traffic is down", and you can verify the first one with primary facts rather than guesses. Every major AI company now publishes its exact user-agent strings, and OpenAI also publishes authorised IP lists, so you can confirm what is genuinely hitting your server. The critical rule: never trust the user-agent header alone, because anyone can spoof it. Verify against the vendor's published data.

  • OpenAI publishes its user-agents and IP files at openai.com/gptbot.json, searchbot.json, and chatgpt-user.json, covering GPTBot, ChatGPT-User, and OAI-SearchBot. Match a logged bot's source IP to those lists to confirm it is genuine.
  • Anthropic documents ClaudeBot, Claude-User, and Claude-SearchBot, and publishes an authorised crawler IP list, in its crawler help article. Note that Anthropic advises against IP blocking, because its bots run from public cloud ranges. The legacy identifiers Claude-Web and anthropic-ai are deprecated and should not appear in current traffic, so seeing them flags a stale robots.txt rule.

It also helps to know which bots can even send you visitors, because most cannot. There are three roles, and only two of them ever drive traffic.

RoleExamplesCan it send you visitors?
Training crawlersGPTBot, ClaudeBot, the Google-Extended tokenNo. They consume bandwidth and never send a visitor.
Search-index crawlersOAI-SearchBot, Claude-SearchBot, PerplexityBotYes, indirectly, by building the AI answer index.
User or retrieval fetchersChatGPT-User, Claude-UserYes, they fetch your page live when a user asks.

This matters for diagnosis. If your logs show heavy GPTBot or ClaudeBot activity but your human sessions are flat, that is training-crawler bandwidth, not a traffic cause. Blocking those bots protects your content from training, but it will not recover a single visitor, because they were never sending any.

One persistent myth deserves killing here. Google-Extended is not a crawler. It has no separate user-agent and no separate IP, because the crawler is still Googlebot. Google-Extended is only a robots.txt token, introduced in September 2023, that controls whether your Google-crawled content is used to train Gemini and for AI grounding. Disallowing it keeps your site fully in Search and fully eligible for AI Overviews, and Google states it does not affect rankings. So Google-Extended cannot be the cause of a Search-traffic drop, and blocking it will not fix one.

How do I stop misdiagnosing every dip as AI?

The failure mode is emotional, not analytical. AI anxiety is high, so every wobble gets filed under AI, and the real, fixable cause goes unaddressed. The antidote is a fixed order of checks that you run every time, before you form an opinion.

  1. Confirm the drop is real, not a reporting glitch or a re-bucketing. Check whether Direct rose as organic fell.
  2. Compare impressions and clicks in Search Console. Both down means ranking or technical. Impressions flat, clicks down, means a SERP feature or weak snippets.
  3. Rule out seasonality with a year-on-year comparison and Google Trends.
  4. Match the drop date to Google's published algorithm updates.
  5. Audit the technical basics: Page Indexing, Crawl Stats, robots.txt, noindex tags, Manual Actions, Security Issues.
  6. Only now, if impressions held and clicks fell and no technical fault exists, treat AI Overviews as the working explanation, and verify any crawler claims against the vendor's published data.

Notice that AI is the last step, not the first. That ordering is the whole point. It means you spend your energy on the causes you can actually fix, and you only reach for the AI explanation when the data has earned it. A publisher who runs this sequence will correctly diagnose a core update, a noindex slip, or a seasonal trough long before they waste a fortnight on crawler blocking that changes nothing.

Getting to that clean read is exactly what a good measurement view should do for you. Ramprt gives ad-funded publishers a plain, Plausible-style read of their existing GA4 with no new tracking script, plus an AI tab that separates AI referrals by engine, shows which AI crawlers are actually hitting your pages, and flags the Direct re-bucketing that fakes a drop. If you want to see whether your decline is AI or something more ordinary, try the free live demo at ramprt.io/demo. For more on the measurement side, see our related guides on why AI traffic hides in your GA4 Direct channel and how to verify AI crawlers in your server logs.

The bottom line

AI is a genuine pressure on publisher traffic, and the click-loss data is real. But it is not the only reason traffic falls, and it is rarely the first reason. Google's own list of causes does not mention AI, the impressions-versus-clicks test tells you in minutes whether you have a ranking problem or a click problem, and a year-on-year comparison catches the seasonality that fools everyone. Check the boring causes first. When you do, you will fix more, panic less, and only blame AI when your data actually says so.

Frequently asked questions

Is AI Overviews really cutting my clicks, or is that overblown?

It is real. Pew Research found users clicked a traditional result in only 8 percent of visits with an AI summary versus 15 percent without, and Ahrefs measured a 58 percent lower average click-through rate for the top-ranking page when an AI Overview is present, as of its December 2025 update. The catch is that this affects clicks, not impressions. If your impressions also fell, an AI Overview is not your main problem, a ranking or technical loss is.

How can I tell an AI click-loss apart from a ranking drop?

Compare impressions and clicks in Search Console. If both fell together, you have a ranking or technical problem, so check algorithm-update dates, the Page Indexing report, and Manual Actions. If impressions held steady but clicks fell, a SERP feature such as an AI Overview, or a weak title or snippet, is stealing the click. That single comparison separates the two causes.

Could my traffic drop be a measurement error rather than a real loss?

Yes. Roughly 35 to 70 percent of real AI-referral sessions arrive with no referrer and get filed under Direct in GA4 rather than shown as AI traffic. So some traffic has not fallen, it has just moved buckets. Before concluding you lost visitors, check whether Direct rose as organic fell. GA4 rolled out a native AI Assistant channel on 7 June 2026, but it classifies on the referrer alone, and Perplexity still lands in Referral, so gaps remain.

Should I block Google-Extended to protect my traffic?

Blocking Google-Extended will not affect your traffic either way, because it is not a crawler. It has no separate user-agent or IP, the crawler is still Googlebot. Google-Extended is only a robots.txt token that controls whether your content trains Gemini. Google states that disallowing it keeps you fully in Search and eligible for AI Overviews and does not change rankings, so it cannot cause, or cure, a Search-traffic drop.

How do I know which AI crawlers are actually hitting my site?

Do not trust the user-agent header alone, since anyone can spoof it. OpenAI publishes its authorised IPs and user-agents at openai.com/gptbot.json, searchbot.json, and chatgpt-user.json, so you can match a logged bot's source IP to those lists. Anthropic documents ClaudeBot, Claude-User, and Claude-SearchBot and publishes an IP list, though it advises against IP blocking. Also remember that training crawlers like GPTBot and ClaudeBot never send visitors, they only consume bandwidth.