Skip to content
Climb Google Maps: Google Business Profile optimization.

Technical SEO

Google Tells Sites How to Let Google-Agent Through Their Firewalls. We Tested Its IP Ranges and Its Signatures

Google published a guide on Nov. 17 for allowing Google-Agent, the user agent its AI agents use to browse sites and act for people, past CDNs, firewalls and bot blockers. Its IP ranges don't overlap Googlebot's at all, so a Googlebot allowlist won't cover it. We checked the ranges, verified Google's signing keys, and wrote a script that does both checks.

A red and white barrier arm across a country road lined with trees

Key takeaways

  • On Nov. 17, 2026, Google published a guide to allowing Google-Agent, the user agent its AI agents use to browse websites and take actions such as booking or adding to cart for a person, through CDNs, firewalls and bot-detection services.
  • Google-Agent is a user-triggered fetcher, and Google says those fetchers generally ignore robots.txt, so a site that wants to stop it has to do so at the firewall.
  • In our Nov. 17 check, Google-Agent's 20 published IP ranges shared no addresses with Google's lists for Googlebot, special crawlers or other user-triggered fetchers, so an allowlist built from those files blocks it.
  • Google signs some Google-Agent requests with Web Bot Auth; its key directory held five Ed25519 keys on Nov. 17, and our script verified the signature Google puts on that directory.
  • Google's checklist says an allow rule must cover on-site search, cart, booking and API endpoints and skip CAPTCHAs and per-IP rate limits, not only let the homepage load.

Google-Agent is the user agent Google's AI agents use when they visit a website to do something for a person: read a page, search a catalog, fill a cart or book a slot. On November 17, 2026, Google published a guide to allowing Google-Agent traffic through CDNs, web application firewalls and bot-detection services, so that these visits aren't blocked as scrapers. The guide applies to every site, in every country, that sits behind bot protection.

If you sell, book or take inquiries online, this is the traffic that will turn into customers when people hand tasks to an AI agent. We tested the parts of Google's guide that are easy to get wrong: the IP list, how it relates to the Googlebot lists you may already trust, and the signatures Google is starting to add. Then we wrote a short script that does the whole check.

What is Google-Agent?

Google describes it as "the name used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request." It is listed with Google's user-triggered fetchers, and Search Engine Roundtable reported its arrival in March 2026, with Project Mariner, Google's browsing agent, as the example product. Google's pages don't say which products use it today.

It comes in two forms, according to Google's list:

  • Mobile: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent)
  • Desktop: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent) Chrome/W.X.Y.Z Safari/537.36

W.X.Y.Z stands for the Chrome version, which changes. Match on the word Google-Agent, not the whole string.

Does Google-Agent follow robots.txt?

Not as a rule. Google puts it with its user-triggered fetchers and says that "because the fetch was requested by a user, these fetchers generally ignore robots.txt rules." A Disallow line for Google-Agent is not a reliable way to keep it out. If you want to block it, do it in your firewall or CDN; if you want it in, robots.txt is not what stands in its way.

What does Google's new guide ask site owners to do?

Google's guide gives three checks for a request that claims to be Google-Agent:

  1. The User-Agent header contains Google-Agent.
  2. The IP address is in user-triggered-agents.json.
  3. If the request is signed, the Signature and Signature-Input headers verify for Signature-Agent: g="https://agent.bot.goog", using the Web Bot Auth protocol.

Requests that pass should skip bot challenges. Some CDNs and bot services verify Google-Agent for you; Google says to check your provider's steps first.

The troubleshooting list is the useful part, because it describes how sites block agents without meaning to:

  • Rule order: the allow rule must run before broad data center or cloud network (ASN) blocks, managed bot rules and "block AI agents" category rules.
  • Challenges: skip CAPTCHAs and JavaScript challenges too, including CAPTCHA widgets that appear on search or form submission.
  • The whole journey: an agent searches, filters, opens product pages, adds to cart and books. Your rule has to cover those pages and the XHR, GraphQL or api.* endpoints behind them, not only the homepage.
  • Rate limits: many users' agents share the same IP addresses, so per-IP limits can throttle them.
  • Fresh ranges: keep IP rules synced with the current file.
  • Proxies: load balancers must pass on the real client IP, the User-Agent and the signature headers.

Our test: Google-Agent's IP addresses are not Googlebot's

Many sites already let Googlebot through by checking Google's published IP lists. We wanted to know whether those rules cover Google-Agent. On November 17 we downloaded all five of Google's IP range files and compared them.

  • user-triggered-agents.json (Google-Agent), generated November 16: 20 ranges, 12 IPv4 and 8 IPv6, about 70,000 IPv4 addresses in all, in 136.121.x.x, 136.122.x.x, 74.125.232.x and 2001:4860:c::.
  • Overlap with the other four files: none. Not one Google-Agent range overlaps common-crawlers.json (317 ranges, Googlebot), special-crawlers.json (272, AdsBot and others), user-triggered-fetchers.json (1,088) or user-triggered-fetchers-google.json (504).

So a firewall rule that trusts Google's crawler lists will challenge or block Google-Agent. There is a second trap: the "automatic" section of Google's verification page, the list people copy into scripts, names the other four files but not the Google-Agent one. It appears only in the table higher up. If you built your allowlist from that list, add the fifth file.

We also looked up five addresses from the Google-Agent ranges in reverse DNS. All five resolved to google-proxy- names under google.com (for example google-proxy-74-125-232-1.google.com), the pattern Google documents for its own user-triggered fetchers. Reverse DNS works as a manual check, but the IP file is faster for a firewall.

Our test: the signatures are real and checkable

Google says it is "experimenting" with Web Bot Auth, a draft standard in which an agent signs each request with HTTP Message Signatures (RFC 9421) and publishes its public keys. Google signs only some Google-Agent requests, so the IP check stays necessary.

The key directory at https://agent.bot.goog/.well-known/http-message-signatures-directory returned five Ed25519 public keys on November 17, with a one-hour cache time. Google signs the directory response itself, so we used it as a real test case: our script rebuilt the signed message, checked it against the key it named, and it verified. The same signature failed when we changed the host name. We also signed test requests with our own key to check the request parts (host, method, path and the Signature-Agent header), and the script rejected a changed path and an expired signature.

A script that checks Google-Agent requests

This is Node.js 20 or later with no packages. It loads Google's current ranges and keys, checks the user agent, tries the signature when there is one, and falls back to the IP list, as Google's page recommends.

verify-google-agent.mjs
// verify-google-agent.mjs: is this request really Google-Agent?
// Usage: node verify-google-agent.mjs <ip> "<user-agent>"
// Node 20+. No dependencies.
import { createPublicKey, verify } from 'node:crypto';
import net from 'node:net';
 
const RANGES_URL = 'https://developers.google.com/static/crawling/ipranges/user-triggered-agents.json';
const KEYS_URL = 'https://agent.bot.goog/.well-known/http-message-signatures-directory';
 
// 1. IP check against Google's published Google-Agent ranges
export async function loadRanges() {
  const res = await fetch(RANGES_URL);
  const data = await res.json();
  const list = new net.BlockList();
  for (const p of data.prefixes) {
    const cidr = p.ipv4Prefix || p.ipv6Prefix;
    const [addr, bits] = cidr.split('/');
    list.addSubnet(addr, Number(bits), p.ipv4Prefix ? 'ipv4' : 'ipv6');
  }
  return { list, created: data.creationTime, count: data.prefixes.length };
}
 
export function ipIsGoogleAgent(ranges, ip) {
  const type = net.isIPv6(ip) ? 'ipv6' : 'ipv4';
  return ranges.list.check(ip, type);
}
 
// 2. Signature check (Web Bot Auth, RFC 9421), for the requests Google signs
export async function loadKeys() {
  const res = await fetch(KEYS_URL);
  const { keys } = await res.json();
  return new Map(keys.map(k => [k.kid, createPublicKey({ key: k, format: 'jwk' })]));
}
 
function componentValue(name, req) {
  const url = new URL(req.url);
  switch (name) {
    case '@authority': return url.host.toLowerCase();
    case '@method': return req.method.toUpperCase();
    case '@path': return url.pathname;
    case '@query': return url.search || '?';
    case '@scheme': return url.protocol.replace(':', '');
    case '@target-uri': return url.href;
    default: {
      const v = req.headers[name.toLowerCase()];
      if (v === undefined) throw new Error(`missing header ${name}`);
      return String(v).trim();
    }
  }
}
 
// req = { method, url, headers: { lowercase-name: value } }
export function verifySignature(req, keys, label = 'g') {
  const input = req.headers['signature-input'] || '';
  const sig = req.headers['signature'] || '';
  const params = input.match(new RegExp(`(?:^|,\\s*)${label}=(\\(.*?\\)[^,]*)`))?.[1];
  const sigB64 = sig.match(new RegExp(`(?:^|,\\s*)${label}=:([^:]+):`))?.[1];
  if (!params || !sigB64) return { ok: false, reason: 'not signed' };
  const keyid = params.match(/keyid="([^"]+)"/)?.[1];
  const expires = Number(params.match(/expires=(\d+)/)?.[1] || 0);
  if (expires && expires < Date.now() / 1000) return { ok: false, reason: 'signature expired' };
  const key = keys.get(keyid);
  if (!key) return { ok: false, reason: `unknown key ${keyid}` };
  const items = [...params.match(/^\((.*?)\)/)[1].matchAll(/"([^"]+)"((?:;[a-z]+)*)/g)];
  const lines = items.map(([, name, flags]) => `"${name}"${flags}: ${componentValue(name, req)}`);
  lines.push(`"@signature-params": ${params}`);
  const ok = verify(null, Buffer.from(lines.join('\n')), key, Buffer.from(sigB64, 'base64'));
  return { ok, reason: ok ? `signed with key ${keyid}` : 'bad signature' };
}
 
// 3. The whole decision
export function isGoogleAgent(ranges, keys, req, ip) {
  if (!/Google-Agent/.test(req.headers['user-agent'] || '')) return { ok: false, reason: 'user agent is not Google-Agent' };
  if (/agent\.bot\.goog/.test(req.headers['signature-agent'] || '')) {
    const s = verifySignature(req, keys);
    if (s.ok) return s;
  }
  return ipIsGoogleAgent(ranges, ip)
    ? { ok: true, reason: 'IP is in user-triggered-agents.json' }
    : { ok: false, reason: 'IP is not in Google-Agent ranges' };
}
 
if (import.meta.url === `file://${process.argv[1]}`) {
  const [ip, ua] = process.argv.slice(2);
  const ranges = await loadRanges();
  const keys = await loadKeys();
  const r = isGoogleAgent(ranges, keys, { method: 'GET', url: 'https://example.com/', headers: { 'user-agent': ua } }, ip);
  console.log(`${ip}: ${r.ok ? 'VERIFIED' : 'NOT VERIFIED'} (${r.reason}); ${ranges.count} ranges from ${ranges.created}`);
}

Run it on an IP and user agent from your server logs:

node verify-google-agent.mjs 74.125.232.5 "Mozilla/5.0 (compatible; Google-Agent)"

In our test run on November 17, it verified addresses from all three Google-Agent blocks (IPv4 and IPv6), rejected the Googlebot address that Google's own docs use as an example (66.249.66.1) when it claimed to be Google-Agent, rejected a reserved test address, and rejected a Google-Agent IP that sent an ordinary browser user agent. In a server or edge worker, import isGoogleAgent, load the ranges and keys once, and refresh them every hour or so. The script handles the signature parts Google's directory and our tests use; Cloudflare's web-bot-auth examples, which Google points to, cover the full standard.

Should you let Google-Agent in?

Our view: yes, for almost every business site. Google-Agent visits because a person asked an agent to do something on your site, usually something you want done, like comparing your prices or booking with you. Blocking it is like locking out customers who send an assistant. The cases for keeping it out are narrower: sites that sell access to their content, or forms that must only ever be filled in by a human.

What we chose for this site: Cloudflare's AI bot blocking is off for thesearchpost.com, so Google-Agent reaches every page like any reader.

What to check this week

  • Look in your logs for Google-Agent and the status codes it got. 403s, challenge pages or 429s mean your bot protection is stopping it.
  • Find your provider's setting for verified bots and agents, then check where it sits against your custom rules and any AI category block.
  • Add user-triggered-agents.json to any allowlist built from Google's other IP files, and refresh it automatically.
  • Walk the journey: search, filters, product or service pages, cart, booking and the API calls behind them should all be covered, not only page loads.
  • Raise or exempt per-IP rate limits for verified Google-Agent addresses.

What we don't know yet

  • Which Google products send Google-Agent today, and how much traffic it brings. Google's pages don't say.
  • What share of requests Google signs, and when signing will stop being an experiment.
  • How each CDN treats Google-Agent by default. Google's guide leaves that to the providers.

For the crawler side of the same problem, see our post on moving to a CDN without blocking Googlebot.

Sources

Photo of Kiran Dasgupta

About the author

Kiran Dasgupta, Editor

Kiran runs Kihan Marketing, a web design, SEO and ads agency, and manages dozens of websites across real estate, home services and healthcare. He writes about SEO, AI search and paid ads news, and what it means for the people who run websites.