Cloudflare Worker

Deploy a Cloudflare Worker that intercepts AI bot visits and sends them to Robot Speed. Works with Cloudflare's free plan.

Prerequisites

  • A Cloudflare account (free)
  • Your domain on Cloudflare with proxy enabled (orange cloud)
  • Your Robot Speed API key (Settings → API in your dashboard)
1

Create a Worker

  1. Go to your Cloudflare Dashboard
  2. Click Workers & Pages in the sidebar
  3. Click Create Worker
  4. Give it a name (e.g. ai-bot-tracker)
2

Paste the code

Replace the default code with the code below, then click Deploy.

export default {
  async fetch(request, env, ctx) {
    // Forward the request to your origin — transparent proxy
    const response = await fetch(request);

    // Detect AI bots from user-agent
    const ua = request.headers.get("user-agent") || "";
    const bots = [
      ["ChatGPT-User", "openai"],
      ["OAI-SearchBot", "openai"],
      ["PerplexityBot", "perplexity"],
      ["Perplexity-User", "perplexity"],
      ["Claude-User", "anthropic"],
      ["Claude-SearchBot", "anthropic"],
      ["Google-Extended", "google"],
      ["Google-Agent", "google"],
      ["MistralAI-User", "mistral"],
      ["xAI-Grok", "xai"],
      ["GrokBot", "xai"],
      ["Grok-DeepSearch", "xai"],
      ["DeepSeekBot", "deepseek"],
      ["cohere-ai", "cohere"],
      ["Applebot-Extended", "apple"],
    ];

    const bot = bots.find(([sig]) => ua.includes(sig));

    if (bot) {
      const url = new URL(request.url);
      const path = url.pathname;

      // Skip noise: feeds, tags, static files
      if (!/\/(feed|tag|category)\b/i.test(path) && !/\.\w{2,4}$/.test(path)) {
        // Fire-and-forget — never blocks the response
        ctx.waitUntil(
          fetch(env.ANALYTICS_ENDPOINT, {
            method: "POST",
            headers: {
              "Content-Type": "application/json",
              "X-API-Key": env.ANALYTICS_KEY,
            },
            body: JSON.stringify({
              bot_name: bot[0],
              bot_provider: bot[1],
              url_path: path,
              user_agent: ua.slice(0, 500),
              host: request.headers.get("host")?.replace(/^www\\./, ""),
              source: "cloudflare_worker",
            }),
          }).catch(() => {})
        );
      }
    }

    // Return the original response — zero latency impact
    return response;
  },
};
3

Configure variables

Go to Worker Settings → Variables, and add:

ANALYTICS_ENDPOINT
https://www.robot-speed.com/api/ai-bot-visits
ANALYTICS_KEY

Your Robot Speed API key — find it in your dashboard: Settings → API

4

Add the route

  1. In the Worker, go to TriggersAdd Route
  2. Route: yourdomain.com/*
  3. Select your zone (domain)
5

Verify DNS proxy

In DNS → Records, make sure your domain has the orange cloud (Proxied) enabled. If the cloud is grey (DNS only), the Worker won't see the traffic.

Verification

Test with this command:

curl -H "User-Agent: Mozilla/5.0 (compatible; ChatGPT-User/1.0)" https://yourdomain.com/

A few seconds later, you should see the mention in your Robot Speed dashboard.