Cloudflare Worker
Deploy a Cloudflare Worker that intercepts AI bot visits and sends them to Robot Speed. Works with Cloudflare's free plan.
Prerequisites
- ✓A Cloudflare account (free)
- ✓Your domain on Cloudflare with proxy enabled (orange cloud)
- ✓Your Robot Speed API key (Settings → API in your dashboard)
1
Create a Worker
- Go to your Cloudflare Dashboard
- Click Workers & Pages in the sidebar
- Click Create Worker
- Give it a name (e.g.
ai-bot-tracker)
2
Paste the code
Replace the default code with the code below, then click Deploy.
export default {
async fetch(request, env, ctx) {
// Forward the request to your origin — transparent proxy
const response = await fetch(request);
// Detect AI bots from user-agent
const ua = request.headers.get("user-agent") || "";
const bots = [
["ChatGPT-User", "openai"],
["OAI-SearchBot", "openai"],
["PerplexityBot", "perplexity"],
["Perplexity-User", "perplexity"],
["Claude-User", "anthropic"],
["Claude-SearchBot", "anthropic"],
["Google-Extended", "google"],
["Google-Agent", "google"],
["MistralAI-User", "mistral"],
["xAI-Grok", "xai"],
["GrokBot", "xai"],
["Grok-DeepSearch", "xai"],
["DeepSeekBot", "deepseek"],
["cohere-ai", "cohere"],
["Applebot-Extended", "apple"],
];
const bot = bots.find(([sig]) => ua.includes(sig));
if (bot) {
const url = new URL(request.url);
const path = url.pathname;
// Skip noise: feeds, tags, static files
if (!/\/(feed|tag|category)\b/i.test(path) && !/\.\w{2,4}$/.test(path)) {
// Fire-and-forget — never blocks the response
ctx.waitUntil(
fetch(env.ANALYTICS_ENDPOINT, {
method: "POST",
headers: {
"Content-Type": "application/json",
"X-API-Key": env.ANALYTICS_KEY,
},
body: JSON.stringify({
bot_name: bot[0],
bot_provider: bot[1],
url_path: path,
user_agent: ua.slice(0, 500),
host: request.headers.get("host")?.replace(/^www\\./, ""),
source: "cloudflare_worker",
}),
}).catch(() => {})
);
}
}
// Return the original response — zero latency impact
return response;
},
};3
Configure variables
Go to Worker Settings → Variables, and add:
ANALYTICS_ENDPOINThttps://www.robot-speed.com/api/ai-bot-visitsANALYTICS_KEYYour Robot Speed API key — find it in your dashboard: Settings → API
4
Add the route
- In the Worker, go to Triggers → Add Route
- Route:
yourdomain.com/* - Select your zone (domain)
5
Verify DNS proxy
In DNS → Records, make sure your domain has the orange cloud (Proxied) enabled. If the cloud is grey (DNS only), the Worker won't see the traffic.
Verification
Test with this command:
curl -H "User-Agent: Mozilla/5.0 (compatible; ChatGPT-User/1.0)" https://yourdomain.com/A few seconds later, you should see the mention in your Robot Speed dashboard.