engineering
2 posts · all posts
How to verify a crawler is really Googlebot, GPTBot, or Meta
Up to 16.7 percent of ChatGPT-User traffic is faked. A User-Agent is a claim, not proof. Here is how to verify an AI crawler's real identity, with code.
How to detect AI agents server-side from HTTP headers
AI bots are 4.2 percent of HTML requests and climbing, yet client-side analytics never sees them. Here is how to detect AI agents server-side, with real code.