My Hatred of AI Scrapers Knows No Bounds

It’s AI bots, scraping your data.
And it’s not the mainstream ones. ChatGPT, Claude and Google tell you who they are in their user agent string. These ones are some shady models being used for who the heck knows what. And it’s just… frustrating. It ends up resembling a Distributed Denial of Service attack, and if you’ve noticed instability on my website recently, that’s why.
It’s hard enough figuring out what to do about the “honest” AI companies. So many people are relying on stuff like ChatGPT for search, part of me doesn’t want to hide my stuff in the hopes there will be discoverability, but I also don’t want the models training on my data. I find myself favoring telling those systems to not index my site.
But it’s frustrating.
Honestly I don’t know what the solution is. I end up spending time playing IP block whack-a-mole when they hit my site, and it burns through the little free time I have. I spent the time I meant to spend writing this weekend dealing with these crawlers and scrapers instead. That graph attached to this post is my bandwidth over the last few days, and you can see when the bots hit. It’s… it’s not great.
And I hate it.







