Bots aren’t the enemy. It’s not knowing which ones to trust 

Around one in five requests bunny.net/edge-security-report-2026/ that hit a website today don’t come from humans. HTTP libraries, AI crawlers, API integrations, and AI agents now make up a big share of internet traffic, and that share is growing fast. That’s pushing security teams toward a different question than the one they’re used to asking. For years, the job was figuring out whether a visitor was a bot or a person, but now the harder call is whether an automated request has any business being there in the first place. 

 

That distinction matters because a lot of today’s automation is entirely legitimate. Modern infrastructure depends on APIs, payment gateways, monitoring systems, CI/CD pipelines, partner integrations, and, increasingly, AI agents acting on a user’s behalf, and blocking automated requests outright would disrupt many of the services businesses depend on every day.  

 

The challenge is separating trusted traffic from the abusive kind, often when both look remarkably similar on the surface. 

 

Identity isn’t the signal it used to be  

For years, user-agent strings and IP reputation provided reasonable indicators of intent. But that approach is becoming unreliable. 

 

Malicious bots now rotate through large proxy networks, distribute requests across thousands of IP addresses, and mimic legitimate browser fingerprints closely enough to slip past filters made to catch obvious bot behaviour. 

 

That means identity checks alone can’t carry the weight. What still gives bots away is how they act. Real users tend to move through a site in fairly predictable ways, while bots often jump straight to deep-linked pages, skipping normal navigation steps or generating requests at volumes and frequencies no human could realistically match. 

 

In just minutes, a credential-stuffing attack can generate thousands of login attempts, while attackers are deploying large-scale, distributed botnets to sustain multi-day assaults that put extreme resource demands on your infrastructure. 

 

While modern bots are becoming more and more sophisticated at mimicking human activity, genuine human traffic still behaves in a hundred small ways that bots can’t quite fake. 

 

Distinguishing legitimate automation from abuse 

For developers and security teams looking to strengthen their defences, several practical changes can make a huge difference:  

• Move beyond per-server rate limiting. When individual servers or points of presence make decisions independently, distributed botnets can spread requests across thousands of sources to stay under the radar while still exhausting your backend systems. Rate-limits, therefore, need to run against synchronised global counters rather than independent ones, so that you can identify coordinated abuse.  
• Classify traffic using multiple signals. User-agent strings are still useful, but they should only be one part of the decision-making process. Combining user-agent data with request patterns, header completeness, and known operator IP ranges provides a much more accurate understanding of what the client is actually doing.  
• Create dedicated policies for AI crawlers. AI crawler traffic has grown to roughly the same volume as traditional search-engine bots, which means we can’t treat all bots equally anymore. Decide, per crawler, if it should be allowed, rate-limited, or blocked, and revisit that list regularly as new AI services continue to emerge.  
• Challenge when uncertain, block when necessary. A challenge asks the client to demonstrate expected browser behaviour through automated verification before stronger enforcement is applied. Legitimate clients typically pass without issue. The upside is you can act on a suspicious signal without shutting out a genuine request. And as much of your infrastructure runs on machine-to-machine traffic these days, that’s not a risk worth shrugging off.  
• Watch for RegEx-based WAF exposure. Some web application firewall engines still rely on RegEx matching techniques that can be vulnerable to catastrophic backtracking. In these scenarios, specifically crafted requests can force your security layer to consume excessive resources, turning the protection layer itself into a denial-of-service target. Modern engines designed around linear-time matching eliminate this class of attack and ensure performance remains predictable. 

The shift developers need to plan for 

None of this means automation is the enemy, of course. In fact, automation is becoming a fundamental part of how the internet operates. What has changed is that the traditional binary distinction between human and bot is no longer particularly useful, and the systems built around it need to be replaced with ones that evaluate behaviour and intent instead. 

 

These days, the real test isn’t whether a request came from a human or a machine. It’s whether it’s behaving like it belongs there. 

 

---

 

Dejan Grofelnik Pelzel, Founder and CEO of bunny.net 

 

Main image courtesy of iStockPhoto.com and tadamichi