Benchmarking Managed Scraping APIs for Anti-Bot Targets

Developers shopping for a managed scraping API face dozens of numbers, most of which cannot be compared to each other or trusted to predict what happens in production. By early 2026, the DEV Community comparison guide had counted dozens of providers, and each one leads with a different headline metric. One quotes uptime. One quotes speed. One quotes a success rate that sounds impressive until you ask what it actually measured.

That last question is where the whole category falls apart. No shared definition of "success" exists across providers. Some count a successful HTTP status code as a win. Others check that the response body actually contains the page content a human visitor would see. ScrapeUnblocker's benchmark writeup points out the problem with the first approach bluntly: anti-bot systems routinely return a challenge page, a CAPTCHA, or an "access denied" screen wrapped in the exact same success status code a real page would carry. Status codes alone cannot distinguish a blocked request from a successful one. A provider can publish a high success rate that is, in plain terms, fiction.

This is a call to read benchmarks the way a contract lawyer reads fine print, because the cost of skipping that step is real and specific. A provider that looks cheap on paper fails exactly on the handful of targets a business actually needs, and the retry logic required to compensate quietly multiplies the real bill several times over. The rest of this piece is built to stop that from happening, by walking through the three dimensions, target difficulty, measurement methodology, and cost, that a headline number never shows.

The anti-bot landscape in 2026

Benchmarks age fast because the thing they are measuring does not sit still. Anti-bot protection in 2026 is a moving collection of systems, hardening faster than most published comparisons can keep pace with.

The scale of the underlying shift explains why. Cloudflare's CEO shared Radar data showing that bots now account for the majority of HTTP requests to web content, a crossover the company had originally expected in 2027. It arrived early, driven largely by AI agents that browse at volumes no human ever would. That single fact cuts both ways for anyone running a scraping operation: it is growing the market for managed scraping APIs at the same time it is giving every protected website a harder incentive to lock the door.

The vendors on the other side of that door are not one thing either. Cloudflare Bot Management, DataDome, Akamai Bot Manager, PerimeterX (now folded into HUMAN Security), Kasada, Incapsula, AWS WAF, and F5/Shape each run detection models built on different logic, and Scrapfly's guide to all eight is clear that no single bypass technique clears all of them. Beating Cloudflare teaches a scraper nothing reliable about beating Kasada.

Defenders also talk to each other now, which scrapers rarely get the courtesy of. A datacenter IP flagged as high-confidence on Cloudflare tends to show up blocked on DataDome within hours and on Akamai within a day. That intelligence-sharing has quietly flipped the economics of evasion against the scraper. Residential proxies, long the default workaround, saw block rates rise meaningfully on the same target lists between 2024 and 2026, while mobile 4G and 5G proxies have held up better simply because carrier IP ranges are much harder to denylist at the scale defenders would need.

Layering makes the problem compound rather than add. A site can run Cloudflare as its CDN while DataDome handles behavioral detection underneath it, so clearing the first gate does nothing but introduce a scraper to the second one. Understanding this stacked, shifting landscape matters because it sets the baseline difficulty any benchmark number has to be judged against. A provider's success rate from three months ago is a photograph of a landscape that has since moved.

How major anti-bot vendors differ in bypass difficulty

Difficulty is several different locks, each built by a different locksmith, and a provider's overall success rate is an average across all of them, one that can hide a complete failure on the lock that actually matters to a given business.

Cloudflare is the one a scraper is most likely to run into. Scrapfly's vendor analysis puts it behind roughly three-quarters of all anti-bot deployments in its dataset, and its stack layers IP reputation, TLS fingerprinting through JA3 and JA4, HTTP/2 fingerprinting, browser fingerprinting through Canvas and WebGL, behavioral analysis, and JavaScript challenges like Managed Challenge and Turnstile, all of which a scraper needs to clear at once. JA4 fingerprinting alone flags a standard HTTP library before a single header or cookie even gets inspected. That is why so many naive scraping attempts die before they produce a single byte of content.

DataDome plays a different kind of game. Scrapfly's guide describes a platform running roughly 85,000 machine learning models, one per client, each trained on that specific site's traffic, processing trillions of signals a day with response times under two milliseconds. That per-client training is the detail that matters most for anyone benchmarking providers: a bypass that works on one DataDome-protected site does not transfer to the next one, because the model guarding it was never the same model to begin with. DataDome added intent analysis in 2025, scoring not just whether traffic looks automated but why the visit is happening, with separate categorization for LLM crawlers specifically. AlterLab's Anti-Bot Difficulty Index, built from production scrape encounters rather than a lab setup, ranks DataDome as the hardest protection type in its dataset, and not by a small margin.

Kasada sits a tier above even that. Its core script, ips.js, runs somewhere between 449 and 655 kilobytes depending on site and version, and it is not simple obfuscation, it builds a complete virtual machine inside the browser. A researcher who reverse-engineered that VM in April 2026, examining its use on Nike, Kick, and Twitch, found a large template array populated through 427 separate probes run in 20 batches, each executing in sequence. Challenge rotation and anti-deobfuscation protections mean any DIY solution breaks within days of an update, and Scrapfly's guide names Kasada as the one vendor where building your own bypass simply is not viable in production. Even managed providers show their widest spread of results on Kasada targets: ScrapeUnblocker's benchmark found hyatt.com, which runs Kasada, returning successfully at more than double the field average across providers. Double the field average on one target is not a rounding error, it is a sign that "success rate" means something different for every vendor a provider is measured against.

Akamai, PerimeterX, Incapsula, AWS WAF, and F5/Shape round out the list, and each needs its own identification approach before any bypass logic gets written. Scrapfly's guide supplies per-vendor signatures, specific cookie names, JavaScript file signatures, response headers, so a developer can confirm which system they are actually looking at rather than guessing. The throughline across all eight vendors is the same: heterogeneity, not difficulty, is the reason a single success-rate number cannot carry the weight vendors ask it to carry.

Target difficulty tiers and unequal success-rate averages

A provider's overall success rate is averaged across sites that have almost nothing in common with each other in terms of difficulty, and any provider that tests mostly against easy sites will post a number that tells a buyer nothing about the sites that actually matter to them.

Most of the internet is still easy. Blogs, small e-commerce storefronts, and public datasets answer a plain HTTP request without complaint, and the DEV Community guide is right to point out that nobody needs a managed API to pull a handful of static pages. The difficulty concentrates in a small set of high-value targets: large retailers, travel and hospitality platforms, review sites, and marketplaces, all sitting behind commercial-grade anti-bot infrastructure. These targets are where provider choice matters most, carrying real weight in whether a scraping pipeline works.

Proxyway's 2025 field data, cited in ScrapeUnblocker's benchmark as a difficulty baseline, shows how steep that drop is. Averaged across providers, shein.com, protected by in-house tooling, returned a successful result only around a fifth of the time. g2.com, running DataDome, came in around a third. hyatt.com, running Kasada, sat under half. lowes.com, running Akamai, landed just over half. Those are not marginal numbers on an edge case, they describe the tier where most providers struggle, full stop. AlterLab built its infrastructure around exactly this unevenness: five escalating tiers, running from plain curl requests at the cheapest end up to CAPTCHA solving paired with full browser automation at the most expensive, auto-escalating based on whatever protection is detected, so the price paid per request tracks the actual difficulty of that request instead of charging one flat rate for everything.

The practical consequence for anyone reading a vendor's marketing page is straightforward. A high overall success rate measured against a target list stacked with easy sites says nothing about what that same provider will do against shein.com or g2.com. Before trusting any headline number, ask what share of the benchmark's targets actually ran Kasada, DataDome, or serious in-house protection; that share applies the number to the job at hand, not the aggregate figure.

What measurement methodology reveals about benchmark validity

Two benchmarks can run the exact same provider against the exact same list of sites and come back with different answers, purely because of how each one was built. That makes the benchmark's design itself something a reader has to check before trusting the output at all.

The most common failure is the one already mentioned: treating a successful HTTP status code as proof of success. ScrapeUnblocker's writeup is direct about why that fails, anti-bot systems hand back challenge pages wrapped in a normal success status, so a status-only check records a block as a win. The fix is content verification, actually reading the response body to confirm it contains real page content rather than a CAPTCHA or an access-denied message. ScrapeUnblocker applied that check across all 6,000 calls per target in its own benchmark run, which is the kind of baseline any serious test needs to clear.

Concurrency is another variable that changes the answer. Proxyway's methodology tested providers at both 2 requests per second and 10 requests per second because providers that look strong at low concurrency often collapse under load. A single concurrency figure only tells half the story. Sample size matters just as much, since anti-bot systems behave probabilistically and a small number of requests per target produces noisy, unreliable results. Proxyway's 2025 report ran a large number of unique URLs per target, and ScrapeUnblocker matched that volume in its own replication to keep the comparison fair.

Recency may be the most underrated variable of all. Scrapeway's Cloudflare benchmark refreshes twice a month, carries no affiliate links, and takes no sponsors, a design built around the fact that anti-bot systems update faster than any static annual report can track. A benchmark published once a year is describing a landscape that, as the earlier section on vendors made clear, can shift meaningfully within weeks.

None of this means vendor-supplied benchmarks are worthless. It means they are usually measured under conditions set up to flatter the provider, favorable targets, controlled traffic, and independent tests against those same targets tend to land on lower numbers. Before acting on any benchmark, run through a short checklist: What counted as success? Which sites were tested, and what protection tier did each one carry? How many requests hit each target? At what concurrency? When was the test run? Who ran it, and what did they stand to gain from the result?

Cost-per-successful-request and the real economics behind headline rates

The number on a pricing page is the price per request sent. It is almost never the price per request that actually comes back with usable data, and that gap grows fastest on exactly the hard targets discussed above.

The math is simple once it is laid out. If a provider succeeds on a hard target only a fraction of the time, hitting a usable volume of successful scrapes means sending several times that number of total requests, and every one of those failed attempts still costs money. The real operational cost is the sticker price divided by the actual success rate on that specific target.

JavaScript rendering pricing makes the gap worse before retries even enter the picture. The DEV Community guide flags one major provider charging ten credits per request for JavaScript rendering against one credit for a standard request, a ten-times jump in effective price that appears only in actual usage, not in the advertised per-request rate. A buyer comparing two providers on their standard pricing tables can miss that one of them charges an order of magnitude more the moment a page needs a browser to render it.

The billing model itself changes who absorbs that risk. A flat rate per request charges the same whether the request succeeds or fails, so every blocked attempt is paid for twice, once in frustration and once in cash. A pay-per-success model flips that arrangement, and Scrapfly is noted for exactly this policy, ranking first in the Scrapeway benchmark, with AlterLab's pay-per-request auto-escalation model approximating the same effect by tying cost to the tier of difficulty actually encountered. Under that structure, the provider eats the cost of a failed attempt rather than the buyer, and the escalating-tier design means a request only pays for the heavier tooling, like CAPTCHA solving or full browser automation, when the target actually demands it. Reading a pricing page without asking which of these two models is in effect is the last way a headline number quietly misleads a buyer who thought they had already done the homework.