Pay-per-crawl and x402: what we could verify, and what we got wrong

Who reported it, and when we saw it

Key facts

• The cloudflare.pay availability endpoint takes the parameter tag, not handle. Passing handle= returns HTTP 400 with code MISSING_TAG. Checked 2026-08-09, when tag=dablock and tag=dabyte both still returned available true, three days after we wrote the internal note telling everyone else to claim their names.
• Correction to an earlier draft of this article: ten thousand crawls a day at a publisher-configured one cent is 100 dollars per day, roughly 3,000 dollars per month. Our earlier figure of 100 dollars per month was wrong by a factor of 30.
• Retraction: we cited 404s at perplexity.ai/publishers and gist.ai/publishers as missing entry points. We had guessed both URLs. The live pages return HTTP 200 on 2026-08-09: perplexity.ai/hub/blog/introducing-the-perplexity-publishers-program at 343,088 bytes, and gist.ai/products/gist-answers at 133,632 bytes.
• That Perplexity 404 carries no information. A control path we invented on the spot returned 12,647 bytes against 12,609 for /publishers, with the same page title and the same challenge-platform marker in both bodies. Five consecutive requests with one identical desktop user agent returned 404, 403, 404, 404, 403 on 2026-08-09.
• Cloudflare's Monetization Gateway post, datePublished 2026-07-01 and dateModified 2026-07-16, closes with: The Monetization Gateway waitlist is open now for Cloudflare customers. A case-insensitive search of the page HTML for commission, revenue share and take rate returns zero matches.
• Cloudflare Wallets, datePublished 2026-08-04: Starting today, you can claim a Cloudflare Wallet handle for your account, which will provide a unique username to help you better connect with merchants. The next sentence reads: Soon, you will be able to set up and use your Cloudflare Wallet to pay for APIs and content.
• Coinbase CDP x402 facilitator, checked 2026-08-09: first 1,000 onchain transactions per month free, 0.001 dollars per transaction above that, verification always free. Networks listed: Base, Polygon, Arbitrum, World and Solana.

What answered like a queue, and what answered like a product

Three Cloudflare surfaces, checked on 2026-08-09, and one of them is a product. The signup page at https://www.cloudflare.com/paypercrawl-signup/ answers HTTP 200, but what it serves is an application form headlined Announcing Cloudflare pay per crawl beta, whose instruction reads: Apply for private beta access by filling out this form. The page's own metadata title ends in Private Beta. The Monetization Gateway post, datePublished 2026-07-01 and dateModified 2026-07-16 in its structured data, closes with: The Monetization Gateway waitlist is open now for Cloudflare customers. An application form and a waitlist are queues, not switches. This is a status rather than an accusation, and statuses expire, which is why every line here carries a date. It is also a much narrower claim than the one we originally set out to make. Cloudflare's rails not being self-serve tells you nothing about whether a publisher can be paid at all. Confusing those two questions is the subject of section three.

The parameter is tag, not handle

The exception is name reservation. Cloudflare Wallets, announced 2026-08-04, ships with a working availability endpoint, and it is programmatic. A GET to https://cloudflare.pay/api/check?tag=NAME returns either available true with a normalized name, or available false with the code TAG_TAKEN. Use handle= instead of tag= and you get HTTP 400 with the error Missing 'tag' query parameter and the code MISSING_TAG. We lost time to that once; the field name is the whole trick. Read the response before drawing conclusions from it. It carries three fields and no owner field, so TAG_TAKEN tells you a name is gone, not who took it. On 2026-08-09 the tags 402, crawl, citation, paywall and data all returned TAG_TAKEN. We are not claiming when they were taken, only that they are. What a handle does today is bounded by Cloudflare's own wording: it provides a unique username to help you better connect with merchants, while storing stablecoins, creating Virtual Wallets for agents and setting spend guardrails are all written in the future tense in the same post. A reserved handle is a reserved name. Our own failure belongs here. We wrote a twelve-name list and treated the list as the work. On 2026-08-09 both tag=dablock and tag=dabyte still return available true. You can run the check against this site's own name and watch it come back unclaimed. Writing the checklist is not doing the item.

We read two 404s as absence, and both programs were live

An earlier draft of this article opened by saying an independent publisher's baseline is zero revenue, and rested that on four checks. Two of them were worthless. We requested https://www.perplexity.ai/publishers and https://gist.ai/publishers, got 404 on both, and wrote them up as missing entry points. We had guessed both URLs; neither is linked from anywhere we can point to. Both programs exist. Perplexity's Publishers' Program is documented at https://www.perplexity.ai/hub/blog/introducing-the-perplexity-publishers-program, which returns 200 and 343,088 bytes of origin HTML titled Introducing the Perplexity Publishers' Program, and which states that when Perplexity earns revenue from an interaction where a publisher's content is referenced, that publisher will also earn a share. ProRata's publisher product sits at https://gist.ai/products/gist-answers, 200 and 133,632 bytes, whose raw HTML reads that Gist Answers is free to implement and generates revenue through three distinct paths, and that Gist Ads revenue is shared 50/50 between ProRata and the publishers. Signup there runs through a partnership agreement or an umbrella agreement rather than a self-serve button, and the page says most partners are live within days. The second failure is worse than the first, because the instrument was broken rather than merely aimed at the wrong target. The 404 from www.perplexity.ai is served by bot management, not by Perplexity's router. A path we invented on the spot, ending zzz-this-does-not-exist-12345, returns a body of 12,647 bytes against 12,609 for /publishers, with the same title and the same challenge-platform marker in both. Five consecutive requests with one identical desktop user agent returned 404, 403, 404, 404, 403, and the default curl agent returned 403 three times running, so the earlier draft's claim of a user-agent correlation was noise we had mistaken for signal. The rule now written into our methodology: a status code is evidence only once the body is confirmed to have come from the origin, which means a control request to a path known not to exist, a control to a path known to exist, and a check of the body for challenge markers. We already held an internal incident record describing this same false positive against a host behind the same CDN, and we ran the check anyway.

Cloudflare narrowed its own crawl-priced model within twelve months

Pay per crawl launched 2025-07-01 by the datePublished in its own structured data, and publishers set the rate themselves; the post describes a configured, domain-wide price. On 2026-07-01, twelve months to the day, Cloudflare published Making AI search smarter and reframed the model: crawling is a crude measure of value. A single page might be crawled once and then cited in thousands of answers, or crawled over and over and never used at all. The same post states that more than 50% of crawl traffic from good bots goes to re-fetching pages that haven't changed. Our arithmetic correction belongs here, because the earlier draft was wrong by a factor of thirty. At a publisher-configured one cent, ten thousand bot requests a day is one hundred dollars a day, which is about three thousand dollars a month, not the one hundred dollars a month we first published. The error cuts against us in the direction that matters: three thousand a month is enough to build a plan on, which makes the vendor re-scoping its own market more consequential, not less. It also sharpens the objection rather than blunting it. If more than half of good-bot crawl traffic is re-fetching unchanged pages, on Cloudflare's own figure, a per-crawl meter bills most heavily for the requests carrying the least new information, which is close to the reasoning Cloudflare itself gave for moving on. The successor is Pay Per Use, described as experiments run with named counterparties, Ceramic.ai and You.com, rather than a feature you enable: Ceramic pays per query when content surfaces in a result, You.com pays on demand for specific premium content. One dated item has a predictable effect regardless of which payment rail ships. Cloudflare has set 2026-09-15 as the date it blocks mixed-use crawlers by default on pages that host ads, and per TechCrunch's 2026-07-01 report the default applies to new customers, new sites created by existing customers, and all existing free customers. Checking which side of that default your zone lands on costs minutes and depends on nothing shipping.

x402 does not need Cloudflare, and that changes the decision

The protocol underneath is separate from Cloudflare's product and is not gated. Cloudflare's own Agents documentation shows x402 in use with the packages x402-hono, x402/fetch, x402/evm and agents/x402, lists ten networks including Base, Ethereum, Polygon, Optimism, Arbitrum, Avalanche, Solana, Aptos, Stellar and Sui, and carries no waitlist language. The x402.org homepage states the flow plainly: if a request arrives without payment, the server responds with HTTP 402, prompting the client to pay and retry. The site also states zero protocol fees, with network fees still payable. A facilitator handles verify and settle so you do not run settlement yourself. Here we have to correct an earlier note of ours which said public facilitators are free. Coinbase CDP's documented pricing on 2026-08-09 is 1,000 onchain transactions per month at no cost, then 0.001 dollars per onchain transaction, with verification always free. Free was close enough to be wrong, and it is exactly the class of vendor fact that needs a retrieval date attached to it. The consequence is the part usually skipped. Running x402 without the Gateway means accepting stablecoins directly, so the question stops being whether to monetize and becomes who books this, under which tax treatment, in which jurisdiction, and who holds the keys. That is a different approval from turning on a CDN feature, and it belongs with whoever owns the accounts rather than whoever owns the CDN.

Where we put 402, and where we refuse to

The boundary we settled on is not in any vendor documentation, so treat it as our position rather than a fact. Split your surfaces into two classes and write the split into your methodology. The free class is anything already measured, already computed, already written; its job is to be retrieved and cited, and a paywall in front of it trades the thing you are trying to build for small change. The paid class is anything produced on request, which burns tokens and wall-clock time per call. On our side that line runs through one dataset. The weekly visibility index is open and free, because being quoted is the point of publishing it. Measuring one named brand on demand is the other side of the line: it fans out across live engines and costs real money each time it is invoked. Technically that is a tool answering 402 until payment is attached, not a page behind a paywall. Two operational rules follow, and both are ours rather than anyone's published policy. Do not declare a price on an endpoint that does not work. In our own operating experience maintaining tool listings, a declared capability with nothing behind it gets read as a defect rather than as a roadmap, and we have no public catalogue policy document to cite for that, so treat it as our practice and not as a rule of the ecosystem. And do not stack two paywalls on one path: a bot-facing paywall and a CDN-side price on the same URL leave precedence undefined and destroy your ability to attribute any traffic change to either one.

What we could not verify, and therefore left out

The Monetization Gateway page is silent on commission: a case-insensitive search of its HTML for commission, revenue share and take rate returns no matches, checked 2026-08-09. Cloudflare Wallets names no funding mechanism, no networks and no assets. Pay Per Use publishes no payout figures. None of those silences appear above as a number. On the two publisher programs we can confirm existence and quoted terms, and not much beyond that. We could not extract a machine-readable publication date from the Perplexity hub pages, and the revenue-share sentence we quote is forward-looking in the post's own tense, so we cannot tell you from the page alone what is running today. We could not establish eligibility for an independent publisher of our size, or payout mechanics, for either program. Those are open questions, not implied negatives, and the difference between those two things is the whole lesson of section three. We also publish no third-party revenue-share ratios here. We hold internal figures on how many sites in a given marketplace actually receive money, but we could not attach both a named source and a retrieval date to them today, and a ratio without both is a rumour with a decimal point. Last, we verified no end-to-end payment: every claim above stops at what an unauthenticated HTTP request can observe. That is a real limit on this article, and it is also why each claim ships with the command that produced it, so you can re-run the lot on a later date and see what moved.

Questions this answers

Machine access

• /api/articles.json — every piece we published, with the measurement each one rests on
• /api/aiv.json — the AI Visibility Index the numbers above come from
• How the index is measured