// analysis
Cloudflare: who counts the content uses AI companies pay for?
Sources and reading context
Topics
Contents12 sections
An article can help an AI answer a question without bringing a reader to the website that published it. Cloudflare, a web infrastructure provider, wants a payment to follow the content into that other product. Its Pay Per Use beta, announced on 30 September 2026, sets up a commercial relationship: an AI company defines a use it will pay for and offers a price; a publisher accepts; reported uses become the basis for billing. 1
That gives publishers another way to sell their work. It also moves a crucial question downstream: who can check how often the content was actually used? In the system Cloudflare describes, the buyer supplies the events. Cloudflare checks whether they map to an enrolled publisher. Validating those records is different from establishing that every eligible use has been reported. The public announcement describes the former without setting out a universal, independent audit of the latter. 1
As of 2 November 2026, the documents reviewed for this investigation establish how the products are intended to work and the terms publicly described. They do not provide a verified aggregate of Pay Per Use payouts, median publisher earnings or a universal fee schedule. Understanding the conditions for a viable market comes before assigning it a financial return. 1, 2
When the answer replaces the visit
The traditional search bargain involved traffic moving in both directions. A search engine crawled a publisher’s pages and sent readers back. Answers delivered inside the search interface can change that exchange. The publisher’s economics then depend on the visits it retains, their value and any revenue it can earn somewhere else.
Pew Research Center provides a useful historical reference. Its study covered browsing by 900 US adults in March 2025. A click on a traditional search result followed 8% of search visits with an AI summary, compared with 15% without one. Links inside the summary received a click in 1% of visits featuring a summary. That last measure concerns a separate group of links. 10
Method and scope
Pew Research Center, published 22 July 2025. Browsing by 900 US adults in March 2025; search results collected 7–17 April. Traditional-result clicks followed 15% of visits without a summary and 8% with one. The 1% rate concerns links inside the summary, among visits with a summary. Queries differ between groups; these observational associations do not quantify a causal revenue loss.
The results illustrate the vulnerability of referral traffic. They do not establish a causal revenue loss. Queries with summaries can differ from queries without them, and the search results were collected on 7–17 April, after the browsing period. Applying the difference directly to a particular publisher’s revenue would create an unsupported estimate. 10
Charging for downstream use addresses a real economic possibility: a source may improve an AI product even when it receives no visit. How much of that value the buyer will share, and which event makes a payment due, are still commercial questions.
Three products with different billing events
Cloudflare’s products operate at different points in this chain. Pay Per Crawl charges for successful access to content. Launched in private beta on 1 July 2025, it is still described as a closed beta in the documentation reviewed. An identified crawler can receive an HTTP 402 response indicating that payment is required; a successful paid retrieval becomes billable. 3, 4, 18, 20
A publisher can set a default price and use dynamic pricing by URL or content. The documented minimum is $0.001 per crawl. That floor belongs to Pay Per Crawl. It says nothing about the average price buyers accept or the price of Pay Per Use events. Security blocking rules continue to take precedence over charging rules. 4, 5, 6
The 402 status is a signal rather than a complete payment system. The HTTP specification reserves it for future use; the surrounding protocol and commercial agreement supply the terms and settlement mechanism. For Pay Per Crawl, Cloudflare acts as merchant of record and describes monthly payouts through a dedicated Stripe Connect account, subject to applicable settlement conditions. 4, 7, 8
Pay Per Use prices an event further downstream: a billable use covered by a buyer’s program. Monetization Gateway, also announced in beta on 30 September 2026, targets interfaces through which software exchanges requests and responses (APIs), tools and data for which the request itself is the consumption. Its launch describes x402 payments in USDC on Base, with Coinbase as facilitator. The initial closed beta is for eligible US-based participants. Those arrangements should not be assumed to apply to Pay Per Use or to a publisher based in France. 1, 16
Method and scope
Functional diagram, without observed volumes. Pay Per Crawl, launched in private beta on 1 July 2025, concerns successful retrieval. Pay Per Use, announced in beta on 30 September 2026, covers an event defined by a buyer programme and reported by the buyer. Monetization Gateway targets APIs, tools and data; its 30 September beta describes x402 and USDC on Base for eligible US actors. Pricing, licensing, availability and settlement arrangements are specific to each product.
The right charging point depends on the product. An API request can deliver the service immediately. A web page might be fetched, stored in an index, left unused or drawn on repeatedly. Requests and uses can therefore move in quite different directions.
One copy, a hundred uses
Consider a deliberately normalized example. One service retrieves a page a hundred times but uses it only once in a covered experience. Another retrieves it once and uses it a hundred times. At a fictional price of one monetary unit per event, access-based and use-based billing produce opposite invoices. [l0g calculation]
Method and scope
l0g teaching example: service A retrieves content 100 times and uses it once; service B retrieves it once and uses it 100 times. One dot represents one hypothetical event. At an equal normalized price of one monetary unit per event, access bills are 100 and 1 units, while usage bills are 1 and 100. Invented volumes and prices, excluding fees; the sources document only the mechanisms.
These are invented counts, not observed traffic. They show why server requests and uses inside an answer measure different things. Refresh frequency, caching policy and the product’s design all affect the relationship between them.
Charging for every crawl makes the buyer pay for material it evaluates and rejects. Charging for use lets the buyer wait until content contributes to its product. Cloudflare argues that this can attract more buyers. Publishers, in turn, become exposed to the contractual definition of use and the way it is measured. 1
A source appearing in a results list, a passage reproduced in an answer and information influencing a recommendation are distinct events. A high price attached to a rare event may yield less than a small price attached to a broad one. Comparing offers requires understanding the denominator as well as the headline rate.
The buyer keeps the underlying records
Under the Pay Per Use beta, an AI company submits events containing the time of use, the source URL and an identifier. Cloudflare aggregates those reports, bills the buyer and schedules monthly publisher payments. Cloudflare says the program terms require complete reporting. 1
Verifying a crawler’s identity helps a publisher decide who can enter. A URL connects a reported use with an enrolled site. Neither, by itself, describes everything that happens later inside the buyer’s systems. Content can move through an index, a cache (a stored copy for reuse) and an answer generator. Establishing the completeness of reported uses would require additional verification; the reviewed public documents leave that question open. 1, 18 ; analysis
On 16 October 2026, Cloudflare described internal monitoring of Verified bots’ behavior, including compliance with robots.txt preferences and responses to HTTP 429 rate-limit signals. Public trust scores and customer-selected thresholds are presented as future steps. These observations of access behavior leave a separate question: the completeness of Pay Per Use reporting inside the buyer’s systems. 21
Method and scope
Functional circuit described in the 30 September 2026 Pay Per Use announcement, with verified crawler identity at the access stage. Content passes through the buyer’s systems; the buyer reports URL, time and an identifier. Mapping a report to a participant and assessing whether all relevant uses were reported are distinct checks. Behavioural checks announced on 16 October concern access, robots.txt and 429 responses; their scope is distinct from observing all downstream uses. The figure assumes no underreporting rate and does not treat a systematic independent completeness audit as established.
A rigorous comparison of programs would examine their billing definitions, treatment of repeated uses, log retention and dispute procedures. Publishers could seek sample-based checks or independent assurance. These are proposed evaluation criteria, not features confirmed as universally available in the beta.
Verification also has privacy implications. Checking a source’s contribution does not automatically require handing a user’s entire conversation to the publisher. Cloudflare says it is working on additional context, such as topics or keywords, without exchanging personal data. The actual protections need to be assessed against the terms of the particular program. 1
Estimated earnings, invoiced amounts and cash received should remain separately visible. Validated events are one stage of the process; adjustments, potential disputes, fees and payment timing can change the eventual cash contribution. An effective dashboard should support that reconciliation rather than make an event total look like a bank balance.
A crawler can walk away
Stack Overflow offers a particularly useful counterexample. In an interview published on 19 February 2026, engineer Josh Zhang said some crawlers stopped making requests after the site began returning 402 responses instead of the 403 refusals they had previously received. 9
This is a commercial partner’s account, recorded in a conversation with Cloudflare. It is neither a representative market study nor a financial assessment of the program. But it highlights a basic buyer option: declining to purchase access. A control can successfully reduce unwanted consumption without generating a sale. 9 ; analysis
In July 2026, Cloudflare described experiments with Ceramic.ai and You.com involving content appearing in search results and access to premium sources. These examples document commercial approaches being tested. They do not establish the number of paying buyers in the September beta, settled volumes or individual publisher earnings. 2, 11
Fresh, specific and hard-to-replace information should have greater bargaining value. This is an economic hypothesis: a buyer weighs the price against the cost of finding another source, the risk of a weaker answer or the option of abandoning the request. Original local reporting and a generic rewrite of a press release present different substitution problems. Public evidence reviewed here does not quantify that difference.
From a quoted price to newsroom revenue
A small model makes the economics inspectable without pretending to forecast them. Imagine one hypothetical month with one million potential downstream uses. Just 20% ultimately fall within participating programs and become covered, billable, reported and validated events. At an assumed $0.002 per event, gross revenue is $400.
Assume aggregate deductions of 10% and $100 in additional monthly costs. The resulting contribution is $260. The deduction rate is an invented model input, not a published Cloudflare commission. The use price is also hypothetical and separate from the Pay Per Crawl minimum. [l0g calculation]
Method and scope
l0g assumptions in nominal US dollars over one fictional month: U = 1,000,000 potential uses, a = 20% covered, q = $0.002 per event, f = 10% aggregate deductions and K = $100 additional costs. Billable events = U × a = 200,000; gross revenue = $400; after deductions = $360; net contribution before tax = $260. Coverage combines several filters, and the deduction rate is a teaching assumption distinct from Cloudflare’s actual fees.
Now define a comparison explicitly: 50,000 fewer human visits, each contributing an average of $10 net per thousand visits, would put $500 at stake. Neither the visits nor their value are derived from Pew or from l0g’s traffic. They provide a comparison in the same currency and over the same fictional month.
To produce $500 after the model’s deductions and additional costs, the use price would need to be about $0.00333 per billable event, with coverage unchanged. At the initial $0.002 price, one third of potential uses would need to become billable. The accompanying data and calculator expose each assumption. [l0g calculation]
Method and scope
l0g calculation: net contribution = U × a × q × (1 − f) − K. U = 1,000,000 per fictional month, f = 10% and K = $100; benchmark B = 50,000 hypothetically lost visits × $10 per thousand visits = $500. The price per thousand billable events meeting that target is 1,000 × (B + K) / [U × a × (1 − f)]. At 20% coverage: $3.333… per thousand; at one third: $2. Coverage axis spans 5%–100%. Volumes are held constant and all inputs are hypothetical; no demand elasticity, revenue forecast or causal loss inferred from Pew is estimated.
l0g / CLOUDFLARE / REVENUE WORKSHOP
Compare usage revenue with a traffic-loss scenario
Explore two contributions for the same fictional month, in US dollars. Every input is a teaching assumption separate from Cloudflare’s actual prices and revenues.
Change the seven assumptions
TEACHING SCENARIO: one million possible uses, 20% covered, USD0.002 per use, 10% deductions and USD100 in costs. The benchmark assumes 50,000 lost visits contributing USD10 per thousand visits.
- Usage contribution before tax
- Human-visit benchmark
- Gap against the benchmark
Usage contribution remains below the visit benchmark in this scenario.
Revenue and threshold details
- Expected billable uses
- Gross revenue
- Revenue after deductions
- Required price at unchanged coverage
- Required coverage at unchanged price
The coverage threshold falls between 0 and 100% of possible uses.
Calculation and scope
U is possible uses, a the covered share, p the price, f deductions and K additional costs. L and r define the visit benchmark H. Percentages become fractions for the calculation. A zero target and zero costs give a threshold of zero; a positive target with a zero base leaves the threshold undefined.
N = U × a × p × (1 − f) − K
H = L × r / 1,000
p* = (H + K) / [U × a × (1 − f)]
a* = (H + K) / [U × p × (1 − f)]
Coverage combines several commercial and technical filters; it does not measure fraud. Volumes stay constant as the price changes. Buyer responses, taxation and payment timing remain outside the model. Outputs are hypothetical monthly contributions before tax. The ≈ sign indicates rounding; very small amounts remain visible in scientific notation.
Calculations run on your device in this page. Settings remain in memory while it is open.
The 20% share combines several filters. Other uses might belong to buyers outside the program or fall outside the contractual definition. It is not an estimate of reporting accuracy or fraud. Higher prices may also reduce demand. The calculator holds volume constant to isolate a sensitivity; it does not estimate buyers’ response.
Scale matters at the publisher level. Shared infrastructure can make small payments viable when separately negotiated contracts would consume them in administrative costs. Yet an attractive rate on a few thousand events cannot fund expensive reporting. Covered volume, monitoring costs and the durability of the commercial relationship determine the outcome.
Separating generative answers from conventional search
Bargaining power also depends on alternatives to joining a paid program. Here the picture changed in 2026. Google says that on 31 August it completed a worldwide rollout of a Search Console control allowing publishers to exclude their content and links from AI Overviews, AI Mode and relevant generative features in Discover. Its documentation says the choice is not a ranking or inclusion signal for other parts of Search. 13, 19
Opting out also removes impressions and visits from those AI features. Training is controlled separately. The setting broadens publishers’ choices without guaranteeing stable traffic or replacement revenue. Describing the choice as accepting all AI uses or disappearing from conventional search would now be outdated. 13
Cloudflare announced separate controls for search, training and agents on 15 September, alongside an Accountable designation based on requirements already met or commitments with specified timelines. The deployed functionality matters as much as the label. On 30 October, Google also announced plans to support a shared AI-training signal emerging from AIPREF standards work, targeting 2027. That remains a prospective commitment. 12, 14
Preferences, access security and billing operate at separate layers. The robots.txt standard explicitly states that it is not a substitute for valid security controls. A preference must be read by its recipient; access enforcement acts on incoming requests; licensing defines authorized and payable uses. Keeping those layers separate avoids attributing to a technical setting the power to resolve every copyright question. 15 ; analysis
Cloudflare’s new position between publishers and AI products
The strongest case for the system is coordination. A small publisher cannot easily negotiate and maintain an integration with every prospective buyer. Shared infrastructure brings identity, enrollment, reporting and settlement together. Lower coordination costs could make previously uneconomic transactions possible. That is a credible proposition to test. 1 ; analysis
It also gives Cloudflare another potential role between the producer and distributor of information. Buyer admission, offer presentation, billing definitions and the information returned to publishers all shape negotiation. A useful intermediary can become a dependency. This is an analysis of incentives, not a legal finding of dominance or abuse.
Other companies are pursuing related models. On 3 February 2026 Microsoft introduced its Publisher Content Marketplace, then a pilot involving content licensing, publisher-defined usage terms and reporting, with experiments in Copilot. Public descriptions allow a comparison of the architectures. They do not establish which offers a publisher the best return without concrete prices, fees and volumes. 17
In the September Pay Per Use beta description, buyers propose the usage definition and price. Cloudflare describes counteroffers and finer pricing by content and use as future ambitions. The distinction matters: being able to decline an offer and being able to change its terms provide different kinds of leverage. 1
The evidence that would establish a market
A useful assessment would follow cohorts of publishers for several months: active paying buyers, usage definitions, gross receipts, deductions, cash received and continued participation. The distribution of payouts matters too. A large aggregate can coexist with negligible earnings for most publishers.
The next task is to connect the use count to its controls: identifiers, deduplication rules, reporting delays, corrections and dispute rights. That would allow an assessment of the records’ reliability. None of the documents reviewed here establishes a representative financial return for the Pay Per Use market. Missing public data leave an open question; they do not demonstrate that no payments exist. 1, 2, 11
Cloudflare has documented infrastructure and a way to organize an exchange. Its value depends on buyers’ willingness to pay, the scope of paid uses and confidence in reporting. For a publisher, the final test is tangible: how much verifiable net contribution does the relationship provide toward producing the next piece of original information?
Method and limitations
This document-based investigation is current to 2 November 2026. Product and control descriptions come from their suppliers and are attributed accordingly. Stack Overflow provides a partner’s account. Pew’s observational evidence concerns 2025. No interviews, commercial account access or production tests were undertaken.
The scenarios use nominal US dollars over one fictional month. Every input is hypothetical. No traffic statistic has been converted into a revenue estimate for l0g or any other publisher. The model excludes demand elasticity, tax and settlement timing. The evidence file distinguishes publication, update and consultation dates for evolving documentation; full original URLs appear below.
Terms. A crawl is an automated retrieval. A paid use is an event defined by a commercial program. An API lets software exchange requests and responses. The model’s net contribution deducts assumed aggregate charges and additional costs before tax; it is not a company’s reported accounting profit.
Further reading
Our investigation into AI agent payments follows settlement rails and spending authority. The intention economy places these models in the changing advertising business. Access to Google search data provides a comparison with another data market, its conditions and its costs.
Sources
- Pay Per Use: when AI uses your work, you should get paid
Buyer-defined paid uses and prices; publisher acceptance; self-reported events; domain matching, billing and monthly payouts, according to Cloudflare.
Passages and scope
Event or period: Beta announced on 30 September 2026. This announcement does not establish a universal independent completeness audit, aggregate settled payouts or a general fee schedule.
- Your Content, Your Rules
Cloudflare describes Pay Per Use experiments with Ceramic.ai and You.com.
Passages and scope
Event or period: July 2026 announcement and experiments. Announced partnerships and commercial mechanisms do not establish cash received by individual publishers.
- Introducing Pay Per Crawl (private beta)
Original Pay Per Crawl launch: access-based charging and Stripe payment management.
Passages and scope
Event or period: Private-beta launch on 1 July 2025. This is the historical launch announcement, not evidence of current general availability.
- What is Pay Per Crawl?
Closed beta; HTTP 200 access or 402 payment signal; Cloudflare as merchant of record; security-block rules override charging.
Passages and scope
Event or period: Documentation updated 28 July 2026. Documentation reviewed without activating or testing a commercial account.
- Set a pay per crawl price
Default retrieval pricing, optional dynamic prices and a minimum of $0.001 per crawl.
Passages and scope
Event or period: Documentation updated 28 July 2026. The minimum belongs to Pay Per Crawl; it is neither a market average nor a Pay Per Use rate.
- Advanced configuration
URI exclusions and per-content dynamic prices supplied by the origin or a Worker through crawler-price.
Passages and scope
Event or period: Documentation updated 28 July 2026. A claim that one immutable price must cover every page would ignore this newer configuration documentation.
- Manage payouts
Dedicated Cloudflare Stripe Connect account; recorded charges are aggregated and reconciled; monthly payouts subject to settlement conditions.
Passages and scope
Event or period: Documentation updated 23 April 2026. No universal settlement-period or minimum-payout amount is established here; these Pay Per Crawl arrangements cannot automatically be transferred to Pay Per Use.
- RFC 9110, HTTP Semantics, §15.5.3
RFC 9110 section 15.5.3 reserves status 402 for future use.
Passages and scope
Event or period: HTTP specification, June 2022. The status code by itself does not define a complete universal payment system.
- Why Stack Overflow and Cloudflare launched a pay-per-crawl model
Josh Zhang reports that some bots stopped sending requests after responses changed from 403 to 402.
Passages and scope
Event or period: Partner interview published 19 February 2026. A commercial partner’s account, not a representative study or a verified revenue statement.
- Google users are less likely to click on links when an AI summary appears
Study of 900 US adults and 68,879 unique Google searches: traditional-result clicks on 8% of visits with AI summaries versus 15% without; summary-source clicks on 1% of summary visits.
Passages and scope
Event or period: Browsing 1–31 March 2025; search results collected 7–17 April 2025. Observational comparison with different query mixes and later-collected results; these data alone do not identify a causal revenue loss.
- Making AI search smarter
Ceramic.ai: payments linked to appearances in search results; You.com: premium sources requested on demand; Cloudflare calls these experiments.
Passages and scope
Event or period: July 2026 experiments. The examples do not establish settled payment volumes or individual publisher returns.
- Cloudflare Helps End the Search-or-AI-Training Tradeoff
Separate controls for search, training and agents; Accountable designation allows requirements already met or dated commitments.
Passages and scope
Event or period: Controls announced 15 September 2026. Commitments to deploy must remain distinct from controls available everywhere now.
- Search generative AI control
Google describes exclusion from generative Search features without making that choice a ranking or inclusion signal for other Search surfaces; model-training controls are separate.
Passages and scope
Event or period: Worldwide rollout noted as of 31 August 2026. Exclusion also removes links, impressions and traffic from the covered AI features; supplier documentation was reviewed without testing the setting.
- Supporting a shared AI training control for websites
Planned shared training preference in robots.txt, drawing on IETF AIPREF work.
Passages and scope
Event or period: Announcement 30 October 2026; support targeted for 2027. A prospective training-control commitment, not a payment mechanism or a fully deployed standard.
- RFC 9309, Robots Exclusion Protocol, §3
RFC 9309 section 3 states that robots.txt is not a substitute for valid content security measures.
Passages and scope
Event or period: Robots Exclusion Protocol specification, September 2022. The technical standard does not settle copyright rights or legal obligations of AI providers.
- Monetization Gateway beta: charge AI agents for consumption with HTTP 402
Request-based x402 payments; USDC settlement on Base through Coinbase’s facilitator; initial beta for eligible US-based sellers and buyers.
Passages and scope
Event or period: Closed beta announced 30 September 2026. A separate product from Pay Per Use; launch eligibility does not establish availability to French publishers.
- Building Toward a Sustainable Content Economy for the Agentic Web
Microsoft describes licensing, publisher-defined usage terms, reporting and experiments in Copilot.
Passages and scope
Event or period: Publisher Content Marketplace pilot announced 3 February 2026. An interested party’s description; these public details do not establish comparative financial returns.
- Verify your AI crawler
Crawler onboarding and authentication through Web Bot Auth and verified-bot policies, with price information in the HTTP response.
Passages and scope
Event or period: Documentation updated 1 July 2026. Verified identity does not measure all subsequent uses inside a buyer’s systems.
- New opportunities, control and insights for website owners
Google’s initial announcement of independent controls for generative Search features.
Passages and scope
Event or period: Initial trial announced 3 June 2026; source 13 establishes later worldwide rollout. The initial test announcement alone cannot establish deployment status on 2 November.
- Pay Per Crawl FAQ
Repeated billable retrievals can be charged again; unsuccessful responses are not billed.
Passages and scope
Event or period: Documentation updated 23 April 2026. Use the newer July documentation for per-content dynamic pricing; the older FAQ describes a price common to charged crawlers, not an immutable price for all content.
- Continuous accountability for Verified bots
Cloudflare describes internal monitoring of repeated behavior by Verified bots, including compliance with robots.txt preferences and responses to direct signals such as HTTP 429. Public trust scores and customer-selected thresholds are described as future steps.
Passages and scope
Event or period: Internal behavioral monitoring announced 16 October 2026; public scores and customer controls described as future steps. Supplier-described observations of access requests and compliance with site preferences; this post does not establish a complete audit of Pay Per Use events inside buyers’ systems. No independent product test or audit was performed here.
This analysis is not investment advice.
// cite this analysis
l0g, “Cloudflare: who counts the content uses AI companies pay for?”, l0g.fr, published November 02, 2026, updated November 02, 2026, https://l0g.fr/en/analysis/cloudflare-ai-content-payments-usage-count/
$ cd ../analysis