What Does It Cost to Run an App Online Each Month? Servers, Serverless or Edge Functions — First Learn to Split the Bill in Three

This is for anyone putting their first app online. The first half covers how to choose and how to estimate the cost; the second half builds an LLM gateway from scratch and works out the bills of seven clouds line by line. Prices were checked on 2026-10-09; how they were queried and the full tables are in the appendix.¹

The conclusion first (for the impatient)

1. Start on free tiers. A personal site on Cloudflare Pages or GitHub Pages costs nothing; Cloudflare Workers is free up to 100,000 requests a day; in China, Alibaba Cloud runs ¥99-a-year offers for both a server and a database. Most apps start on a few yuan to a few dozen yuan a month.
2. Do not pay for waiting. If your app spends most of its time waiting on others (calling an LLM, calling third-party APIs), avoid serverless billed by duration: it charges for every second you wait.
3. At volume, watch the bytes you carry and the entries you write. By then CPU is the cheapest line on the bill.
4. If your users are in mainland China, use a Chinese cloud and get an ICP filing first.

Every sentence above is explained below.

Who this is for

• You have an app you want to put online: a personal website, a mini-program or app backend, an AI app that calls an LLM, or a small tool that processes images or video. You have never bought a server and cannot quite say how serverless differs from one.
• After the first half you will be able to read what cloud providers sell, split your app into three bills, pick a way to start, estimate the monthly cost, and know when to switch.
• The second half is for those who want the details: an LLM gateway as a worked example, assuming you know how to call an LLM API and that streamed output arrives piece by piece over SSE. You can skip it without losing the conclusions of the first half.
• The article does not cover model inference itself (GPUs, inference engines), and touches compliance only where it shapes the choice.

Glossary

What is on a cloud provider’s shelf

The first visit to a cloud provider’s site shows dozens of products. There are really only three places to put your code, plus a class of parts someone runs for you:

Five characters keep turning up in this article; meet them first:

An ordinary request

User：Ask the LLM for me: do I need an umbrella tomorrow?

Your service：Got it, passing it to the LLM now.

Twenty seconds later

LLM：Tomorrow... light... rain...

Your service：I was busy for 10 ms. The other 20 seconds I waited for it to finish, word by word.

Cloud：Which raises the question: what do I bill you for?

You：That is exactly what this article works out.

“Your service” in the comic is the program you put in the cloud. It works for about 10 ms, yet each request waits 20 seconds for the LLM to finish talking. A cloud provider has three ways to bill it, and the bills can be several times apart:

Billed by CPU time (Cloudflare Workers)

Cloud：I only charge for the 10 ms your service actually works. Waiting is free.

Billed by duration (Lambda, Function Compute)

Cloud：From the moment the request arrives until it ends: 20 seconds, every one of them.

Your service：But I spend most of it staring at the wall!

Billed per machine (cloud servers)

Cloud：You pay for every machine you rent, busy or not.

Besides running code, two more bills are easy to miss: egress (data leaving the cloud; most clouds bill it per GB, Cloudflare does not) and bookkeeping (databases and caches; some are paid per instance, some per read and write).

Learn to split the bill: your app in three parts

Going live for the first time

You：I want to launch a mini-program. Should I buy a server or go serverless?

Cloud：Hold on to your wallet. What does your app do all day?

Cloud：Does it wait on others, compute, or carry things around? Does it keep books?

You：So I have to take it apart first.

Splitting the bill takes three questions plus one constraint, and each has a way to find your own answer:

1. Does its time go to waiting or to computing?

Look at what your endpoints do:

• They call an LLM or third-party APIs (payments, maps, SMS), or run slow database queries, and a call waits for seconds → a waiting app.
• They process images and video, render PDFs, compress, or run a local model → a computing app.
• They read or write the database and return within tens of milliseconds → a light app: little waiting, little computing.

2. How many bytes does each request move?

Open your app in a browser, press F12 for the developer tools, switch to the Network panel and click a request to see its size. For images and video, look at file sizes; for an AI app, look at how long the prompt and answer are (a Chinese character is about 3 bytes, an English word about 5–6). A few KB is light; hundreds of KB or more is heavy.

3. Does it keep books entry by entry?

Does each request read or write the database? Does it deduct a balance, count calls or rate-limit in real time? If so, bookkeeping is a fixed cost for you, and providers bill it in very different ways.

Plus one constraint: where are your users? Mostly in mainland China means a Chinese cloud, and an ICP filing before the site serves anyone; overseas platforms such as Cloudflare have no mainland nodes and are slow from there. Mostly overseas, and you have many more options.

Then estimate the volume: requests per month ≈ daily active users × requests per user per day × 30. For example, 1,000 daily users tapping 50 times a day is 1.5 million requests a month.

With those numbers, the monthly bill is roughly four items added up:

Whichever column is cheaper depends on which row your app spends the most in.

Find your place: how four common apps start

First, how the four apps start:

To start, use the free tiers and new-customer prices first:²

When should you switch? Watch which of the three bills grows first: more requests and longer waits mean duration-billed serverless should give way to an always-on server or edge functions; more traffic means looking at egress prices (Cloudflare and Oracle are cheap); more bookkeeping means Cloudflare’s per-operation billing keeps growing while your own database is paid per instance. The LLM gateway example in the second half works these three bills out from 10 million to 1 billion requests a month.

A few more common services, sorted the same way:

Table: common services sorted by the three questions (this article’s judgment, not official advice).

For computing apps, what matters is the price of a CPU-hour:

Workers’ CPU costs about 3.6 times an Alibaba Cloud monthly machine and about 1.5 times an AWS on-demand machine per CPU-hour.³ Sounds steep? But machines are bought for the peak and sit idle much of the time: against Alibaba Cloud, if your machines average below about 30% use, paying per millisecond comes out cheaper. Duration-billed Lambda is the most expensive per CPU-hour, yet for computing apps that is perfectly fair, since every second it is up it is computing.

So the same serverless product is a trap for waiting apps and a good deal for occasional computing. Computing that runs flat out all day is still cheapest on machines. Mind the hard limits too: Workers allows at most 5 minutes of CPU and 128 MB of memory per request, too little for large image processing or long video transcoding.⁴

A worked example: building an LLM gateway from scratch

The first half was the method; this section takes one concrete app and works out its three bills line by line. An LLM gateway is a good pick because it runs up nearly all three: it waits a lot, computes little, carries medium to heavy luggage on every request and keeps books entry by entry.

The example’s conclusions first, who pays what at three volumes (switch volumes with the tabs on top):

1. Small? Pick Cloudflare. At 10 million requests a month it costs a little over ¥100, while the cheapest self-hosted setup costs about ¥1,400. A cloud provider bills you a floor: two machines, a high-availability database and a Redis instance cost the same with nobody using them.
2. Large? It depends on how fat the requests are. At 1 billion requests a month, self-hosting on Alibaba Cloud, Tencent Cloud or the big three Western clouds costs only 30% to a little over 100% more than Cloudflare. The smaller the requests, the better self-hosting looks; the bigger they are (coding agents, say), the better Cloudflare looks.
3. Oracle does not play by the rules. Its first 10 TB of egress each month is free, so at volume self-hosting in Oracle US East is about three times cheaper than Cloudflare. The price is running your own machines.

Right, let’s build. Say you are launching an LLM gateway: customers come with an API key, you forward their requests to a model and bill them for what they use. We will add one part at a time, and after each part see where the bill grows.

Stripped down, a gateway does six things:

View source (Mermaid)

mermaid11 lines of code

flowchart TD
    C(["Client"]) --> A["1 Look up the API key<br>who is it, which models"]
    A --> B["2 Take a concurrency slot, hold balance<br>return 429 if over the limit"]
    B --> R["3 Pick an upstream line"]
    R --> U["4 Forward the request body upstream"]
    U --> S["5 Relay the SSE stream as is<br>about 20 s, almost no CPU"]
    S --> F["6 Read usage, release the slot<br>settle actual usage, write one log"]
    classDef wait fill:#d0ebff,stroke:#1971c2
    classDef work fill:#fff3bf,stroke:#f08c00
    class S wait
    class A,B,R,U,F work

Figure: the minimal causal chain of one LLM gateway request (illustrative, not any product’s implementation). The yellow steps compute, but each takes only milliseconds; the blue step waits, for anything from ten-odd seconds to several minutes.

These six steps make an LLM gateway differ from an ordinary API gateway in three ways:

10 ms out of 20 s is 0.05%. The gateway spends 99.95% of its time waiting.

Step 1: take the request, then wait

First the core part: take the request, forward it upstream, relay the answer back as is. What this step costs depends on where you stand:

• Workers: bills only those 10 ms of CPU; the 20 seconds of waiting cost nothing.⁶ A 20-second stream costs the same as a 0.2-second one.
• Duration-billed Lambda, Function Compute, SCF: bill all 20 seconds, every one of them, and often one instance can serve only one request at a time.⁷
• Your own machines: waiting holds a connection, not a CPU; but you buy machines for peak concurrency and pay for them idle.

How big is the gap? At 1 billion requests a month, the duration-billed options cost two to ten times Cloudflare:

The cheaper duration-billed options in the chart all run many requests per instance. At small volumes they even undercut self-hosting slightly, since they skip the always-on machines and load balancer; as volume grows it flips.

If you want to see this step in code, here is the Workers version as pseudocode, with comments marking which lines wait and which compute (illustrative, not any platform’s real source):

// Pseudocode (illustrative): one request through a Workers-based LLM gateway
export default {
  async fetch(req, env, ctx) {
    const key = await env.KEYS.get(apiKeyOf(req), "json");         // 1 KV read: waits on the network, no CPU
    if (!key) return new Response("invalid key", { status: 401 });

    const limiter = env.LIMITER.get(env.LIMITER.idFromName(key.id)); // one DO per API key
    const slot = await limiter.acquire(estimateCost(req));           // 2 take a slot + hold balance (1 row written)
    if (!slot.ok) return new Response("too many requests", { status: 429 });

    const upstream = await fetch(pickLine(key), forward(req));       // 3, 4 wait for the upstream's first bytes, no CPU
    const { readable, writable } = new TransformStream();
    ctx.waitUntil(pipeAndFindUsage(upstream.body, writable, async (usage) => {
      await limiter.release(slot.id, usage);                         // 6 release the slot + settle (1 row written)
      await env.USAGE.send({ key: key.id, usage });                  // 6 usage message; a consumer sums per minute into D1
    }));
    return new Response(readable, upstream);                         // 5 a 20-second stream; CPU only for copying chunks
  },
};

On Alibaba Cloud or Tencent Cloud the same six steps become one long-running process (a Higress plugin or your own Go service) plus a Redis instance: key lookups hit an in-process cache and fall back to Redis on a miss; taking a slot and holding balance use Redis INCR / DECR plus a Lua script; usage is batched in the process and written to the database in bulk or sent to a message queue.

The architecture is the same; what differs is how each step is billed.

Step 2: keeping the books

A gateway that bills by usage has to keep books: on the way in each request takes a concurrency slot and holds some balance, on the way out it settles what it actually used, and a usage record is left behind.

Every request makes two entries in the books

User：I paid up front. Take what I use.

Your service：Then I hold a little on the way in, and settle what you actually used on the way out.

On Cloudflare

Cloud：Every entry is billed. A billion requests is two billion entries.

On your own Redis

Cloud：Redis is paid per instance. One entry or a hundred million, same price.

Here the two sides bill in opposite ways.

Cloudflare bills every entry. The largest item is DO row writes, about ¥13,000 a month at 1 billion requests. Slots and balances must be written to the DO’s built-in SQLite rather than kept only in memory, because an idle DO may fall asleep and wake up with its in-memory counts gone, while a stream lasts 20 seconds.⁸ The optimized design, which sums usage inside the DO, flushes once a minute and drops Queues, cuts the bill by more than a fifth. Three more things to settle up front:

• A DO is single-threaded and handles roughly a thousand requests a second. So use one DO per API key, shard large customers’ keys, and never use one DO as a global rate limiter.
• KV is eventually consistent: a change takes a minute or more to reach every location, so revoking a key or changing a balance cannot rely on it alone.⁹
• A D1 database is capped at 10 GB, and the cap cannot be raised. A usage table summed per minute fills it in just over a month, so summarize periodically, split databases by month and move cold data to R2.¹⁰

A self-hosted Redis is paid per instance. A 1 GB two-replica Redis costs ¥77 a month and is rated at 100,000 operations a second; at 1 billion requests a month the peak needs only about 4,600.¹¹ Key lookups, counters and balances all sit on it, and ten times the requests do not move its price at all.

Split the bill at 1 billion requests a month into three parts and you can see at once whose bill the bookkeeping lands on:

More than 80% of Cloudflare’s bill is bookkeeping; 60–90% of the Alibaba Cloud, Tencent Cloud and AWS bills is egress; Oracle’s three parts are about the same size. Egress is the next step.

Step 3: carrying the luggage

A gateway does not just pass messages; it carries luggage. Each request moves 50 KB on average: 30 KB is the customer’s prompt, which the gateway delivers upstream, and 20 KB is the model’s answer, which goes back to the client. For a gateway, forwarding the prompt counts as egress too.

Your service does not just pass messages

User：My question is 30 KB, and the answer is another 20 KB.

Your service：I carry the question to the LLM, then carry the answer back to you.

On the Chinese clouds and the big three

Cloud：Every byte that leaves needs a ticket.

Your service：Even the questions I pass on to the LLM for users?

Cloud：Even those.

On Cloudflare and Oracle

Cloud：No ticket needed. For Oracle, the first 10 TB a month.

Cloudflare needs no ticket at the door, the subrequests a Worker makes are free, and the terms put no cap on bandwidth.¹² The Chinese clouds charge ¥0.5–0.8 per GB, and the big three Western clouds about the same. Oracle’s first 10 TB each month are free, and beyond that North America costs under a cent per GB.

So the bytes per request all but decide who is cheaper:

• Light luggage: self-hosting is cheaper. Below about 34 KB out per request, self-hosting on Alibaba Cloud beats Cloudflare (about 24 KB against Cloudflare’s optimized design). Short Q&A is around 5 KB, where self-hosting costs about a third of Cloudflare.
• Heavy luggage: Cloudflare is cheaper. Coding agents’ requests often carry hundreds of kilobytes, and there Cloudflare is several times cheaper. AWS breaks even even earlier than Alibaba Cloud.
• Oracle is cheapest across the whole range, only drawing level with Cloudflare at three to four hundred kilobytes.
• The smaller the volume, the lower the break-even point. At 10 million requests a month the clouds’ floor already costs more than Cloudflare’s whole bill, so they never break even.

So “Cloudflare is cheaper” holds only when volume is small or the luggage is heavy.

The Chinese clouds have one more lever of their own: if the upstream model lives on the same cloud’s internal network (calling Alibaba Cloud Model Studio from Alibaba Cloud, say), the forwarded prompt need not cross the public internet. Model Studio’s private connection is currently offered only in Beijing and Hong Kong; with the gateway in the same region, at 1 billion requests a month the egress line drops from about ¥38,000 to about ¥19,000 (private-connection fees included). A gateway in Shenzhen reaching Beijing across regions saves almost nothing.

Step 4: turn the volume up a hundredfold

All the parts are in. Now turn the traffic from 10 million to 1 billion requests a month, the three tabs on the chart at the start of this section.

• At small volumes, the floor decides. Cloudflare has no floor and the clouds do, so at 10 million it is an order of magnitude cheaper.
• At volume, bookkeeping and luggage decide. Cloudflare’s money goes to bookkeeping and the other clouds’ to egress, and the two end up only 30% to a little over 100% apart. Oracle’s egress is almost free, which puts it in front.
• CPU barely moves the bill. Doubling the gateway’s CPU time from 5 ms to 20 ms changes Cloudflare’s monthly bill by only 6%, and a self-hosted setup still fits on four 8-vCPU machines. When you are billed by CPU time, the 20 seconds of waiting cost almost nothing.

And one fact that should make you relax: the gateway’s infrastructure is small change next to the model bill. Per request, Cloudflare costs about ¥0.00003 and self-hosting on Alibaba Cloud about ¥0.000045; a call with 7,500 input tokens and 300 output tokens costs about ¥0.017 at DeepSeek V4.1-Flash peak prices, so the gateway is only 0.2–0.3% of it, and even with the far cheaper Qwen qwen-flash just 2–3%.¹³

So the bill alone should not decide the platform. Latency, reachability and stability matter at least as much.

Step 5: move next to your users

The last part: where the entry point sits, and how far it is from your users.

From China Telecom in Shenzhen, the Chinese clouds’ entry points in Shenzhen, Guangzhou and Hong Kong are all around 10 ms away; packets to Cloudflare leave the country over the carrier’s backbone and land in Los Angeles, 162 ms away.¹⁴ Singapore is not necessarily close either: in the same city, Alibaba Cloud is 57 ms away while AWS and Oracle take over 200 ms, about as much as US East. Chinese clouds’ overseas regions usually interconnect far better with Chinese carriers.

What does that do to the first token? A new HTTPS connection needs at least three round trips (TCP, TLS 1.3, sending the request), so at 162 ms per round trip the first token arrives about 0.45 seconds later; with a reused connection, about 0.15 seconds. Chat users will notice; long agent tasks barely will.

Three more things:

• China Network cannot rescue this architecture. Without China Network, users in mainland China connect to data centers outside the mainland. China Network needs the Enterprise plan, a separate subscription, an ICP filing or license for every apex domain, and content review by JD Cloud; its product list includes Workers, KV and R2 but not DO, D1 or Queues (Cloudflare does not say whether these fail on JD Cloud nodes or fall back to overseas). *.workers.dev is blocked in mainland China, so you must use your own domain.¹⁵
• For overseas models, the long hop cannot be avoided. Neither OpenAI’s nor Anthropic’s list of supported API regions includes mainland China or Hong Kong, so a gateway calling them must exit from a supported region such as Singapore, Japan or the United States; Hong Kong can only relay.¹⁶ Seen from a mainland user, the long hop happens either on “user → Cloudflare Los Angeles” or on “offshore gateway → US upstream”, so when calling overseas models Cloudflare’s latency handicap shrinks a lot and the difference is mostly line quality.
• Both sides have had major outages, of different shapes. On 2025-06-12 a third-party cloud that KV depended on went down and AI Gateway’s error rate peaked at 97%; on 2025-11-18 a core proxy failure made KV return many 5xx errors too. A gateway that reads KV or a DO synchronously on every request goes down with the whole world, with no other zone to fail over to. Alibaba Cloud Hong Kong Zone C was down for more than ten hours on 2022-12-18 after a cooling failure, but the failure was confined to one zone, so a multi-zone deployment had a way out.

Traps to step around before launch

Beyond the bill, every cloud has a few traps that can double it, or cut requests off halfway.

Three Cloudflare traps

• Keeping a DO awake for the whole stream. Route the stream through the DO, or arm a setTimeout in it as a lease timeout, and the DO is billed by wall-clock duration, adding up to about ¥215,000 a month at 1 billion requests. Use setAlarm for lease timeouts. Since 2026-10-01, pending outbound calls and waitUntil in a DO also keep it awake (for up to 15 minutes), which makes this easier to hit.
• AI Gateway logs the full prompt and response by default. Accounts that create their first AI Gateway on or after 2026-09-24 pay for logs at Workers Logs prices, about ¥94,000 a month at 1 billion requests. Turn off payload logging, or logging altogether. Also, with Cloudflare’s Unified Billing each gateway allows only 200 requests a minute, which the average rate at 10 million requests a month already exceeds, so a reseller has to bring its own upstream keys (BYOK).
• The default log settings. New Workers have logs on by default and write an invocation log for every call, and by the docs each DO RPC writes one too, so one request makes about 4 log entries. At Cloudflare’s own average of 4.84 KB per entry, the log bill grows about twentyfold. Turn off invocation logs or sample them on the gateway and DO Workers.¹⁷

Four Chinese-cloud traps

• Offshore tiered pricing must be switched on by hand. Alibaba Cloud Hong Kong, Singapore and Tokyo charge less per GB than Shenzhen, so at 1 billion requests a month Hong Kong comes out cheaper than Shenzhen even though its machines cost nearly twice as much. But since 2024-12-12, ECS and EIP are no longer billed on the tiered price (Alibaba Cloud calls it CDT) automatically; you must “upgrade to CDT billing” by hand (free, and irreversible). Without it, Hong Kong costs about ¥21,000 more a month at 1 billion requests. Singapore and Tokyo are the other way round: without the upgrade their base price is below the first tier, so within a range of volumes not upgrading is cheaper.
• Do not send upstream traffic through NAT. Since 2025-09-26 Alibaba Cloud’s NAT gateway bills processed traffic, so routing upstream requests through NAT adds about 23% on top of egress. A pay-by-traffic public IP on each ECS instance is cheaper.
• Long-lived connections are cheap, but timeouts cut them. Alibaba Cloud ALB’s capacity charge is driven by bytes processed, and 23,000 long-lived connections come to fewer than 8 capacity units; Tencent Cloud’s shared CLB does not bill capacity at all. But ALB’s request timeout defaults to 60 seconds, so raise it before long reasoning streams start returning 504.
• The managed AI gateway is not cheap. Alibaba Cloud’s AI Gateway (built on the open-source Higress) has multi-model routing, fallback, consumer API keys and token rate limits, but the smallest size costs about ¥4,000 a month (documented price); you still need your own Redis and database for balances and usage, so at small volumes it costs more than three times the whole self-hosted setup. The Serverless edition, billed from 2026-09-01, has no size floor in the thousands (the enterprise edition’s instance fee is about ¥176 a month), so at small volumes it costs about the same as self-hosting; but it bills public traffic in both directions at ¥0.8/GB, outside the tiered prices, so it gets more expensive at volume.

The four Western clouds: the big three keep the same books as the Chinese clouds, Oracle is the exception

The big three’s egress prices are in the same range as the Chinese clouds’, the equivalent of ¥0.6–0.8 per GB in the first tier and a little over ¥0.5 at volume; their machines, databases and logs cost more, logs at $0.50 per GB being over 8× Alibaba Cloud’s log service. The two effects cancel out, and at volume the big three and the Chinese clouds end up with similar bills. At small volumes AWS US East costs 50% more than Alibaba Cloud Shenzhen, and Azure US East nearly three times as much.

If your clients are in mainland China, the traffic Google Cloud sends back to them falls under the price list’s “to mainland China” rate, $0.20–0.23 per GiB (Google does not say how the destination is determined); pricing all egress that way (an upper bound) raises US East at 1 billion requests a month from about ¥44,000 to about ¥81,000.

Oracle’s egress is almost free: the first 10 TB each month cost nothing, and beyond that it is $0.0085/GB in North America and $0.025/GB in Asia-Pacific; traffic within a region and data handled by the load balancer are not billed separately, and log ingestion is free. So at 1 billion requests a month, 50 TB of egress costs about ¥2,300 in US East, against about ¥38,000 for the same traffic on Alibaba Cloud Shenzhen. The costs hide elsewhere:

• The 10 TB free tier applies per source region group: US East and Singapore each get 10 TB, and regions in the same group share one.¹⁸
• Singapore has only one availability domain (Oracle’s term for an availability zone), so you cannot spread across zones, only across fault domains.
• A pay-as-you-go account gets only 6 OCPUs per availability domain and one region by default, so at volume you must request a quota increase first.
• The NAT gateway caps concurrent connections to the same destination address and port: about 20,000 per availability domain in a three-domain region such as US East, and about 65,000 in Singapore. 23,000 streams in one availability domain going to one upstream would hit it, so machines are better off exiting through their own public IPs (reserved public IPs are officially free).
• The smallest high-availability MySQL is billed as 3 instances, $170 a month, the largest item at small volumes.

Default load balancer timeouts are the easiest trap before launch:

Serverless and edge functions: do not count on them at volume

Each duration-billed product has its own temper:

• Alibaba Cloud Function Compute: bills the configured size, not actual use, so the vCPU is billed in full for the 20 seconds spent waiting upstream and never enters the “light sleep” state in which vCPU is free. The only saving is concurrency per instance: at 1 request per instance the compute charge is about 95 times that at 100.
• Tencent Cloud SCF: by default one instance handles only one SSE connection at a time, and at volume the peak also exceeds the default concurrency quota per region.
• AWS Lambda: default concurrency is 1,000, so volume needs a quota increase.
• Google Cloud Run: one instance can handle up to 1,000 requests at once, the best fit for long-lived connections of the lot, but it still bills instance wall-clock time; it also writes request logs automatically at $0.50 per GiB (an exclusion filter can turn them off), which the chart leaves out.
• Azure: Functions Flex defaults to 16 concurrent requests per instance, so 1 billion requests a month would need 1,447 instances, above the 1,000-instance cap; Container Apps runs once concurrency is raised to 100.
• OCI Functions: one instance handles one request at a time, synchronous calls last at most 300 seconds, and the result only comes back after the function finishes, so it cannot stream.

Edge functions are blunter still: Chinese edge functions cannot serve as a general LLM gateway, because of their limits rather than their price:

Reasoning models often take more than 10 seconds to the first token and more than 120 seconds for long outputs, and agents’ long-context requests easily exceed 1 MB; neither Chinese product has strongly consistent state like a DO, so rate limits and balances still have to go back to a central region. Overseas, neither CloudFront Functions nor Lambda@Edge can relay a 20-second SSE stream, and Azure Front Door’s docs state that it does not support SSE. Edge functions can stand in front of a gateway for caching and authentication pre-checks; they cannot replace it.

As for managed AI gateways overseas, the big three have them and Oracle does not, and none is built for reselling:

• Azure API Management: all three v2 tiers pass SSE through and offer per-token rate-limit policies such as llm-token-limit; at small volumes Basic v2 costs about $148 a month (Microsoft positions it for development and testing). The bottleneck is the cap of 2,048 concurrent backend connections per upstream host: one SSE stream holds one connection, so the peak at 100 million requests a month already exceeds it. The classic tiers state the cap per unit while v2 just says 2,048; read per unit, 1 billion requests a month needs 12 units at about $18,000 a month, and read literally, 12 separate instances plus an extra layer to split traffic. HTTP/2 to the upstream is still in preview in v2.
• AWS: Bedrock AgentCore Gateway (inference targets since 2026-07) can proxy OpenAI, Anthropic and compatible endpoints, passing SSE through unchanged, and since 2026-08 can cap RPM, TPM and concurrency per user identity; but built-in authentication is only IAM, JWT or none, custom checks need an interceptor, which the docs say does not yet work in streaming mode, and there are no per-customer API keys, per-customer usage bills or prepaid balances. API Gateway REST APIs have supported streamed responses since 2025-11, but cannot read usage from the stream.
• Google: Apigee passes SSE through, but its per-token rate-limit policy only works on the most expensive proxy type, about $73,000 a month in call fees alone at 1 billion requests.
• Oracle: no general LLM gateway; its generative AI service only calls models in its own catalog.

Who really runs an LLM gateway on Workers

Only first-hand evidence counts here: official blogs, official docs, deployment configs in repositories and articles signed by founders.

Three things stand out:

1. Apart from Cloudflare itself, none of them is “all Cloudflare”. The closest is Helicone: Workers, DO, KV and Queues on the request path are all on Cloudflare, while analytics and the billing backend are on AWS. This article’s Cloudflare side keeps even billing in D1, which is more aggressive than what these companies actually do.
2. Everyone’s pain is in the bookkeeping layer, not compute. AI Gateway’s log storage, Helicone’s wallet holds, OpenRouter’s database checks on low balances, the Pydantic gateway’s soft KV limits: all of them.
3. Unkey left for reasons specific to its workload. Its reasons: cache reads had to go over the network, with p99 above 30 ms; to work around statelessness it stacked up DO, Queues, Workflows and more; exporting data was painful; customers could not self-host. After moving, latency fell to one sixth. But Unkey has to finish key verification within 10 ms per request, whereas an LLM gateway request lasts 20 seconds anyway, so tens of milliseconds of state reads weigh far less, and that conclusion does not carry over as is.

On the Chinese side the mainstream approach is a long-running gateway process plus Redis, not edge functions:

• Higress: the gateway open-sourced by Alibaba, accepted into the CNCF Sandbox on 2026-03-15, about 9,500 stars. The project says it powers the Qwen app, Model Studio’s model API and PAI, but publishes no traffic figures. Its token rate limiting uses Redis for global counters, the same design point as the DO counters on the Cloudflare side.
• Alibaba Cloud AI Gateway: in an official customer case, all of China Pacific Property Insurance’s LLM traffic goes through the AI Gateway, close to 100 million tokens a day.
• Chinese edge functions: I found no named LLM gateway customer.

How to choose

Start from your app. Once you have split the bill, it goes roughly like this:

1. Users mostly in mainland China? Use a Chinese cloud, and get an ICP filing before the site serves anyone.
2. Only static pages (a personal website, blog, portfolio)? Use static hosting plus a CDN; it costs next to nothing.
3. A waiting app (calls an LLM or third-party APIs)? For overseas users, edge functions; for users in China, one small always-on server; not duration-billed serverless.
4. A computing app (image or video processing)? Duration-billed serverless when the work is occasional, machines when it runs flat out.
5. A light app (an ordinary backend)? Start on free tiers or the cheapest server plan, and redo the three bills once the volume is real.

The decision tree and lists below are for LLM gateways.

View source (Mermaid)

mermaid12 lines of code

flowchart LR
    %% layout: tree
    Q(["Where are your paying users?"])
    Q --> CN["Mainland China"] --> CN1["Mostly Chinese models<br>→ Alibaba / Tencent Cloud mainland: long-running gateway (Higress or your own) + Redis<br>avoid Function Compute / SCF; edge functions only in front"]
    CN --> CN2["Overseas models too<br>→ mainland entry + offshore node for overseas models<br>exit from a region OpenAI / Anthropic support (e.g. Singapore, Tokyo)<br>upgrade Hong Kong nodes to CDT; work out Singapore and Tokyo first"]
    Q --> OS["Overseas"] --> OS1["Small, no machines to run<br>→ Cloudflare Workers + one DO per key<br>billing split by month or an external database; logs without payloads"]
    OS --> OS2["Large and egress-heavy<br>→ self-host on Oracle (first 10 TB of egress free each month)<br>already on AWS / GCP / Azure: egress bill similar to Chinese clouds"]
    Q --> BOTH["Both"] --> BOTH1["→ two entry points, each close to its users<br>billing in one place only, no dual writes"]
    classDef q fill:#fff3bf,stroke:#f08c00
    classDef pick fill:#d3f9d8,stroke:#2f9e44
    class Q q
    class CN1,CN2,OS1,OS2,BOTH1 pick

Figure: a decision tree for choosing by where users are (this article’s judgment, not official advice).

When to choose Cloudflare:

• Your users are mostly overseas, or they are developers calling overseas models;
• You are still small (tens of millions of requests a month or fewer) and want a near-zero floor with no machines to run; no self-hosted setup at the six cloud providers can match it;
• Your requests are large (agents, long context, multimodal), so egress would dominate a cloud provider’s bill;
• Your team can accept bookkeeping-driven design work such as one DO per key and periodic D1 archiving.

When not to use Cloudflare:

• Your users are mostly in mainland China and sensitive to time to first token (chat products);
• You are at around 1 billion requests a month with short requests (under about 30 KB out per request), where self-hosting on a Chinese cloud or one of the big three is cheaper;
• Your users are overseas, you are large, and you are willing to run your own machines: self-hosting on Oracle is about three times cheaper;
• You need strongly consistent global state (global concurrency, a credit pool across keys) and do not want to shard it yourself;
• You need nodes in mainland China but cannot afford Enterprise, or you rely on DO, D1 or Queues.

Closing

Back to “your service” in the comics. In a whole day, the time it actually spends working may add up to less than a minute; the rest of the time it is waiting for the LLM to finish, carrying data for users, and writing entries in the books. No cloud bills it only for the work; each one picks one or two of waiting, carrying and bookkeeping to bill instead.

So when you go live for the first time, split the bill first and start on free tiers; by the time the volume is real, you will know which line to watch. Change the kind of app and the answers to the three questions change, and so does the conclusion: an LLM gateway is bound by waiting, by carrying bytes and by state, and comparing two clouds on CPU price compares the smallest line on its bill.

Rule: first ask three questions — who bills you for waiting, who bills you for carrying bytes, who bills you for state. Then look at which side your users are closer to.

Appendix: how the prices were queried, and the full numbers

Every number in the main text comes from here. “10M / 100M / 1B” in the table headers are requests per month.

How the prices were queried

• The two Chinese clouds were priced through their official command-line tools. Alibaba Cloud through aliyun CLI 3.5.1 (DescribePrice, GetPayAsYouGoPrice), Tencent Cloud through tccli 3.1.180.1 (InquiryPrice*, DescribeDBPrice). Every call is a read-only pricing query, and no resource was created.
• All Cloudflare numbers come from the official pricing and limits pages. Cloudflare has no pricing API; wrangler 4.148.0 and the Cloudflare API only report your own account’s usage.
• The four Western clouds were priced through public pricing APIs that need no login (AWS Price List, Azure Retail Prices API, Oracle’s price list API); Google Cloud’s pricing API needs an API key, so its numbers come from the official pricing pages. US East and Singapore, all on-demand, 720-hour months. Reserved instances or Savings Plans can save another 30–50%, but only on machines, not on egress.
• List prices only. Alibaba Cloud’s pricing APIs also return account discounts (my account, for example, gets 15% off load balancers and NAT), which are left out. Where no API works, the documented price is used and marked; Alibaba Cloud’s AI Gateway, for example, has no pricing module in the billing system.
• Two verification rounds on the same day: every item the first rounds had marked unverified was checked against official sources; what could be confirmed is now in the text, and what still cannot (mostly things that need an account, a load test or a real bill) is still marked.

# Alibaba Cloud: ECS monthly price (40 GB ESSD PL0 system disk, no bandwidth)
aliyun ecs DescribePrice --RegionId cn-shenzhen --ResourceType instance \
  --InstanceType ecs.c9i.xlarge --PriceUnit Month --Period 1 \
  --SystemDisk.Category cloud_essd --SystemDisk.PerformanceLevel PL0 --SystemDisk.Size 40

# Alibaba Cloud: total price of 50,000 GB of public egress on CDT tiers
aliyun bssopenapi GetPayAsYouGoPrice --region cn-hangzhou --ProductCode cdt \
  --ProductType cdt_DataTransfer_public_cn --SubscriptionType PayAsYouGo --Region cn-shenzhen \
  --ModuleList.1.ModuleCode internet_traffic --ModuleList.1.PriceType Usage \
  --ModuleList.1.Config 'Region:cn-shenzhen,internet_traffic:50000,charge_type:PayByTraffic,isp:BGP'

# Alibaba Cloud: Redis (Tair) and RDS monthly prices
aliyun r-kvstore DescribePrice --RegionId cn-shenzhen --ZoneId cn-shenzhen-c \
  --InstanceClass redis.shard.large.ce --OrderType BUY --ChargeType PrePaid --Period 1 --NodeType MASTER_SLAVE
aliyun rds DescribePrice --RegionId cn-shenzhen --ZoneId cn-shenzhen-c --Engine MySQL --EngineVersion 8.0 \
  --DBInstanceClass mysql.n4.large.2c --DBInstanceStorage 100 --DBInstanceStorageType cloud_essd \
  --PayType Prepaid --UsedTime 1 --TimeType Month --Quantity 1 --OrderType BUY --CommodityCode rds

# Tencent Cloud: CVM monthly price (50 GB general-purpose SSD, bandwidth set to 0)
TENCENTCLOUD_REGION=ap-guangzhou tccli cvm InquiryPriceRunInstances --cli-unfold-argument \
  --Placement.Zone ap-guangzhou-6 --ImageId img-mmytdhbn --InstanceType SA9.LARGE8 \
  --InstanceChargeType PREPAID --InstanceChargePrepaid.Period 1 \
  --SystemDisk.DiskType CLOUD_BSSD --SystemDisk.DiskSize 50 \
  --InternetAccessible.InternetChargeType TRAFFIC_POSTPAID_BY_HOUR \
  --InternetAccessible.InternetMaxBandwidthOut 0 --InstanceCount 1

# AWS: egress tiers from us-east-1 to the internet (public Price List file, AWSDataTransfer service)
curl -s https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AWSDataTransfer/current/us-east-1/index.json

# Azure: egress tiers in eastus (Retail Prices API)
curl -s "https://prices.azure.com/api/retail/prices?\$filter=serviceName%20eq%20'Bandwidth'%20and%20armRegionName%20eq%20'eastus'%20and%20meterName%20eq%20'Standard%20Data%20Transfer%20Out'"

# Oracle: egress originating in North America (price list API, B88327 = Outbound Data Transfer - Originating in North America, Europe, and UK)
curl -s "https://apexapps.oracle.com/pls/apex/cetools/api/v1/products/?currencyCode=USD&partNumber=B88327"

Key unit prices

The full monthly bills

Sources and check dates

All checked on 2026-10-09.

• Cloudflare pricing and limits:
  • Workers pricing, Workers limits
  • Durable Objects pricing, Rules of Durable Objects
  • How KV works, Queues pricing
  • D1 pricing, D1 limits
  • Observability pricing (from 2026-12-01)
  • AI Gateway pricing, AI Gateway limits
  • China Network, China Network available products
• Alibaba Cloud (prices from aliyun CLI pricing APIs, rules from the docs; pages in Chinese):
  • CDT public egress, Upgrade to CDT billing
  • ALB billing rules, NAT Gateway billing
  • Function Compute billing, Concurrency per instance
  • AI Gateway sizes and prices
  • ESA Functions and Pages limits, ESA function billing
  • Model Studio over a private connection, PrivateLink billing, AI Gateway Serverless billing
• Tencent Cloud (prices from tccli pricing APIs, rules from the docs; pages in Chinese):
  • Public egress prices, CLB LCU billing
  • SCF billing, SCF SSE, Web function multi-concurrency
  • EdgeOne plans, EdgeOne overage prices, Edge function limits
• Cases and outages:
  • Cloudflare post-mortems: 2025-06-12, 2025-11-18
  • Alibaba Cloud Hong Kong Zone C outage notice (2022-12)
  • Higress, Alibaba Cloud AI Gateway product page
  • GreatFire: workers.dev (third-party monitoring), Cloudflare community: workers.dev blocked in China
  • Cloudflare terms update (2023), Cloudflare log datasets (average entry size)
• The four Western clouds (prices from public pricing APIs or official pricing pages, rules from the docs):
  • AWS: EC2 on-demand pricing (including data transfer), Lambda pricing, Lambda response streaming, ALB idle timeout, AgentCore Gateway inference targets
  • Google Cloud: Network pricing, Cloud Run pricing, Load balancer timeouts, Apigee pricing
  • Azure: Bandwidth pricing, Application Gateway and SSE, Application Gateway 502 troubleshooting (timeout retries), API Management AI gateway capabilities, API Management limits (concurrent backend connections), Front Door, WebSocket and SSE
  • Oracle: Price list (networking), VCN pricing, Press release on 10 TB per regional zone (2021-11-10)
• Model prices: DeepSeek pricing, Qwen qwen-flash
• Tool versions: aliyun CLI 3.5.1, tccli 3.1.180.1, wrangler 4.148.0 (latest on npm: 4.149.0); no CLI was used for the four Western clouds, only the public pricing APIs above.