Count every call. Balance every cent.

I will separate admission, measurement, pricing, accounting, and presentation. Before a call, admission reserves budget and checks hard quotas. During/after the call, the gateway emits idempotent usage events. Pricing applies the rate card effective for that request contract. A double-entry ledger records provisional and finalized value. Provider receipts reconcile differences. Dashboards and invoices are projections of the ledger, not parallel sources of truth.

I need to clarify whether we bill pass-through cost or markup, prepaid versus postpaid, currency/tax, provider contract time basis, token categories, streaming cancellation, free tiers/credits, hard versus soft budgets, maximum tolerated overspend, invoice close/reopen policy, and which source is authoritative when local and provider counts disagree.

Units include input, cached-input, output, reasoning, image, audio seconds, embeddings, batch discounts, tool invocations, data egress, and fixed operation fees. Each provider adapter maps native receipts to canonical meter dimensions without erasing provider detail. Users see per-run estimates; admins set monthly org budgets plus project/user/model/tool quotas, alerts, and approval thresholds; finance closes invoices, posts credits, and audits discrepancies.

Functional flows: quote/reserve, stream usage, extend reservation, finalize/release, ingest provider receipts/files, price, reconcile, aggregate, alert, invoice, correct, export. NFRs: P99 admission under 50 ms, usage durability 99.999%, dashboard lag under two minutes, hard-budget overspend bounded to a configured exposure, invoice error under one basis point, seven-year ledger retention, and tenant/currency isolation. Payment collection and tax filing are outside the first design.

1. One logical provider request maps to one canonical usage identity even if gateways/reconcilers retry.
2. Money uses integer minor units or micros with explicit currency; token quantities use integers; no binary floating-point accounting.
3. A price is selected by a documented business timestamp and contract/rate-card version pinned to the request.
4. Reservation + posted charges cannot exceed a hard budget except the explicitly configured in-flight exposure.
5. Ledger entries append and balance debits/credits; corrections and invoice adjustments never edit prior entries.
6. A closed invoice pins its line aggregation and ledger cutoff. Late facts create a credit/debit memo or controlled reopen—not silent change.
7. Every customer-facing amount traces to usage events, price calculation, provider receipt status, and ledger entries.

Sixty million requests/day is 694 requests/s average. Per request mock cost is 3,000/1M × $2 + 1,200/1M × $8 = $0.0156. Daily provider cost is about $936,000. With a 1.30 reservation headroom, admission may hold $0.02028 per expected request, or $1.217M total if every request were simultaneously reserved—which it is not; reservations follow concurrency and expiry.

If each request emits start, two streaming increments, terminal estimate, and provider-reconciliation event, that is 300 million events/day or 3,472/s average; the supplied 25,000/s peak is plausible. At 700 bytes/event, raw event log is ~210 GB/day. Hot online budget state is small; immutable event/ledger retention and high-cardinality aggregation dominate storage.

ProviderRequest owns the platform/provider correlation and pinned rate contract. UsageEvent is immutable, dimensioned, and deduplicated by provider/gateway sequence. PriceVersion maps canonical meter dimensions to integer micros and rounding/minimum rules. Reservation holds a maximum liability against a hierarchical budget. LedgerTransaction contains balanced entries such as customer accrued usage, provider cost accrual, reserved liability release, credit, and adjustment.

Admission is a command with an idempotency key and estimated maximum units. It returns one reservation and signed permit. Streaming increments reference monotonically increasing cumulative counters so retries are differences, not double charges. Finalize closes the local estimate and releases unused hold. Provider ingestion records raw receipt plus normalized facts. Invoice close pins ledger cutoff and query manifest.

The model gateway asks the admission service with org/project/user, model/provider, maximum token/tool plan, and run idempotency key. Policy computes hierarchy and price estimate, atomically reserves allowance in the budget shard, and returns a short-lived permit. The gateway invokes the provider and emits cumulative usage snapshots through a durable regional collector, then finalizes with local tokenizer/provider fields and response status.

The canonicalizer validates dimensions and dedupe sequence, the pricing service selects the request’s pinned rate version, and accounting posts balanced provisional ledger entries plus releases unused reservation. Provider receipt ingestion arrives by API/file/webhook later, normalizes exact provider units, and a reconciler posts variance entries. Materialized aggregates feed alerts/dashboard. Invoice close reads finalized ledger entries up to cutoff, applies contract/tax/credit rules, stores line lineage, and signs the invoice manifest.

I distinguish a hard financial budget from high-rate operational quotas. The authoritative budget shard owns available = limit + credits − posted − active_reservations. Admission performs compare-and-set/transactional decrement for the requested maximum, with idempotency and expiry. Only 5,000 one-dollar reservations succeed; the rest queue, downgrade, require approval, or fail according to policy. Regions route that organization to one budget authority or consume preallocated bounded regional escrow.

Streaming usage is cumulative. When actual use approaches the hold, the gateway requests an extension before allowing more output/tool calls. Failure to extend stops generation safely and records why. Cancellation/finalization posts actual charge and releases the difference. Expiry alone cannot release an active call: gateway heartbeats or lease ownership prevent double release; a sweeper reconciles abandoned reservations after a grace period.

For token-rate limits, local token buckets can tolerate bounded drift. For money, any regional escrow is part of the explicit maximum overspend envelope and is periodically rebalanced—never unlimited eventual counters.

The gateway emits cumulative observed usage throughout the stream, so the ledger has a provisional lower bound even if the terminal callback is lost. It actively cancels the provider request on client disconnect when supported, records cancellation acknowledgment or UNKNOWN_PROVIDER_STATE, and retains enough reservation for the configured worst-case continuation until reconciliation or timeout policy.

We post provisional usage to the dashboard but mark it estimated. When the provider receipt arrives with the final cumulative units, its idempotency identity is provider account + request ID + receipt version. Reconciliation compares the exact canonical dimensions and appends a variance: debit for additional usage or credit for overestimate. If the customer invoice closed, policy either creates an adjustment on the next invoice or controlled credit/debit memo; the old invoice remains immutable.

Alerts distinguish missing terminal receipt from real cost anomaly. Repeated provider lag can increase reservation factor or disable hard-budget use for that provider.

There are at least two valuations: provider cost and customer charge. Each has an effective-dated rate contract and explicit selection basis. I prefer pinning the price version when a logical provider request starts; if contract says per-usage chunk time, each cumulative delta carries provider event time and is priced accordingly. The rule is data, not worker-local clock.

The six-hour agent may create several provider requests and tool calls; each usage event stores occurred_at, recorded_at, region, provider request, run, and pinned price IDs. Bitemporal storage handles backdated contract corrections. Currency conversion pins an approved FX rate/version and rounding order. Invoice lines aggregate integer micros by meter/price/currency before final minor-unit rounding.

We test exact boundaries, daylight-saving transitions, cached versus input tokens, batch discounts, minimum charges, canceled calls, and provider pricing revisions. “Current price × total tokens” is not auditable.

Gateway snapshots use (request_id, meter, cumulative_sequence); duplicate/out-of-order snapshots collapse to the maximum accepted cumulative value, with negative deltas rejected or treated as explicit corrections. Provider receipts use their immutable receipt/version IDs; duplicate same-version events are no-ops, while corrected versions append a supersession and variance.

The source-authority matrix is meter-specific. Provider invoiceable units usually win for provider cost. Customer charging may use provider units or a contractually defined local tokenizer; the decision and tolerance are versioned. A 3% discrepancy opens a reconciliation case, blocks finalization above threshold, and tracks tokenizer/model/provider version. We never overwrite local observation to make the books match.

Invariant monitors compare total provider cost accrual to provider statements, customer charges to contract rules, reservations to active runs, and every ledger transaction’s debits to credits.

Usage records should reference prompt/response hashes and classification, not raw content. Tenant/account IDs live in every partition and ledger key. Provider credentials remain in the model gateway; metering sees request identity and units, not secrets. Fine-grained roles separate budget admin, price admin, invoice close, credit issuance, support view, and audit. High-value adjustments use dual approval.

Ledger/event stores use encryption, append-only writer roles, integrity checkpoints, WORM exports, and account-level reconciliation. Idempotency keys are tenant-scoped and unguessable. Audit captures who changed rate cards, budgets, credits, invoice state, and policy. Regional usage collectors persist locally through control-plane outage and never route EU content; only privacy-safe meter dimensions replicate globally where policy allows.

Health: admission latency/denial, reservation contention/age/leaks, usage collector lag/loss/duplicates, pricing failures, unpriced events, provider receipt lag, reconciliation variance, ledger imbalance, aggregate lag, alert delivery, invoice close duration, and regional escrow exposure. Quality: percentage finalized from provider truth, cost/charge variance, invoice dispute and correction rate, budget overspend envelope, per-provider tokenizer difference, orphan requests, and line-level audit completeness.

Rollout shadows gateway counts against provider bills, shows estimates without enforcement, enables soft alerts, then hard budgets for low-risk providers, then invoices in parallel with finance reconciliation. Rate-card changes run historical test vectors and canary accounts. Every stage retains old projections, rollback, manual override with reason, and a global stop on new reservations.

The Token Ledger reserves budget before a provider call, meters cumulative usage idempotently during and after it, prices each dimension under a pinned effective-dated contract, and posts balanced append-only ledger entries. Provider receipts arrive later and append reconciliation variances. Dashboards are provisional projections; closed invoices pin a cutoff and receive explicit adjustments rather than history edits.

Hard budgets use one authority per account or bounded regional escrow; token quotas can be looser. The central trade-off is admission availability versus overspend certainty: I make the exposure explicit and bounded rather than pretending eventually consistent counters enforce money exactly.