Agent Native
Loading...
Back to Patterns

Updated 2026-07-26

Architecture Pattern

Token Cost Attribution Pattern for Multi-Tenant Agents (2026)

An architecture pattern for attributing LLM token spend to tenant, user, and feature: OTEL gen_ai semconv normalization across renamed attribute generations, streaming usage capture, cache-hit accounting at 0.1x pricing, and where Langfuse, Helicone, and LiteLLM fit.

Agent Native•Pattern: Token cost attribution pattern (per-tenant, per-user, per-feature showback for agent platforms)•2026-07-26

Problem

Multi-tenant agent platforms can answer 'what did the provider bill us this month' but not 'which customer, agent, or feature spent it.' Provider dashboards aggregate at the API-key level; a single key typically serves every tenant. Without per-call attribution you cannot price plans against real margin, detect the one tenant whose retry-looping workflow consumes 40% of spend, or run showback when finance asks why the invoice doubled. The trap in 2026 is that the telemetry layer everyone standardizes on is itself unstable: the OpenTelemetry GenAI semantic conventions are still marked Development in the open-telemetry/semantic-conventions repo, and the wild mixes attribute generations - gen_ai.usage.prompt_tokens/completion_tokens were renamed to gen_ai.usage.input_tokens/output_tokens in semconv v1.27.0, and gen_ai.system became gen_ai.provider.name in v1.37.0 (version history per john-hodge.com's 2026 migration writeup and the opentelemetry.io gen-ai attribute registry). An attribution pipeline that keys on one spelling silently drops the spend recorded under the other.

Apply this pattern when more than one tenant, product feature, or internal team shares an LLM budget and someone must eventually answer for the bill: multi-tenant agentic SaaS, internal platform teams doing chargeback/showback, or any product pricing on outcomes while paying per token. It is the measurement complement to spend limits - our cost-guardrail pattern covers caps and kill switches; this pattern covers knowing who spent what, which is what makes those caps settable at sane values in the first place. If you run a single-tenant app with one feature, provider-level dashboards are enough - skip the pipeline.

Components

  • Identity context propagation: a tenant_id, user_id, and feature/route label attached at the request boundary and carried through every LLM call - as OTEL span attributes alongside the gen_ai.* set, as Langfuse trace fields (userId, tags, metadata), as Helicone headers (Helicone-User-Id, Helicone-Property-* custom properties), or as LiteLLM request metadata/tags, depending on the stack
  • Instrumentation layer emitting token usage per call: the OTEL GenAI conventions define gen_ai.client.token.usage as a histogram metric plus gen_ai.usage.input_tokens / gen_ai.usage.output_tokens span attributes (opentelemetry.io gen-ai registry); auto-instrumentation libraries emit whichever semconv generation they were built against
  • Normalization stage (the step nobody ranking for this query mentions): a collector transform or ingestion mapper that coalesces gen_ai.usage.prompt_tokens → input_tokens, completion_tokens → output_tokens, and gen_ai.system → gen_ai.provider.name, plus legacy llm.* attributes from older instrumentations - because mixed-generation telemetry is the documented norm while the spec is in Development (dev.to and sentryml.com writeups both recommend dual-emission and pinning OTEL_SEMCONV_STABILITY_OPT_IN during migration)
  • Streaming usage capture: OpenAI-compatible streams return usage only when stream_options: {"include_usage": true} is set - usage arrives in the final chunk with earlier chunks carrying usage: null (OpenAI cookbook and developer forum); Anthropic streams report usage on message_start/message_delta events; LiteLLM's proxy can inject the flag globally via always_include_stream_usage (docs.litellm.ai)
  • Cache-aware pricing table: a versioned map of model → input/output/cache prices. Anthropic bills 5-minute-TTL cache writes at 1.25x and cache reads at 0.1x of the base input price, and OpenAI's cached-input discount is model-dependent - 50% on the GPT-4o generation, ~90% on the GPT-5 family - so attribution that prices all input tokens at the base rate overstates cached-heavy tenants' costs by up to 10x on the cached span, and a hardcoded '50% off' assumption is itself wrong for current OpenAI flagships (platform.claude.com pricing docs; openai.com/api/pricing)
  • Aggregation store and rollup jobs: per-call cost facts keyed by (tenant, user, feature, model, timestamp) rolled up into daily/monthly showback views
  • Reporting surface: dashboards, per-tenant exports, and margin views joining attributed cost against plan revenue - either built on the observability vendor (Langfuse and Helicone both group cost by user/property natively) or on your own warehouse

Flow

  1. 01Assign identity at the edge: resolve tenant_id, user_id, and feature the moment a request enters the platform, and propagate them via context (OTEL baggage, framework middleware) so no call site can forget them.
  2. 02Instrument every model call with both the token counts and the identity attributes on the same span/event. If you use an auto-instrumentation library, record which gen_ai.* generation it emits - that decides your normalization rules.
  3. 03Normalize at ingestion: run a transform that maps every historical and current attribute spelling to one canonical schema (input_tokens/output_tokens/cache_read/cache_write + provider + model). Pin instrumentation versions and OTEL_SEMCONV_STABILITY_OPT_IN so a dependency bump cannot silently rename your billing keys mid-month (the failure sentryml.com explicitly warns about).
  4. 04Capture usage on the streaming path explicitly: set stream_options include_usage on OpenAI-compatible calls and read the final chunk; never estimate streamed output by counting deltas client-side - tokenizer drift makes those numbers unbillable.
  5. 05Price each call with the model's current rates including cache multipliers: cost = input × price_in + output × price_out + cache_read × 0.1 × price_in + cache_write × 1.25 × price_in (Anthropic multipliers; substitute the provider's actual cached-input discount elsewhere). Version the price table with effective dates - promo pricing like Claude Sonnet 5's $2/$10 per MTok through August 31, 2026 ($3/$15 after, per platform.claude.com) means the same call costs 50% more the next day.
  6. 06Roll up into showback views per tenant/feature/month, and reconcile the total against the provider invoice - the delta is your unattributed spend, and it should trend toward zero.
  7. 07Alert on attribution gaps (calls arriving without tenant_id) and on per-tenant spend-rate anomalies - attribution data is also your earliest runaway-loop detector, feeding the caps in the cost-guardrail pattern.

Tradeoffs

Build on an observability vendor vs. roll your own pipeline

Langfuse (userId/tags/metadata on traces), Helicone (Helicone-User-Id and Helicone-Property-* headers), and LiteLLM (per-key/team/end-user budgets and tags with built-in cost tracking) each give per-tenant cost grouping out of the box - for most teams that beats building an OTEL pipeline. The vendor path costs you schema control: your tenant taxonomy lives in their property model, and warehouse-level joins against revenue data require their export path. Teams with an existing OTEL stack and a finance-grade showback requirement usually end up owning the normalization and rollup stages themselves.

Gateway-level vs. in-app attribution

Attributing at a gateway (LiteLLM proxy, Helicone) is one integration point and catches every caller, but the gateway only knows what the request tells it - per-feature attribution still requires the app to send the feature label. In-app instrumentation knows the business context natively but must be maintained across every service that calls a model. Most production setups do both: identity headers set in-app, cost computation at the gateway.

Exact provider parity vs. local price table

Computing cost from a local price table gives instant per-call numbers but drifts from the invoice whenever prices change (the Sonnet 5 promo→standard flip is a scheduled 50% jump), a new model ships, or the provider bills something you don't model (tool-use tokens, server-side search). Reconciling monthly against the provider's billed totals and treating the residual as an error budget keeps the local table honest without waiting on billing exports for every dashboard.

Attribution granularity vs. cardinality cost

Keying metrics by tenant × user × feature × model explodes cardinality in a metrics backend - gen_ai.client.token.usage as a histogram with high-cardinality identity labels is exactly what makes Prometheus-style TSDBs fall over. The standard resolution: spans/events carry full identity for the billing pipeline (stored in a columnar/warehouse store), while the metrics path keeps only low-cardinality labels (model, provider, coarse plan tier) for operational dashboards.

Failure modes

  • 01Mixed-generation attribute drop: half the fleet emits gen_ai.usage.prompt_tokens (pre-v1.27.0 instrumentation), half emits input_tokens; the rollup sums only the new key and under-reports spend for every service pinned to the older library - no error, just quietly wrong showback.
  • 02Streaming blind spot: OpenAI-compatible streams without stream_options include_usage return no usage object at all, so every streamed call books as zero cost until someone reconciles against the invoice.
  • 03Cache-blind pricing: charging all input tokens at base rate makes cache-heavy tenants look 10x more expensive on cached spans than they are (Anthropic cache reads bill at 0.1x), distorting exactly the margin analysis the pipeline was built for.
  • 04Stale price table: the accumulator keeps the promo price after it expires (Sonnet 5's $2/$10 becomes $3/$15 on September 1, 2026) and the platform 'attributes' 33% less cost than the invoice shows.
  • 05Unattributed retries and background jobs: retry wrappers, cron agents, and eval runs that bypass the middleware carry no tenant_id, accumulating an 'unknown' bucket that erodes trust in every per-tenant number.
  • 06Subagent leakage: parent runs are attributed but spawned subagents open fresh contexts without propagated identity, so multi-agent tasks book a fraction of their true cost to the tenant.
  • 07Dependency-bump rename: an instrumentation library upgrade flips attribute names to the newer semconv generation mid-month; dashboards keyed to the old names flatline while spend continues - pin versions and the stability opt-in flag.

Implementation notes

  • 01Normalize with a coalesce, not a migration: at ingestion read COALESCE(gen_ai.usage.input_tokens, gen_ai.usage.prompt_tokens) and COALESCE(gen_ai.provider.name, gen_ai.system) so both generations remain valid inputs forever - cheaper than chasing every emitter, and required anyway while third-party instrumentations lag the spec.
  • 02On OpenAI-compatible streams, send stream_options: {"include_usage": true} and read usage from the final chunk (choices is empty on that chunk); on LiteLLM proxy set always_include_stream_usage: true once instead of patching every client (docs.litellm.ai).
  • 03Capture cache token fields separately: Anthropic returns cache_creation_input_tokens and cache_read_input_tokens alongside input_tokens in usage - total prompt size is the sum of all three, and each prices differently (1.25x, 0.1x, 1x base input).
  • 04If you adopt a vendor: Langfuse wants userId/sessionId/tags/metadata on the trace; Helicone wants Helicone-User-Id plus Helicone-Property-<name> headers per request; LiteLLM wants user/team/key plus metadata.tags - in all three, put tenant_id in a dedicated field, not free-text metadata, or you lose indexed grouping.
  • 05Emit gen_ai.client.token.usage as the operational metric but treat spans as the billing source of truth: histograms lose the per-call identity you need for showback, and the GenAI semconv is still Development status - schema-on-read over stored spans survives spec churn better than baked metric labels.
  • 06Version the price table with effective_from dates and pin a calendar reminder on every announced price change (the Sonnet 5 promo expiry on 2026-08-31 is a known event) - then reconcile monthly totals against the provider invoice and alert if the residual exceeds a few percent.
  • 07Feed the output into enforcement: per-tenant attributed spend is the baseline that makes the budgets in our cost-guardrail-pattern-for-agent-platforms page settable from data instead of guesses - the two patterns share the identity propagation layer, so build it once.