AI cost has a telemetry and instrumentation problem before it becomes a billing and ROI problem. The invoice tells you what you spent. It does not tell you which team, feature, prompt version, eval run or agent loop spent it, or whether the spend produced anything useful.
That is the gap AI FinOps exists to close. This post covers the five sources AI cost telemetry comes from, the one record every source should emit, what to collect at each stage of the AI lifecycle, and how to roll it up into numbers finance and product can act on.
What is AI FinOps telemetry?
AI FinOps telemetry or Telemetry for AI FinOps is the usage and cost data captured at every point where an application, agent or data platform calls a model: tokens, model, latency, outcome, computed cost and the business context that says who ran it and why. Along with billing data, it is the foundation for FinOps for AI: allocation, chargeback, anomaly detection and unit economics such as cost per task. Even though telemetry is closely related to instrumentation, they are slightly different concepts. Instrumentation captures the measurements while telemetry carries the results.
Why does Cloud FinOps telemetry fall short for AI?
Cloud FinOps runs on resource-level billing: an instance, a bucket, a tag, an hourly rate. AI breaks that model in four ways.
- The billable unit is a request, not a resource. One API key serves dozens of features. Without request-level telemetry, a $40K line item is one undivided number.
- Cost per request swings by orders of magnitude. A short classification call costs a fraction of a cent. A long-context agent run costs dollars. Averages hide the outliers that drive the bill.
- Spend is pooled. Provisioned throughput, reserved GPUs and shared vector stores serve many workloads. Allocation needs usage telemetry, not invoices.
- Cost and quality are coupled. A cheaper model that fails 30% more often is not cheaper once retries, escalations and human review are counted.
The fix: treat every model call, eval run, pipeline job and agent step as an event that carries its own cost and its own context.
The five sources of AI cost telemetry
No single source sees everything. Enterprises pull AI telemetry from five source types. Most need at least three: one close to the request, which carries context, one close to the provider, which catches shadow usage, and one close to the bill, which carries the authoritative cost.
Billing exports and invoices sit beside these five as the ground truth. Gateway and SDK cost is computed from tokens and a price table. The billed number, after commitments and discounts, is what finance recognizes. Reconcile the two monthly; a persistent gap means a stale price table, a missed discount or traffic going around your instrumentation.
Two sources are gaining fast
Data platforms are becoming AI routing layers. For example, Databricks Unity Gateway logs routed model and MCP requests to a system table with per-user budgets, and Snowflake records Cortex AI function credits per query with user, role and query tag.
If AI runs inside the data platform, this source is not optional. Provider usage APIs are the only view of the prototypes, coding tools and scripts that bypass your gateway. Map every key and project to an owner and this source tells you how much the gateway is not seeing.
One record for every AI cost event
Agree on a record before anything else. Every model call, from any source, emits the same core fields. Use the OpenTelemetry GenAI attribute where one exists so tools and teams speak one vocabulary.
The last row is where most programs fail. Token counts answer “how much?” and never “who?” or “for what?” Propagate context tags from the calling service through the gateway or SDK wrapper so they land on every span automatically, not by developer discipline. Two rules from the spec: record billable tokens when the provider reports both used and billable, and omit a count rather than estimate it.
What to collect at each lifecycle stage
The core record is the floor. Each stage adds a few signals that explain why cost moves.
Three stages deserve a closer look
Evals are a hidden cost center. A 2,000-case suite scored by an LLM judge on every pull request across three candidate models can out-cost the application it protects, and judge calls are often the larger share.
Pipelines are where AI FinOps meets data FinOps. An AI unit cost that ignores the embedding refresh or feature pipeline on Snowflake, Databricks or Google BigQuery understates the real number.
Agents are where real-time telemetry stops being optional. A single task can trigger dozens of model and tool calls, the count is decided at runtime by the model, and a runaway loop can burn a week of budget in hours. A monthly invoice arrives too late.
Two standards
OpenTelemetry GenAI semantic conventions cover the usage side: span names, attributes and metrics for model calls, embeddings, retrieval, agents, tools and MCP. The two client histograms, gen_ai.client.token.usage and gen_ai.client.operation.duration, answer most operational cost questions alone. The conventions are still marked Development and attributes changed in each recent release, so pin a version and plan for migrations. Reference: opentelemetry.io
FOCUS, the FinOps Open Cost and Usage Specification, covers the billing side. It normalizes cloud, SaaS and data platform billing into one schema for cost, consumed quantity, pricing unit, tags and commitments. Version 1.4 is current, and the FinOps Foundation has said future versions will expand coverage of AI spend. Reference: focus.finops.org
OpenTelemetry says who used what. FOCUS says what it costs at the contracted rate. If possible, join them on provider, model or SKU, time window and account.
From telemetry to unit economics
Telemetry gains value when it rolls up to a unit that gives business teams value. Pick audiences and report them the same way every hour, day and month. Report distributions; p95 cost per task usually matters more than the mean. Never report a cost metric without its paired quality metric.
Where to start
- Inventory. Find every API key, cloud AI project, GPU pool and AI SaaS subscription. Assign an owner. Revoke the orphans.
- Put model traffic through one layer. A gateway or shared SDK wrapper that emits the core record and stamps context tags on every call. Turn on usage tracking in every data platform that routes model calls, and pull provider usage APIs for whatever bypasses the gateway.
- Adopt the OTel GenAI conventions, pinned to a version. In the gateway, the SDK wrapper and your agent frameworks.
- Ingest billing in FOCUS and reconcile monthly. Computed cost against billed cost, per provider. Investigate any gap above a few percent.
- Wire outcomes to the same IDs as cost. Task success, eval scores and user feedback on the trace or run ID. Then publish unit economics per use case: cost per outcome, waste rate, forecast variance.
The thread through all of it is simple. Every place AI consumes resources should emit an event that says what was used, what it cost, who it was for and whether it worked. Get that right and FinOps for AI becomes a reporting exercise instead of a forensic one.
Revefi joins these sources, from the model API to the Snowflake, Databricks and BigQuery system tables, into one cost and usage view, and its RADEN agent acts on what it finds. See how at revefi.com.
Sources
- OpenTelemetry Semantic Conventions for Generative AI, OpenTelemetry project
- FOCUS Specification, FinOps Foundation
- FinOps Foundation Launches FOCUS 1.3, Linux Foundation, December 2025
- Managing Cortex AI Function costs with Account Usage, Snowflake documentation
- Unity Gateway, Databricks developer documentation
- Revefi, www.revefi.com
