Can the footprint of a managed AI service be measured without visibility over the infrastructure?

Not directly, and nobody can. You see neither the hardware running your requests, nor the real utilisation rate, nor the provider's batching policy. Every estimate therefore rests on assumptions about the underlying model and the hardware, documented and bounded by an uncertainty range. What matters is less absolute precision than the ability to compare two configurations and measure the effect of a change.

Are the figures published by model providers reliable?

The environmental footprint figures published by AI model providers come without detailed methodology, without any statement on hardware or utilisation rate, and sometimes by actors whose interest lies in understating the result. One widely quoted figure on the consumption of a typical request has drawn substantial debate over its origin and its assumptions. We use them as markers, never as a reference.

Should regulation be awaited before acting?

Regulation on the environmental footprint of AI is arriving. The European artificial intelligence regulation requires providers of general-purpose models to declare their energy consumption and environmental impact, and the annual data centre declaration under the Energy Efficiency Directive applies from 500 kW of installed IT power. The quality of the data produced remains to be assessed, but the move towards mandatory disclosure is under way. In the meantime, the usage levers can be mobilised immediately.

Does reducing AI's footprint degrade the quality of results?

Rarely, and often the opposite. A shorter, better structured message generally produces a more relevant answer than a verbose context. A shared context repository improves consistency of results across teams. A model sized for the task avoids needlessly expansive answers. The only real trade-off concerns batching, which introduces a delay, hence the importance of reserving it for uses that tolerate one.

How does this fit with the FinOps practice?

The token is both the billed unit and the energy consumption unit, so both practices address the same object with the same levers. It is the most favourable case one encounters, since there is almost no trade-off to settle between the two. We produce the indicators in both units and on the same data, so that the AI lead, FinOps and sustainability stop reasoning on three different figures.

Where does one start in practice?

With a conversation with the teams using AI, ahead of any tooling. Are we using several models or one by default? Are we repeating the same tasks with different wordings? Are we sending back a business context on every request that could be shared? Which requests genuinely need an immediate answer? Those four questions surface most of the potential, and they require no prior instrumentation.

GreenOps for Artificial Intelligence

Contrary to conventional wisdom, most of the footprint of AI does not come from training models but from their daily use, where everything remains to be optimized. We measure this impact at the token level (the volume of text processed) and we activate the levers to reduce it, which almost always lowers the bill along the way.

Schedule a conversation See the other GreenOps domains

CHALLENGES

Engagement framework and macro challenges

On the organisational side, AI is usually driven by recent teams with autonomous budgets, while Flexera shows that 47 % of large enterprises now have a dedicated governance function. These structures operate in watertight silos relative to the FinOps or sustainability teams, so consumption grows faster than visibility. On the methodological side, provider-managed services fully hide the hardware used, the real load factor and the batching mechanisms. Every estimate therefore rests on assumptions that must be formally written down rather than guessed.

Engagement scope

Dimensions and stakes addressed in the field

The token as the unit of measurement and of steering

On generative AI services, the billed unit and the energy consumption unit coincide: the token, a fragment of text processed by the model. Total consumption is the energy per input token multiplied by the number of input tokens, plus the energy per output token multiplied by the number of output tokens. In enterprise use, input tokens dominate by far: summarising a twenty-page document consumes thousands of input tokens for a few hundred output tokens. That asymmetry orients the levers, since acting on what is sent to the model yields more than acting on what it returns. Tokens are not limited to text: images, files, audio and video are also tokenised, at far higher unit costs.

Energy per token, what moves it by an order of magnitude

Energy per token is not a constant but the result of six factors: model size in active parameters, meaning those actually engaged on each request, architecture, the level of quantisation applied, the generation of graphics processor running it, the length of the message processed, and above all the request batching rate. A seven-billion-parameter model consumes a fraction of what a seventy-billion-parameter model consumes for the same request. On repetitive, well-defined tasks such as classification, translation or extraction, a small specialised model can consume over 90 % less energy than a large general-purpose model producing an equivalent result. For your families of use, we establish the real gap between the model used by default and the model that would suffice.

Prefix caching and message structure

When the beginning of a message is identical from one request to the next, the provider can reuse work already done instead of redoing it. The mechanism is automatic, but it depends entirely on structure: it only triggers if the first tokens are strictly identical, a threshold set at 1,024 tokens on Azure OpenAI for instance. An assistant querying a twenty-page internal policy illustrates the point. Placing the instructions and the document first, the user question last, means the document is processed only once. Reversing that order has it reprocessed on every request. The difference is invisible to the user and considerable on consumption. The indicator to follow is the share of tokens served from cache.

Request batching and the latency trade-off

A graphics processor handles ten batched requests for less total energy than ten isolated ones. The fixed cost of loading the model and allocating memory is spread out, and the throughput obtained more than offsets the higher power drawn. Batching obviously does not apply everywhere. It is ruled out for a conversational agent or live coding assistance, where the user expects an immediate answer. It becomes relevant for report analysis, ticket triage, content labelling at volume or bulk document summarisation, anything that tolerates a delay. The useful work consists in sorting uses by one question only: does this processing have to complete now, or can it wait?

Frugal usage and message design

Three recurring patterns inflate consumption without improving the result: sending a whole document when one section would do, repeating verbose system instructions on every request, reprocessing unchanged data that could have been cached. To these is added the absence of any instruction on output length, whose energy effect is demonstrated. These levers are less about engineering than about practice: writing short, precise instructions, industrialising message templates rather than letting everyone rewrite their own context, sharing a validated enterprise context repository, which improves both the consistency of the quality produced and the cache rate. Finally, the upstream trade-off remains, since deterministic processing is sometimes enough where a generative model is called out of habit.

Region, timing and constraints specific to AI

The usual location and time-shifting levers apply, with three specific restrictions. Region choice is often narrow, sometimes absent, on fully managed global services. Graphics processor capacity is concentrated in a small number of regions, which mechanically constrains placement regardless of environmental considerations. Finally, some features, batch processing among them, are not available everywhere, and optimising location can close off a more effective optimisation option. Added to this is rack power density, which requires liquid cooling and shifts the energy and water efficiency profiles of the facilities concerned, invalidating the use of fleet averages.

METHODOLOGY

A pragmatic approach, matched to your maturity level

01
Map the uses and their volumes
Inventory of the AI services consumed, the teams using them and the input and output token volumes per use case. Identification of ungoverned uses, common wherever adoption preceded any framework. This step establishes where consumption actually concentrates, which rarely follows the assumed split.
02
Build the energy estimation model
Application of energy per token by model and hardware family, from published data and the available reference work, then conversion to emissions and water consumption through the calculation chain already established on our other engagements. Assumptions on the underlying model, the presumed hardware and the batching rate are documented, and their effect on the result quantified.
03
Work the levers use case by use case
For each significant use: fit between model and task, caching potential given message structure, eligibility for batching given the latency constraint, reduction of the volume sent. Each lever is quantified in euros and in gCO₂e, with the engineering effort and quality risk set alongside.
04
Anchor it in team practice
Translation into applicable rules: routing to the model suited to the nature of the task, a shared context repository, message structure conventions that guarantee caching, a processing queue for deferrable uses, indicators followed jointly by the AI, FinOps and sustainability leads. Those three functions currently decide separately on the same object.

WORKED EXAMPLE

Training or usage, where to put the effort

The dominant intuition attributes most of AI's footprint to model training. Two published orders of magnitude contradict it.

Cumulative usage emissions against those of one training run: Mistral AI reports that one of its smaller models processing a quadrillion tokens emits in the order of 3,000 ktCO₂e, whereas training a larger model emits around 20.4 ktCO₂e. Google has stated that its Gemini model processed 1.3 quadrillion tokens, which gives the scale reached in production.
One training run against one day of service: A mid-sized model takes around 1 GWh to train. Serving nine billion queries, a single day of activity for a mainstream search engine, consumes in the order of 26 GWh. Over a year, training becomes negligible next to cumulative inference.

Training is accounted for as an upstream emission, amortised over the volume served, in the same way as hardware manufacturing. You do not steer it. Inference depends on decisions your teams take every day: which model is called, how much context is sent, what response length is requested, which requests wait and which go immediately. The rise of agentic systems, which chain several requests to complete a task, will only widen that imbalance.

DELIVERABLES

Sample deliverables

Map of AI uses by team, use case and input and output token volumes
Energy estimation model by model family, with documented assumptions and their effect on the result quantified
Model-to-task fit analysis, quantifying the gap between the model used by default and the model that would suffice
Caching diagnostic: current rate, reachable potential, message structure conventions to adopt
Sorting of uses by latency constraint and identification of the workloads eligible for batching
Shared enterprise context repository, improving result consistency and the reuse rate
Levers costed in euros and in gCO₂e, with the associated engineering effort and quality risk
Set of indicators that plugs into your existing cost and footprint dashboards
Position note on ungoverned uses and the proposed path back into the framework

KPIS

Steering indicators

Input and output tokens consumed by use case and by team
Share of input tokens served from cache
Share of tokens processed in batch rather than one by one, on uses not constrained by latency
Energy and emissions per request, and per business functional unit
Average input and output message length by use case
Cost and emissions related to the same unit, followed jointly

FAQ

Frequently asked questions

Environmental measurement
The measurement method applied to AI usage.
Optimisation and eco-design
The same frugality principles, outside the AI perimeter.
Energy sources and location
Where training and inference actually run.
Artificial Intelligence & GenAI
The same inference cycle, steered on the cost side.