Five million tokens consumed overnight, while their user was asleep.
An autonomous AI agent was running on its own, sending one request after another and using GPUs somewhere in the United States. That is roughly 1,000 times the energy consumption of an average user with no one at the wheel.
It was with this almost mundane scenario that James Martin opened the discussion at Sopht Connect 2026. The founder of BetterTech shared the stage with Amiel Sitruk, founder of Terra Cognita, an AI engineering firm specializing in frugal and ethical AI.
One laid out the problem, backed by data. The other explained how to address it, drawing on real-world customer cases.
Together, they raised a question that is becoming increasingly difficult to avoid: how can businesses keep deploying AI without allowing its environmental, energy and financial costs to become unsustainable?
The bill is coming
The numbers are worth revisiting because they put the current AI debate into perspective.
The GPUs powering generative AI consume significantly more energy than traditional CPUs.
As the AI boom drives demand for computing power, data center electricity consumption is on track to triple in both the US and Europe. Associated water consumption is expected to quadruple.
Reasoning models, which break a task down into multiple steps before producing an answer, can consume up to 30 times more energy than standard models. And an AI agent an autonomous system that delegates tasks to other AI systems in a loop can consume up to 1,000 times more than a simple web search, according to the International Energy Agency.
Colossus is perhaps the clearest symbol of where this trajectory is heading. xAI built its supercomputer in just four months to make it one of the largest AI computing facilities in the world. To move that quickly, Elon Musk did not wait for a grid connection. Instead, methane-powered generators were brought in by truck and installed in a disadvantaged Memphis neighborhood already exposed to industrial pollution. Nitrogen oxide levels there nearly doubled within a year.
Twelve similar gas-powered data center projects have since been approved in the United States.
And according to James's calculations, if you look at Anthropic's compute budget, roughly one in three Claude prompts would correspond, in energy terms, to this type of infrastructure.
In other words, this is no longer just a macro-level issue. It shows up in every AI query.
The efficiency trap
This is where things get counterintuitive.
Faced with this trajectory, the industry's instinctive response is to make models more efficient: fewer parameters for the same output, leaner architectures, better hardware. All of this is necessary. But it is not enough and in some cases, it can even backfire.
Sacha Luccioni, who has been researching these issues for years, recently brought together leading experts in Paris to explore precisely this question. The consensus was striking: as models become more efficient, the industry also makes them bigger. The gains in efficiency are then cancelled out by increased scale.
This is the rebound effect applied to AI.
Optimizing technology without reconsidering how it is used is like running faster on a treadmill that keeps accelerating.
A different approach is needed. And that is what Amiel went on to explain.
What is frugal AI?
Frugal AI is often misunderstood. It is not an environmental compromise that means delivering a worse service. It is a structured, documented engineering approach.
France has developed a national framework for frugal AI with AFNOR and incorporated the approach into its national strategy through 2030.
The principle has two dimensions: efficiency and sufficiency.
First, efficiency: for a given need, choose the model and architecture that consume the least resources.
Then, sufficiency: question the need itself before committing to a solution.
Does this really require an LLM? Could a smaller, older model or even a rule-based system solve the problem more effectively? Does it need AI at all?
Amiel illustrated the idea with an iceberg.
Companies often arrive with a problem already framed as a generative AI use case. But beneath the surface, the underlying need is usually more modest, more specific and easier to solve another way.
Five examples of frugal AI in practice
The most convincing evidence comes from real-world applications. Amiel shared several examples from Terra Cognita's projects, each challenging a common assumption about AI.
1. A small model can outperform a large one
For a document-processing use case involving handwritten and multimodal content including photographs of administrative documents with shadows, stamps and irregular handwriting the obvious choice might have been Gemini 2.5 Pro, with its two trillion parameters.
Instead, Terra Cognita selected a small specialized model calibrated on the customer's own data.
The result? Comparable performance at a fraction of the cost.
Lesson one: a well-calibrated model can often outperform a large general-purpose model for a specific task.
2. Models can be dramatically reduced without losing their value
The second example came from the Frugal AI Challenge organized by France's Ministry for Ecological Transition.
A 4-billion-parameter Gemma model was subjected to aggressive optimization techniques, including pruning, quantization and knowledge distillation.
The result: 80.7% of the original capabilities were retained while using just 14% of the initial energy.
An existing model can often be significantly reduced without sacrificing what actually makes it useful.
3. Generative AI does not have to be the default
A customer was receiving around 250 purchase-order emails every day and wanted to automate information extraction.
The obvious solution would have been to connect an LLM.
Terra Cognita took a different route. The team identified recurring templates and built a traditional deterministic pipeline, using generative AI only as a fallback for unusual cases.
The result: the same outcome at a much lower cost.
Generative AI became the safety net, not the engine.
4. Sometimes, non-generative AI is simply better
On an aircraft manufacturing line, a customer needed to detect visual anomalies.
A 2023 non-generative algorithm called PatchCore proved lighter, faster and more accurate than the generative solutions that were tested.
The same principle applied to sentiment analysis carried out with INA on tens of thousands of tweets. Lightweight Transformer models running on CPUs were enough to achieve the desired results.
Generative AI is only one subset of AI and sometimes, it is simply not the right tool for the job.
5. Sometimes, you don't need AI at all
The fifth example was the most radical.
A customer wanted to optimize an employee incentive system. The team first analyzed the data and investigated where the actual problems were.
The conclusion? AI was not needed at all.
The solution was to talk to the operator, change the instructions and redesign the incentives.
The result: tens of thousands of euros saved almost immediately, with zero development cost and zero additional AI footprint.
“I love AI,” Amiel concluded, “but we need to stop the FOMO.”
The companies that get the most out of AI will not necessarily be the ones that use it everywhere. They will be the ones that know when not to use it.
Making AI impact visible
Choosing a smaller model, compressing it or building a deterministic pipeline all require the same thing first: knowing what you are consuming.
Without measurement, there is no frugal AI, only guesswork.
One of the most striking examples came from SNCF. Its roughly 300,000 employees do not use ChatGPT directly. Instead, they access SNCF-GPT, an internal LLM interface that integrates EcoLogits to display the carbon footprint of each response.
For a standard query, the counter sits at around 8–9 grams of CO₂.
But when a user asked the model to search the internet for information, the figure jumped to 3,000 grams of CO₂, around 300 times the impact and 400 times the cost.
In that situation, searching for the information yourself makes more sense.
This kind of behavior only becomes possible when AI's environmental and financial impact is visible at the point of use.
Why frugal AI is a business issue
There is another point worth making: in today's environment, the environmental argument alone is not always enough to change a decision-maker's mind.
The strength of frugal AI is that it addresses several business priorities at once.
Smaller models, often open source, are easier to audit, making them better suited to evolving regulatory requirements. They can be deployed where organizations choose, strengthening digital sovereignty. They cost less to run, improving profitability. And they reduce dependence on a handful of hyperscalers whose ability to meet their own climate commitments is increasingly being questioned, including by the companies themselves.
Frugal AI is not a compromise. It is better engineering.
For organizations looking to scale AI without losing control of their costs, compliance or strategic autonomy, this may be the approach that matters most in the months ahead.
A big thank you to James and Amiel for such a clear-eyed discussion and for giving us plenty to think about when it comes to the future of AI.