AI’s Cost Revolution: Understanding the Power of Token Economics
AI is no longer just answering simple questions. It is now running agents, writing code, and even powering physical systems. This shift is making the cost of tokens a bigger worry for developers and businesses.
Jigar Halani, Senior Director of Enterprise Solutions Architecture & Engineering at NVIDIA South Asia, explained the growing “token economy.” He broke it into four clear parts: utility, demand, supply, and monetization.
His main point was simple: “When you waste too much, it is when the concept of saving comes into the picture.”
Token Use Is Rising Fast
Token consumption is climbing quickly. Halani shared that NVIDIA had planned to use 16 trillion tokens across the company in 2026. By August, the company had already gone past 18 trillion tokens. This number did not even include tokens used through tools like Microsoft Copilot.
For developers, the first question should not just be “Which model should I use?” It should be “Which model is right for this particular task?”
A simple query does not always need a large model. Halani explained that when tasks become more complex, latency problems increase and speed drops. The type of model needed also becomes more advanced.
Choosing the right model for each workload, instead of using one big model for everything, can help control AI costs.
Understanding Demand in Agentic AI
Demand becomes more complicated when companies move to agentic AI. An AI system may create tokens not only from the user’s first request, but also through reasoning, feedback, tool calls, and repeated loops.
For CIOs and CTOs, Halani offered a simple way to think about token demand. Start with three numbers: the number of people using the system at the same time, the number of requests each person makes, and the average number of tokens used per request. Then add details about the type of work, how people use the system, and whether requests can be answered from a cache instead of creating new tokens.
“The more you can hit the cache instead of generating a new token, the more you can reduce cost, by at least one-fifth or one-eighth. That’s the standard metric in the market right now for how much cost saving you can do in your token economics,” he said.
Managing Supply Through Better Efficiency
On the supply side, Halani said teams need to improve three layers: model efficiency, system efficiency, and software efficiency.
“Not having knowledge of systems and hardware used to be acceptable on your interview profile. Those days are gone,” Halani said. “The new era has come.”
Developers will need to understand the systems behind their applications. This includes memory, networking, storage, and how work is shared across GPUs. For large models, the question is not only how many GPUs are used. It is also how those GPUs talk to each other and whether they should be in the same rack or spread across different racks.
This change could even reach everyday devices. Halani suggested that companies may start checking whether employee laptops can run smaller AI models locally. This would reduce the need to send every request to an API, the cloud, or a central AI system.
The Real Value of a Token
For Halani, the bigger opportunity is not only about cutting token costs.
“The future is not about how much you use tokens and make yourself more efficient,” he said. “The future is how much you are feeding the right data, training the right model, or giving a post-training to that model, and thereby making it more and more intelligent to give you a more intelligent token back.”
This approach can create what he called customer “stickiness.” The more an AI system understands a returning customer, the more valuable it becomes to that person. As a result, the customer may be willing to pay more for it.
“That’s where the token monetization piece will come into the picture and that will be the most profitable business that you will do,” he said.
Looking Ahead
As agentic AI and physical AI grow, the systems behind them will become more complex. For developers, the lines between software, models, hardware, and costs are starting to blur.
The message from Halani is clear. Managing tokens wisely is important today. But the real long-term value will come from making each token smarter through better data and better-trained models.