Much of the attention around artificial intelligence has focused on building larger and more capable models.
Training those systems requires substantial computing resources. It also produces the demonstrations that capture headlines.
But once a model has been trained, it must be put to work. Every time someone asks a question, analyzes a document or uses an AI feature in an application, the system performs another calculation.
That activity is called inference. And as AI becomes part of more products and workflows, running models could become a business as important as building them.
Training and Inference Are Different Jobs
Training is the process of adjusting a model using data so it can learn patterns and produce useful outputs. It can involve intensive work across many processors over a concentrated period.
Inference happens when the trained model is used. A single request may require less computing than an entire training run, but an application can receive many requests, repeatedly, from users around the world.
The distinction matters because the infrastructure serving those requests must meet different needs. A research team training a model may prioritize access to a large cluster. An application serving customers may care just as much about response time, availability and the cost of each request.
The Economics of Repeated Use
Consider a company that adds an AI assistant to a customer service product. The model may have been trained before the company chose it. Yet the cost of answering requests continues as customers use the service.
If use grows, those recurring costs can grow too. The provider must decide what to charge, which tasks justify a more capable model and how to maintain a useful experience without spending more than the service earns.
There is no fixed relationship between training and inference spending. It depends on model design, usage, pricing and how efficiently the work is performed. Improvements in efficiency can reduce the cost of an individual request even as the number of requests increases.
That makes the business question more complex than simply counting processors.
Speed, Reliability and Location
Running AI is not only a matter of making a model available. People expect applications to respond quickly and consistently. A delay that is acceptable for a lengthy report may be frustrating in a conversation or an interactive tool.
Providers must also plan for periods of high demand, equipment failures and security. Some tasks may be handled in a central data center, while others may benefit from processing closer to the user or on a device. Each choice involves tradeoffs in cost, performance and control.
Software matters as much as hardware. Scheduling requests, choosing an appropriate model and making efficient use of memory can all affect the service a customer receives.
A Market With Room for Different Approaches
Some organizations may train their own models and operate the infrastructure that serves them. Others may use models and computing services supplied by different companies. Still others may specialize in helping customers deploy, monitor or improve applications.
Those businesses will not necessarily compete in the same way. A provider serving an occasional research workload has a different customer relationship from one supporting a business application every hour of the day.
Over time, customers may judge AI services less by the size of the model and more by whether they solve a problem reliably at a reasonable cost.
The Question Is Useful Work
Training will remain essential. Without it, there is no model to run. Continued research may also change the kinds of systems that are deployed and the resources they need.
Whether inference ultimately becomes the larger business is impossible to know in advance. What is clear is that building a model and delivering useful work from it are separate challenges.
The growth of AI will depend on both. The companies that understand how to turn computing into a dependable service may be just as important as those that train the models themselves.