The cost of IT needed to fuel the current AI revolution can only be described as staggering. Funding for the major AI supercomputers—those powering companies like OpenAI and Anthropic—has been doubling roughly every year since they were first built. The GPT-4 cluster alone cost hundreds of millions of dollars, while xAI’s Colossus supercomputer clocked in at $10 billion. And we’re still in the early innings, with ever-increasing demand for AI fueling more infrastructure investment. For those of us who spend a lot of time thinking about where durable value gets created in technology, this trajectory is hard to ignore—and even harder to overstate.
With AI, we’re not just watching a new software category emerge. We’re watching something more like an industrial buildout. And it’s fueling our conviction that much of the value to be created with AI will lie not just in software, but in multiple layers of a new, emerging AI-infrastructure stack.
AI is a fundamentally different kind of workload
Traditional software is code, plus CPUs, plus storage. It’s relatively lightweight, fast to deploy, and easy to swap out. AI systems are something else entirely: data, accelerators, power, networking, and orchestration all working in concert. A useful analogy: Training an AI model is like building a factory. Running inference is like operating that factory at scale, around the clock, under pressure.
That shift from software workload to industrial workload has enormous implications for where value gets created in AI and which companies capture it. In every prior computing wave, from client-server to the Internet and cloud, the infrastructure layer ended up being where some persistent value accumulated. Not always the flashiest layer. Not always the easiest to demo. But the one that everything else depended on.
AI is following the same pattern, but its infrastructure layer is being built more quickly and with greater capital intensity than anything that came before.
Why infrastructure wins over time
There are three dynamics that consistently favor infrastructure players over application-layer bets.
Cost curves beat feature cycles: The companies that can relentlessly drive down cost per unit of output tend to win over time, regardless of who has the most impressive product at launch.
Reliability beats novelty: When a workload becomes mission-critical, consistent uptime and predictable latency matter more than new capabilities.
And defaults beat differentiation: Once a technology becomes the thing developers reach for first, switching costs accumulate quietly and compound into a real moat.
Infrastructure is where persistence lives.
The scale moment reveals everything
Everyone looks good before they start scaling. You can build a reasonably functional AI application with off-the-shelf infrastructure, and it will work fine—until it doesn’t. The scale moment is when GPU bills spike, when latency starts affecting conversion rates, when an outage hits revenue. That’s when the architecture decisions you made six months ago either hold up or fall apart.
This is also why the best infrastructure companies are so hard to appreciate early. Their value is invisible in a demo. Lower inference cost, higher utilization and uptime, predictable latency under load—none of these generate applause in a pitch meeting. But they show up dramatically in the P&L once you’re operating at scale.
Baseten*, for example, delivers AI Whisper speech/recognition transcription more than 10x faster than the OpenAI baseline. That kind of performance edge doesn’t look exciting on a slide. But it looks like survival when your inference costs are eating your margins.
The stack we’re watching
The AI infrastructure stack spans five interconnected layers:
- Compute — the GPUs, accelerators, and custom silicon doing the heavy lifting.
- Data and storage — how data moves and gets structured for training and inference.
- Networking and systems — the latency and bandwidth that determines how efficiently everything communicates.
- Orchestration — deployment, scheduling, cost control, and security.
- And finally the models themselves, where the algorithmic value ultimately lives.
Each layer has emerging leaders. The interesting question, the one we spend most of our time on, is which of those leaders will prove genuinely irreplaceable. Our framework is simple: Does this company relieve a real bottleneck, meaningfully lower the cost per unit of intelligence, or become so deeply embedded that ripping it out isn’t worth contemplating? Thin wrappers and undifferentiated middleware will leak value to the layers above and below them. The companies that anchor to a real constraint in the stack will accumulate it.
The next frontier: Beyond terrestrial constraints
The constraints are no longer purely computational. Power availability, cooling capacity, network latency, and data sovereignty are all becoming genuine bottlenecks as AI infrastructure scales. We’re starting to see the limits of what traditional data centers can provide, and new infrastructure paradigms, in how we build, cool, connect, and locate compute, are emerging to reshape those cost and resilience curves.
Our thesis
We believe the next decade of AI will be shaped less by which application wins than by which infrastructure layers become indispensable. The companies worth backing are the ones building systems that matter more at scale than they do in a demo–systems where early architectural decisions compound into durable technical moats, longer customer relationships, and the kind of power-law outcomes that infrastructure businesses have always produced when they get the timing right.
The industrial buildout is underway. We’re investing in the stack that makes it run.
The post AI Infrastructure: The New Industrial Stack appeared first on Battery Ventures.