We May Be Laying Fibre Again
Voice AI may be entering its own infrastructure-building period. The harder question is which parts of the stack will matter once conversation with software becomes ordinary.
In the late 1990s, telecom companies laid an extraordinary amount of fibre around the world. The logic for doing so was fairly straightforward: the internet was growing quickly, data consumption was going to increase, and somebody needed to build the infrastructure that would carry all that traffic. Billions of dollars followed that thesis into new networks, submarine cables and fibre running underneath cities. Companies such as Global Crossing raised enormous amounts of capital to build for an internet that everyone expected to be much larger than the one that existed at the time.
The internet did become much larger. The problem was that it didn’t happen quickly enough for many of the companies building for it. When the dot-com bubble collapsed, the industry was left with far more capacity than it could use. Bandwidth prices fell, telecom companies struggled under the debt they had accumulated during the build-out, and some of the most ambitious builders went bankrupt. Global Crossing filed for bankruptcy in 2002, having spent billions building a network for demand that had not yet arrived at the scale its economics required.
A lot of that fibre was described at the time as “dark fibre”, strands that had been laid but were not yet carrying traffic. Over the following decade the description started to look almost funny. Broadband expanded, Google and YouTube grew, Netflix started streaming video, smartphones arrived, cloud computing took off, and the amount of data travelling across the internet eventually became difficult to comprehend.
I’ve been thinking about that period quite a bit recently because I wonder if there is something in it that helps explain where we are with Voice AI.
Infrastructure appears before the use case is fully visible.
Many useful layers are being built at once.
Fibre was laid before there was enough traffic to justify the cost of building it.
The eventual demand came from products and behaviour that were difficult to predict at the time.
Voice AI has many layers today. It is too early to know which will remain distinct businesses.
I spend a lot of my time around Voice AI companies in London and, through our own work, increasingly around the less glamorous problem of getting these systems to work in production. The industry is still early enough that I don’t think anyone can say with much confidence what its eventual shape will be. There are companies building voice models, transcription, inference, orchestration, telephony, agent platforms and various combinations of all of them. There is a lot of capital going into the space and a lot of excitement about what happens when talking to software becomes a normal way of using it.
What is much clearer to me now than it was even a year ago is that Voice AI is genuinely useful. For a long time, machine speech was mostly useful in narrower ways. Transcription got very good. IVRs could route calls. Siri and Alexa could respond to commands. There were applications in accessibility, dictation and customer service. But the frontier now is increasingly conversational, and that turns out to be a much bigger engineering problem than simply getting a machine to recognise speech or generate a convincing voice.
The fibre builders of the 1990s correctly understood where the world was going. Many still went bankrupt getting there. Voice AI may be entering its own infrastructure-building period. The question isn’t whether people will talk to machines. Increasingly, they already do. The question is which pieces of today’s rapidly expanding stack will still matter once talking to software becomes ordinary.
The difficult part is the conversation
One of the things that becomes obvious when you work with Voice AI is how many different things have to happen for what feels to the user like a very simple interaction. Speech-to-text and text-to-speech are important, but neither gives you a conversation. You also have to deal with turns: when someone has finished speaking, when the agent should respond, what happens when the person pauses halfway through a sentence, how quickly the agent should answer, and what the system should do when someone interrupts it halfway through a response.
Turns sound like a small problem until you get them wrong. If an agent waits too long after every sentence, the conversation feels sluggish. If it responds too quickly, it constantly interrupts people. A pause can mean that I’ve finished talking, but it can just as easily mean that I’m thinking about what to say next. This is one of the reasons latency matters so much in Voice AI. The system isn’t only being judged on whether it eventually produces the correct answer. It is being judged on whether the entire exchange feels responsive enough to sustain a conversation.
The architecture underneath that conversation is changing as well. OpenAI’s recent work on realtime voice, for example, separates some of the immediate conversational work from deeper reasoning and tool use, allowing the voice interaction to continue while other work happens asynchronously. That distinction becomes important once an agent has to do something rather than merely answer a question. If I call my bank to ask about a transfer, the person I’m speaking with can acknowledge me, ask another question or continue talking while checking another system. A useful Voice AI agent needs some version of the same ability. It has to maintain the conversation while reasoning, retrieving information or performing an action somewhere else.
This is where my own understanding of Voice AI has changed. We have been involved in enterprise deployments, including work with a large financial institution serving millions of customers and, more recently, a deployment around public-sector revenue collection involving transport offences. These are useful environments for understanding the technology because the agent isn’t simply answering general questions. It has to know why it is speaking to someone, understand the response, retrieve the right information, deal with disagreements or unexpected questions, and in some cases perform an action against a system of record.
A call like that can pass through a surprising number of systems. If it is a traditional phone call, there is the carrier and telephony infrastructure required to connect it in the first place. The audio has to be streamed into the AI system, the speech interpreted, turns managed and context maintained. A reasoning model may need to decide what the person means and what should happen next. The agent may then have to query a customer record, check a payment, retrieve an offence, update a CRM or trigger another workflow before generating a response, converting that response back into speech and sending it through the telephone network. The customer doesn’t care about any of this. They care that when they finish speaking, the response comes back quickly enough and makes enough sense for the conversation to continue.
01Everything starts with imperfect human speech: accents, interruptions, pauses and changes of mind.
02Audio becomes a stream the system can interpret, while it continues listening for the next turn.
03A pause is ambiguous. The agent has to distinguish “I’m finished” from “I’m thinking”.
04The agent retrieves context, decides what to do and may query other systems while keeping the conversation moving.
05The response is rendered and delivered quickly enough that the exchange still feels like a conversation.
This is why I’m increasingly sceptical of looking at Voice AI primarily as a competition between voice models. Models are obviously important, and improvements in them are responsible for much of what is now possible, but production Voice AI is beginning to look more like a systems problem. A very good transcription model cannot compensate for terrible latency. A realistic voice cannot compensate for bad turns. A powerful reasoning model is limited if it cannot retrieve information from the system where the customer’s data lives. An agent that works beautifully over the web may encounter a completely different set of constraints when you try to make thousands of calls over traditional telephony.
The interesting part is that telephony is not merely a historical comparison for what is happening in Voice AI. It is still part of the infrastructure. A frontier model running on modern GPUs can reason about a customer’s request, use another model to generate speech, interact with a bank through APIs, and then deliver the result through telephone infrastructure whose origins predate the internet. We tend to describe technological eras as though one replaces another, when in practice they often accumulate. The internet was built partly on top of telecommunications infrastructure, cloud computing was built on top of the internet, modern AI depends heavily on the cloud, and Voice AI is now sending some of that intelligence back through the telephone network.
That relationship matters because distribution is one of the reasons voice is interesting. Businesses don’t need to convince billions of people to adopt a new communication device before an AI agent can reach them. The phone is already there.
The economics get harder at scale
Working on enterprise deployments also forces you to think differently about compute. On-premise deployment is attractive to banks, governments and other organisations that are sensitive about where their data goes, and it is technically possible to run increasingly capable models inside their own environments. The harder question for realtime Voice AI is how to make the economics work when demand is uneven.
Cloud infrastructure gives you some elasticity. If a workload suddenly requires more GPU capacity, you can provision more of it and pay for the period in which you need it. On-premise, you need to have enough physical capacity available for the peak workload you are promising to support, including during the hours when much of that capacity might be sitting idle. This becomes particularly important with voice because the relevant number isn’t simply how many calls an agent makes in a day, but how many conversations it has to support concurrently. Ten thousand calls spread throughout a day and ten thousand simultaneous calls may look similar in a usage report, but they require very different infrastructure.
Elastic workload. Capacity can follow demand because calls arrive over time.
Peak workload. Infrastructure has to be ready before the busiest minute begins.
This is one reason I’m not yet convinced that on-premise Voice AI has reached its natural economic form. That isn’t the same as saying it doesn’t work. We know it can work. But the calculation depends on the price of the hardware, the size of the models, the concurrency you need to support and how often you can reuse the capacity you have bought. Those variables are changing quickly. Models are becoming smaller and more efficient, inference is getting cheaper, and hardware will continue to improve, so the answer three years from now may be very different from the answer today.
The pricing model for the software itself has a similar problem. The SaaS industry spent years getting comfortable with seats because the product was generally something an employee used to do their job. Voice agents blur that relationship because the software is increasingly doing part of the job itself. Charging by the minute is an obvious alternative and several Voice AI companies do this because minutes map reasonably well to the underlying cost of inference, speech and telephony. Retell, for example, currently exposes much of its pricing this way. The problem is that minutes are a measure of consumption rather than value.
Consider two calls. One agent spends eight minutes speaking to a customer and achieves nothing. Another spends three minutes speaking to someone and recovers a payment. The first conversation consumed more infrastructure, while the second created more value. This is part of the argument for outcome-based pricing, which companies such as Sierra have pushed more aggressively: instead of paying for the amount of software consumed, the customer pays when the agent completes something valuable.
In practice, I think this will be messier than either model suggests. A minute has a reasonably identifiable cost, while an outcome requires agreement about what counts as success and who deserves attribution for it. If an AI agent reminds someone about an outstanding payment and the person pays three days later through another channel, did the agent create the outcome? What happens when the agent does useful work that doesn’t end in the specific event being measured? These aren’t theoretical questions for us. We’re already having versions of them in enterprise deployments, and I don’t think the market has settled on an answer. We are still working through it ourselves.
There is a connection here to the fibre era that I find useful. Infrastructure can become strategically important while simultaneously becoming cheaper to consume. The enormous increase in demand for bandwidth did not guarantee that every company selling bandwidth would have attractive economics. Voice AI may create something similar in parts of its stack. Speech generation can become ubiquitous while the price of generating speech collapses. Realtime inference can become essential while competition drives down its margins. A technology becoming more important does not necessarily mean the companies providing each component become more valuable.
Production has a way of ruining a good demo
There is another part of the Voice AI stack that is harder to model because it isn’t software at all: the person on the other end of the call. Real people don’t behave like evaluation datasets. They interrupt agents, mumble, speak over them, change the subject, become angry, deny information that is sitting in the system of record, ask questions nobody anticipated and occasionally behave in ways that make perfect sense to another human but are difficult to encode into a test.
We’ve been paying a lot of attention to telemetry from our deployments for this reason. Once the basic system works, the interesting questions become increasingly granular. How long can the agent wait before someone assumes the call has dropped? How quickly should it respond without constantly cutting people off? How does that change with different accents or speaking patterns? When should the agent give up on a line of questioning? When does a disagreement require a human rather than another attempt by the model? What happens to the completion rate when you change the speed of the voice or the way it handles a turn?
Some of these problems will improve naturally as models improve, but I’m not convinced all of them reduce to model intelligence. A system can understand exactly what someone has said and still respond at the wrong moment. It can reason correctly and use the wrong tone. It can successfully complete a task while leaving the person on the other end frustrated with the interaction. The more Voice AI becomes an interface rather than a novelty, the more these details matter because the benchmark stops being another AI system and becomes the ordinary phone conversations people have had all their lives.
This is also where the industry still has a lot to learn from actual deployments. There are plenty of Voice AI demos, benchmarks and funding announcements, but comparatively little public discussion about what happens when these systems meet real telephone networks, enterprise systems and thousands of people who don’t know or care how the underlying technology works. Part of that is understandable. The industry is young, companies are competing intensely and everyone is playing their cards close to their chest. But it means we are documenting surprisingly little of what is being learned during what may turn out to be an important infrastructure build-out.
What are we actually laying?
The obvious temptation is to look at all the companies being created around Voice AI and try to decide which layer will win. I’m not sure that’s possible yet. Speech-to-text, text-to-speech, speech-to-speech models, realtime inference, telephony, orchestration, observability, agent platforms and vertical applications are all solving real problems today. The more difficult question is which of those problems will remain sufficiently difficult and differentiated to support standalone businesses once the technology matures.
Some parts of today’s architecture may simply disappear into the models. If native audio models become sufficiently fast and reliable, the current separation between transcription, reasoning and speech generation could become much less important. Orchestration may become easier as models improve at tool use. The cost of generating a minute of realistic speech may fall to the point where voice itself is no longer much of a differentiator. None of this would make Voice AI less important. It could make it far more important while making individual parts of today’s stack less valuable.
Other layers may prove surprisingly durable. Telephony still has to connect the agent to people who use ordinary phones. Enterprise systems still have to expose the information and actions that make an agent useful. Reliability, observability and permissions become more important rather than less important when software starts performing consequential actions. There is also the possibility that the real advantage comes from controlling enough of these layers to optimise the system as a whole, particularly when milliseconds of latency, inference costs and failure rates start to matter at scale.
I don’t know which of these turns out to be right, and I think pretending otherwise would make the discussion less interesting. We are close enough to the technology to see the constraints changing almost month by month, but still early enough that many of the categories we use today may not survive.
This is why the telecom comparison has stayed with me. The fibre builders weren’t wrong that the internet would need enormous amounts of bandwidth. What they couldn’t know was the eventual shape of the demand they were building for. They were laying fibre before YouTube existed, before Netflix became a streaming company, before AWS existed, before TikTok and before anybody was building enormous GPU clusters to train models that could hold a conversation. They could see the direction of travel without being able to see what would eventually travel on the infrastructure.
Voice AI feels a little like that to me now. We can see enough to know that talking to software is becoming useful and that, in many cases, it is a more natural interface than asking people to navigate another dashboard or learn another piece of software. What is harder to know is which parts of the infrastructure we are assembling around that interaction will eventually matter.
The fibre builders of the 1990s correctly understood where the world was going. Many still went bankrupt getting there. Voice AI may be entering its own infrastructure-building period. The question isn’t whether people will talk to machines. Increasingly, they already do. The question is which pieces of today’s rapidly expanding stack will still matter once talking to software becomes ordinary.
That is the part I want to spend more time writing about. Not just what Voice AI can do, which is changing quickly enough that any list will become outdated, but what we are learning from trying to deploy it: where the costs actually appear, what breaks at scale, how people behave when the demo becomes a real phone call, and which assumptions we are being forced to change along the way. There isn’t yet a definitive account of where this market ends up, and I suspect anyone offering one today is mostly guessing. For now, being inside the build-out and documenting what we can actually see seems more useful.