Full-Duplex Voice AI, Explained: Inside GPT-Live-1

Ask someone a question and the reply lands about 200 milliseconds after you stop. That peak holds across a worldwide sample of ten languages, according to a 2009 PNAS study on turn-taking. It is faster than anyone can plan a sentence, which tells you people start building their answer while you are still talking.

Voice assistants have never worked that way. They wait for you to finish, then think, then speak. On September 10 OpenAI shipped GPT-Live-1 in the API, and the notable part is not that it got faster. It is that it stopped waiting.

Why turn-based voice always felt off

The standard voice stack is a relay race. Microphone audio goes to a voice activity detector, the detector decides you are done, a speech-to-text model transcribes, an LLM reads the transcript and writes a reply, a text-to-speech model says it out loud.

Every leg costs time. The one that ruins the feel is the first, because deciding when a person stopped talking is called endpointing, and endpointing is a guess. Here is the guess, written out, from OpenAI's own Realtime VAD docs:

{
  "turn_detection": {
    "type": "server_vad",
    "threshold": 0.5,
    "prefix_padding_ms": 300,
    "silence_duration_ms": 500
  }
}

Five hundred milliseconds of silence before the pipeline even starts thinking. Drop it to 200 and the assistant talks over you every time you pause to find a word. Push it to 800 and every exchange gets a hole in it. No value is correct, because silence never means the same thing twice. A breath, a thought, a bad connection, and the end of a sentence all look identical to a detector that only measures quiet.

OpenAI's patch for this was semantic_vad, a classifier that scores whether your words sound finished and stretches the timeout when you trail off into "uhhm". It helps. It is still a guess, just a better-read one.

Add it up and the old path takes 1.41 seconds to take its turn, measured on the Full-Duplex-Bench suite we come back to at the end. People take 0.2.

Half-duplex, full-duplex

The words are borrowed from radio. A walkie-talkie is half-duplex: one side transmits, the other listens, and you say "over" because the channel cannot physically carry both directions at once. A telephone is full-duplex. Both ends send and receive at the same moment, which is why you can cut someone off mid-word, and why you both say "sorry, go ahead" in unison.

Every voice assistant until about a year ago was a walkie-talkie wearing a telephone costume.

A full-duplex model keeps the input stream and the output stream open together. It is still listening while it speaks. Three things fall out of that, and they are the three things that made older assistants feel robotic:

• It can say "mhmm" or "got it" while you are still mid-sentence, so you know it is there.
• It stops mid-word when you barge in, instead of finishing its paragraph into the void.
• Silence stops being a signal. You can pause to think without the assistant assuming you handed it the floor.

On the numbers OpenAI published at launch, GPT-Live-1 posts 0.798 seconds of turn-taking latency against 1.41 for gpt-realtime-2.1, and 80.1% on interactivity against 45.4%. Still four times slower than a person. Close enough that the rhythm survives.

In your own code the change is mostly a deletion. The migration guide says to stream audio continuously and let GPT-Live decide when to speak. Manual audio commits and voice-turn triggers go away. So do the terminal events:

There is no response.done any more. Nothing marks the end of a spoken response, because in a real conversation nothing does. If you built a state machine that waits for "the assistant finished", it has nothing left to wait for.

The voice model is not the brain

Here is the part worth sitting with. GPT-Live-1 is not smart, and that is deliberate.

Look at what the model page admits to. Knowledge cutoff of July 31, 2025. No structured outputs. No fine-tuning. No image or video input. Exactly one endpoint, v1/live/sessions, with Chat Completions, Responses and Realtime all unsupported. OpenAI's own summary is that it "can listen and speak at the same time, and delegate reasoning and tool use to a backend agent."

So the thing holding your conversation is a mouth and a pair of ears. The thinking happens elsewhere, in a model you choose: GPT-6 Astra, Codex, ChatGPT Work, or something of your own.

Two ways to wire the delegation. Responses delegation hands it to OpenAI: GPT-Live calls the Responses model you configured, passes the conversation context, and takes the result back. Client delegation keeps your application in charge of what runs, what context it sees, and which results ever reach the voice layer.

Either way you now write two prompts instead of one, and keeping them separate is the whole skill:

• session.instructions governs how the agent talks. Personality, pace, how chatty its backchannels are, what it does when interrupted.
• delegation.responses.instructions holds the business rules. Tool definitions move to delegation.responses.tools.

The prompting guide is unusually prescriptive about the voice half. It wants three labelled sections, with concrete conditions rather than vague ones, and one rule above the rest: delegate before giving an answer that depends on backend work, and do not guess the result while waiting.

You are Maya, a calm voice assistant for a dental clinic.
Speak warmly and at an unhurried pace. Stop talking the
moment the caller starts.
 
Backend tools
- Appointments: check open times, book, move, or cancel.
 
Delegate when
- The caller asks what times are free.
- The caller wants to book, move, or cancel anything.
 
Don't delegate when
- The caller greets you or thanks you.
- You only need to confirm a name or a date you just heard.

Splitting the mouth from the brain buys two real things. You can swap the reasoning model without anyone noticing a change in the voice, and the fast path stays fast, because saying "let me check that for you" never has to queue behind a reasoning model.

It also splits your bill. The $0.05 per minute, billed per second, covers the voice layer alone. Backend model and tool usage is charged separately. A ten-minute call is fifty cents of mouth plus whatever the brain ran up, and how much that is depends entirely on how often your prompt says to delegate. Latency and cost now both live in a policy you write in English, which is a strange place for an SLO to live. If you want the wider context on what inference actually costs in 2026, we covered the LLM price war earlier this year.

What you give up

The rest of the trade-offs are smaller but worth knowing before you commit:

• No terminal response event, so "is it done speaking" is no longer a question the API answers.
• No structured outputs and no fine-tuning on the voice layer. Anything schema-shaped belongs to the backend.
• Concurrent sessions are capped by tier: 25 at Tier 1, rising to 500 at Tier 5. A phone line that goes viral hits that wall before it hits a rate limit you are used to thinking about.
• One endpoint. If some part of your app wanted to reuse this model for plain text, it cannot.

About that "+30 points"

The headline number is a 30 percentage point gain over gpt-realtime-2.1 on Full-Duplex-Bench. That benchmark is real and independent, not a vendor chart. It comes from a 2025 paper by Guan-Ting Lin and co-authors, and it scores four behaviours: pause handling, backchanneling, turn-taking, and interruption management.

What it does not settle is what good behaviour even looks like. The v1.5 follow-up ran five state-of-the-art agents through overlapping speech and found them split into two camps. Some are "responsive", reacting fast to anything they hear. Others are "floor-holding", filtering overlap out and carrying on. Both are defensible. Which you want depends on whether your agent is taking a drive-through order in a noisy car or sitting in on a medical intake, and a single composite score will never tell you that.

So what changes on Monday

If you are running a cascaded voice stack, the machinery you built to fake naturalness is now dead weight. Endpointing heuristics, barge-in handling, filler-word injection, the state machine tracking whose turn it is, all of it moves inside the model. One team quoted in the launch coverage cut roughly 80% of their voice codebase. Treat that as their number, not a promise.

What replaces it is a design problem. You are picking a brain, writing a delegation policy, and deciding how much silence your agent is allowed to fill while it waits on that brain. A fast mouth wired to a slow backend still feels slow, and now the slowness is your architecture rather than the vendor's model.

Two places to go from here. If you are choosing and wiring the backend, Build a Simple AI Agent in Python walks the loop that sits behind the voice. And before you let a voice agent call real tools for a stranger on a phone line, read up on prompt injection, because none of it gets safer when the untrusted input arrives as sound.