The Realtime API gave developers speech without a text relay

The conversation

On 1 October 2024, OpenAI introduced a public beta of the Realtime API, writing that "developers can now build fast speech-to-speech experiences into their applications." The announcement describes the API as supporting "natural speech-to-speech conversations" through "six preset voices," open "today" "to all paid developers." It draws a direct line to the model doing the work, GPT-4o, which OpenAI had introduced on 13 May 2024 as "a single new model" trained "end-to-end across text, vision, and audio" that could respond to audio "in as little as 232 milliseconds," averaging 320 milliseconds.

What the documents show

The October announcement states the mechanism plainly: "the Realtime API lets you create a persistent WebSocket connection to exchange messages with GPT-4o," replacing a pipeline that, in its own words, "often resulted in loss of emotion, emphasis and accents, plus noticeable latency" when transcription, text and speech models were stitched together separately. OpenAI's stated prices for the beta, $0.06 per minute of audio input and $0.24 per minute of audio output, are the vendor's own rates, not an independently audited cost. The same announcement records a same-month update: as of 17 October 2024, "audio inputs and outputs are now available in the Chat Completions API," a separate, non-realtime path OpenAI built alongside it.

The system boundary

The announcement is explicit that the Realtime API sat apart from interfaces already on file, neither the Assistants API's hosted-thread model nor a plain Chat Completions call, but a session-based connection a developer's own server or client opens and holds. As retrieved on 16 September 2026, OpenAI's current guide to the API describes two separate connection modes, WebRTC and WebSocket, a split that did not exist in that form at the October 2024 launch, when the announcement described only a WebSocket connection; the same current guide also pairs the API with the separately launched Agents SDK, a pairing the original announcement does not mention.

Where it fails

OpenAI's own announcement frames the API as a beta built on one company's model, so a builder evaluating it in 2024 was choosing a specific vendor's voice pipeline, preset voice set and session design, not a general standard; nothing in the cited documents describes interoperability with a different vendor's speech model. The announcement does not describe what happens to a call in progress if the connection drops, leaving that failure mode to the surrounding application rather than to the protocol itself.

• Does a voice product's documentation describe its session and reconnection behavior, or only its steady-state design?
• Is a stated latency or price figure the vendor's own beta number, and has current documentation updated it?
• What happens to a system built against an early transport, such as WebSocket-only, once a vendor adds an alternative?

The Realtime API's first documentation describes a narrower thing than "conversational AI" in the abstract: one company's model, reached one specific way, with a beta price list attached, and the layer above it has already been rebuilt once in the two years since.