Realtime API
Low-latency, bidirectional voice and text over a WebSocket using the OpenAI Realtime protocol.
The Realtime API is a WebSocket that speaks the OpenAI Realtime protocol. Yunxin proxies the connection to a realtime-capable provider (OpenAI or Gemini Live), so existing OpenAI Realtime clients work against Yunxin by changing only the URL.
Connect
wss://api.yuhuanstudio.com/v1/realtime/ws
Pass the model and an auth token as query parameters:
wss://api.yuhuanstudio.com/v1/realtime/ws?model=model-id&token=YOUR_TOKEN| Query parameter | Required | Description |
|---|---|---|
model | Yes | A realtime-capable model id (see Realtime models). |
token | Yes | Your auth token, supplied as a query parameter on the WebSocket URL. |
const ws = new WebSocket(
`wss://api.yuhuanstudio.com/v1/realtime/ws?model=model-id&token=${TOKEN}`
);The token travels in the URL. Use a short-lived, scoped token rather than a long-lived secret, and never embed it in shipped client code.
Event flow
The session follows the standard OpenAI Realtime lifecycle, exchanged as JSON events:
- Session — on connect, the server emits
session.created. Sendsession.updateto configure modalities (text,audio), the voice, and audio formats. - Audio buffer — append microphone audio with
input_audio_buffer.append, theninput_audio_buffer.commit(or rely on server-side voice-activity detection). The server signalsinput_audio_buffer.speech_started/speech_stoppedas it detects speech. - Response — request output with
response.create. The server streams it back asresponse.text.delta/response.audio.deltachunks and signals completion withresponse.done.
Common client events
| Event | Description |
|---|---|
session.update | Configure session parameters (modalities, voice, formats). |
input_audio_buffer.append | Append a base64 audio chunk. |
input_audio_buffer.commit | Finalize the buffered audio as an input. |
input_audio_buffer.clear | Discard buffered audio. |
response.create | Request a model response. |
response.cancel | Cancel an in-progress response. |
Common server events
| Event | Description |
|---|---|
session.created / session.updated | Session established / reconfigured. |
input_audio_buffer.committed | The buffer was accepted as input. |
input_audio_buffer.speech_started / speech_stopped | Voice activity detected / ended. |
response.text.delta / response.text.done | Streaming text output. |
response.audio.delta / response.audio.done | Streaming audio output. |
response.done | The response is complete. |
error | An error occurred (carries an error object). |
These are standard OpenAI Realtime events. For the exact payload of any individual event, follow the OpenAI Realtime protocol — Yunxin proxies it faithfully.
Example: voice chat
const ws = new WebSocket(
`wss://api.yuhuanstudio.com/v1/realtime/ws?model=model-id&token=${TOKEN}`
);
ws.onopen = () => {
ws.send(JSON.stringify({
type: "session.update",
session: {
modalities: ["text", "audio"],
voice: "alloy",
},
}));
};
ws.onmessage = (event) => {
const message = JSON.parse(event.data);
switch (message.type) {
case "response.text.delta":
process.stdout.write(message.delta);
break;
case "response.audio.delta":
playAudio(message.delta); // base64 audio chunk
break;
case "response.done":
console.log("\n--- response complete ---");
break;
case "error":
console.error(message.error);
break;
}
};Realtime models
GET /v1/realtime/modelsLists the models that support realtime sessions. You can also filter the main catalog with the
Models API: GET /v1/models?type=realtime.
Realtime availability depends on the configured upstream provider (OpenAI or Gemini Live). If no realtime-capable provider is available, the connection is closed with an error event.
How is this guide?