Realtime API

Yunxin exposes one OpenAI-compatible event surface and adapts it to OpenAI Realtime or Google Gemini Live inside the provider layer. Provider credentials never leave the gateway.

1. Discover routable models

GET /v1/realtime/models

Every returned route has passed provider availability, credential, capability, tier, and billing checks.

2. Create a model-bound session

POST /v1/realtime/sessions
Content-Type: application/json

{
  "model": "public-model-id",
  "modalities": ["audio"],
  "voice": "alloy",
  "instructions": "Answer briefly",
  "target_language": "zh-TW"
}

target_language applies to Live Translate. Live Transcribe uses modalities: ["text"]. The response contains websocket_url and a short-lived, single-use client_secret.value. The ticket is signed for the exact model and, when applicable, the API-key identity that passed authorization.

3. Connect without putting the ticket in the URL

const session = await fetch("/v1/realtime/sessions", {
  method: "POST",
  credentials: "include",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ model: "public-model-id", modalities: ["audio"] }),
}).then((response) => response.json());

const root = new URL(location.origin);
root.protocol = root.protocol === "https:" ? "wss:" : "ws:";
const ws = new WebSocket(new URL(session.websocket_url, root), [
  "yunxin-realtime",
  `yunxin-auth.${session.client_secret.value}`,
]);

The ticket is transported as a WebSocket subprotocol so it does not appear in URLs or access logs. It is atomically consumed before the gateway opens the upstream provider socket.

Modes

Client events

// PCM16 mono audio, base64 encoded
ws.send(JSON.stringify({ type: "input_audio_buffer.append", audio: base64Pcm }));
ws.send(JSON.stringify({ type: "input_audio_buffer.commit" }));

// Text turn (dialog models)
ws.send(JSON.stringify({
  type: "conversation.item.create",
  item: { role: "user", content: [{ type: "input_text", text: "Hello" }] },
}));
ws.send(JSON.stringify({ type: "response.create" }));

Google Live automatically completes a text turn, so its provider adapter consumes the redundant response.create. Provider operations without an honest equivalent return an explicit unsupported_event error rather than pretending success.

Server events

Client events are limited to 1 MiB and decoded audio chunks to 256 KiB. Paid token-priced realtime routes remain hidden until their modality-specific charge is representable; free and explicit per-request routes can be exposed safely today.