兔子API 兔子API
中

use-case

Speech, transcription, and realtime audio guide

Direct answerTranscription, speech synthesis, and realtime voice are separate pipelines focused on recognition accuracy, voice rights, and low-latency duplex transport.

Updated · Reviewed

Three audio workloads

Transcription converts audio to text and depends on language, timestamps, diarization, and noise. Speech synthesis converts text to audio and depends on voice, emotion, format, and first-byte streaming. Realtime conversation combines capture, VAD, transcription, reasoning, and playback. Their APIs and latency goals differ.

Input and transport

Validate format, sample rate, duration, and size for offline files and prefer streaming upload. Realtime uses WebRTC, WebSocket, or provider protocols and must handle reconnects, backpressure, interruption, and session expiry. Never place raw audio in ordinary application logs.

API selection

Transcription, Speech, and Realtime use separate models or endpoints. This site provides Bearer authentication only for adapted paths; language, timestamps, voice cloning, and realtime tools remain model-specific.

Evaluation

Measure character or word error, terminology, and speakers for transcription; naturalness, intelligibility, similarity, and first byte for TTS; end-to-end latency, barge-in, and echo for realtime. Include noise and weak networks.

Rights and privacy

Voice can be sensitive biometric data. Obtain demonstrable consent before cloning, label synthetic content, and verify retention, deletion, geography, minors, and prohibited-use rules.

Use cases

  • Text-to-speech
  • Transcription
  • Realtime voice

API protocols

  • /v1/audio/speech
  • /v1/audio/transcriptions
  • /v1/audio/translations

FAQ

Can transcription, speech synthesis, and realtime conversation share one endpoint?

Usually not. Transcription is audio to text, TTS is text to audio, and realtime conversation also needs VAD, duplex transport, and interruption. Verify models, endpoints, upload formats, and WebRTC or WebSocket protocols separately.

How can realtime voice latency and stutter be reduced?

Use supported streaming transport, control chunk and sample formats, and implement backpressure, reconnects, session expiry, and barge-in. Measure first-byte and end-to-end latency separately and avoid buffering an entire large file on the request path.

Which metrics should evaluate an audio model?

For transcription, measure character or word error, terminology, and speakers; for TTS, naturalness, intelligibility, similarity, and first byte; for realtime, end-to-end latency, barge-in, and echo. Include noise and weak networks.

May user recordings be logged or used for voice cloning automatically?

No. Raw audio and voice features may be sensitive data, so minimize collection, encrypt storage, support deletion, and keep only redacted metadata in normal logs. Obtain demonstrable consent before cloning and label synthetic output.

  • OpenAIOpenAI recommends Responses API for new projects. Direct and gateway calls both use a Bearer header, but require separate OpenAI and site keys, and the gateway endpoint must be implemented for the selected model.
  • MiniMaxThe mainland OpenAI-compatible base is https://api.minimaxi.com/v1 and Anthropic compatibility uses /anthropic. Pay-as-you-go and Token Plan have separate keys, while this gateway uses its own token and adapted route.
  • Mistral AIA first direct Mistral call uses the Mistral base URL, Authorization: Bearer, and an exact model ID. This site exposes only the adapted /v1/chat/completions route and requires this site's base URL and key. Batch, Guardrails, and Moderation are official Mistral capabilities, not evidence that this route automatically supports or enables them.
  • iFLYTEK SparkSpark's general HTTP API, Spark-X1.5, and Spark-X2 use different base URLs. Direct calls use the APIPassword for that service, while this gateway uses its own Bearer token and adapted path.

Official sources

  1. OpenAI Audio Guide Official