# Speech, transcription, and realtime audio guide

Transcription, speech synthesis, and realtime voice are separate pipelines focused on recognition accuracy, voice rights, and low-latency duplex transport.

## Three audio workloads

Transcription converts audio to text and depends on language, timestamps, diarization, and noise. Speech synthesis converts text to audio and depends on voice, emotion, format, and first-byte streaming. Realtime conversation combines capture, VAD, transcription, reasoning, and playback. Their APIs and latency goals differ.

## Input and transport

Validate format, sample rate, duration, and size for offline files and prefer streaming upload. Realtime uses WebRTC, WebSocket, or provider protocols and must handle reconnects, backpressure, interruption, and session expiry. Never place raw audio in ordinary application logs.

## API selection

Transcription, Speech, and Realtime use separate models or endpoints. This site provides Bearer authentication only for adapted paths; language, timestamps, voice cloning, and realtime tools remain model-specific.

## Evaluation

Measure character or word error, terminology, and speakers for transcription; naturalness, intelligibility, similarity, and first byte for TTS; end-to-end latency, barge-in, and echo for realtime. Include noise and weak networks.

## Rights and privacy

Voice can be sensitive biometric data. Obtain demonstrable consent before cloning, label synthetic content, and verify retention, deletion, geography, minors, and prohibited-use rules.

## Use cases

- Text-to-speech
- Transcription
- Realtime voice

## API paths

- /v1/audio/speech
- /v1/audio/transcriptions
- /v1/audio/translations

## Related guides

- [OpenAI](https://api.tu-zi.com/en/docs/providers/openai): OpenAI recommends Responses API for new projects. Direct and gateway calls both use a Bearer header, but require separate OpenAI and site keys, and the gateway endpoint must be implemented for the selected model.
- [MiniMax](https://api.tu-zi.com/en/docs/providers/minimax): The mainland OpenAI-compatible base is `https://api.minimaxi.com/v1` and Anthropic compatibility uses `/anthropic`. Pay-as-you-go and Token Plan have separate keys, while this gateway uses its own token and adapted route.
- [Mistral AI](https://api.tu-zi.com/en/docs/providers/mistral): A first direct Mistral call uses the Mistral base URL, `Authorization: Bearer`, and an exact model ID. This site exposes only the adapted `/v1/chat/completions` route and requires this site's base URL and key. Batch, Guardrails, and Moderation are official Mistral capabilities, not evidence that this route automatically supports or enables them.
- [iFLYTEK Spark](https://api.tu-zi.com/en/docs/providers/iflytek-spark): Spark's general HTTP API, Spark-X1.5, and Spark-X2 use different base URLs. Direct calls use the APIPassword for that service, while this gateway uses its own Bearer token and adapted path.

## Frequently asked questions

### Can transcription, speech synthesis, and realtime conversation share one endpoint?

Usually not. Transcription is audio to text, TTS is text to audio, and realtime conversation also needs VAD, duplex transport, and interruption. Verify models, endpoints, upload formats, and WebRTC or WebSocket protocols separately.

### How can realtime voice latency and stutter be reduced?

Use supported streaming transport, control chunk and sample formats, and implement backpressure, reconnects, session expiry, and barge-in. Measure first-byte and end-to-end latency separately and avoid buffering an entire large file on the request path.

### Which metrics should evaluate an audio model?

For transcription, measure character or word error, terminology, and speakers; for TTS, naturalness, intelligibility, similarity, and first byte; for realtime, end-to-end latency, barge-in, and echo. Include noise and weak networks.

### May user recordings be logged or used for voice cloning automatically?

No. Raw audio and voice features may be sensitive data, so minimize collection, encrypt storage, support deletion, and keep only redacted metadata in normal logs. Obtain demonstrable consent before cloning and label synthetic output.

## Sources

- OpenAI Audio Guide: https://developers.openai.com/api/docs/guides/audio

## Documentation source

Source: https://api.tu-zi.com/en/docs/use-cases/audio

Reviewed: 2026-08-08

This document is an integration reference. Provider-native APIs may differ from this site's APIs; use the corresponding API Reference and model marketplace to confirm the site URL, authentication, model ID, group and parameters. This document does not certify live success for every model or upstream service. It contains documentation and examples, without the current user's credentials or request history.