Speaker Diarization

Detect speakers and understand who said what.

Pre-recorded

Speaker diarization is the process of detecting multiple speakers in an audio, and understanding which parts of the transcription each speaker said.

On multi-channel audio, diarization runs separately on each channel (see with multi-channel audio).

Enabling diarization

Diarization is enabled by sending the diarization parameter in the transcription request:

Pre-recorded

{
  "audio_url": "<your audio URL>",
  "diarization": true
}

Response

When diarization is enabled, each utterance will contain a speaker field, whose value is an index representing the speaker. Speakers will be assigned indexes by order of appearance (i.e. the 1st speaker will be speaker 0, the 2nd speaker 1, etc).

Pre-recorded

{
  "transcription": {
    "utterances": [
      {
        "words": [...],
        "text": "it says you are trained in technology.",
        "language": "en",
        "start": 0.7334100000000001,
        "end": 2.364,
        "confidence": 0.8914285714285715,
        "channel": 0,
        "speaker": 0,
      },
      ...
    ]
  }
}

Improving diarization accuracy

You can improve the accuracy of the diarization by providing the model with hints regarding the expected number or lower/upper bounds ofspeakers using the diarization_config.num_of_speakers, diarization_config.min_speakers and diarization_config.max_speakers parameters respectively.

Important: These parameters are hints, not hard constraints. The actual number of speakers detected by the model may not comply with the provided parameters.

Key	Type	Description
diarization_config.number_of_speakers	number	Guiding number of speakers - instructs the model to detect an exact number of speakers in the audio.
diarization_config.min_speakers	number	Instructs the model to detect no less than this number of speakers in the audio.
diarization_config.max_speakers	number	Causes the model to detect no more than this number of speakers in the audio.

With multi-channel audio

When diarization is enabled on multi-channel audio, diarization runs separately on each channel. Diarization parameters (number_of_speakers, min_speakers, max_speakers) apply per channel.

Speaker indexes are then normalized across channels so IDs stay unique in the result. For example, with two channels and two speakers on each, speakers are numbered from 0 to 3 (four speakers total).

Because diarization is run independently per channel, the same physical speaker appearing on two channels will receive two different speaker IDs.