In addition to speech recognition, speaker diarization plays a central role in speech processing, primarily addressing the question of “who said what and when.” This involves analyzing the vocal characteristics of different individuals, segmenting the original audio signal, and categorizing it based on each person’s unique features. Unlike speaker identification, which aims to recognize specific individuals, speaker diarization focuses on distinguishing different speakers without requiring prior information about them, making it applicable to a broader range of real-world situations.

Speaker diarization is applied to the analysis of various audio data, including media broadcasts, conference conversations, online content on social networks, and court hearings. It also contributes to improving the accuracy of speech recognition in multi-speaker environments.

Speech recognition and diarization tasks can be performed in (a) offline or (b) streaming (real-time) modes. In the former case, the input typically consists of a single speech recording, where the entire content can be utilized, including multiple processing passes, with no strict limitation on the required computation time. The latter case, which is more complex, has become crucial today as data exploration fields face an increasing challenge: the rapid analysis of varied data streams, ideally in real-time.

[…]