In telephony everything hinges on the fact that audio is not text. Until a call is transcribed no language model can work with it, and until the transcript says who is speaking you cannot assess an agent at all. So a working setup is built from parts: speech recognition, speaker separation, meaning extraction and a voice reply. Each part can then be swapped and measured on its own, and a failure in one place does not turn the whole thing into a black box.
Speech recognition turns a recording or a live stream into text. Everything downstream depends on this step: if the model garbles product names and surnames, the analysis is built on fiction. Pay attention to phone-grade audio, background noise and how the model handles your language mixed with Latin-script brand names. Without this part you are back to listening to a sample of calls by hand.
Speech recognition in 99 languages, including Russian. The de facto standard for transcribing calls and meetings. Hugging Face's faster Distil-Whisper is English only.
Sber's models for Russian speech recognition, among the most accurate for Russian. Includes emotion recognition, v3 with punctuation, and a multilingual version (Russian, Kazakh, Kyrgyz, Uzbek).
Fast NVIDIA speech recognition models, including streaming ones for real-time use. Parakeet TDT v3 and Nemotron 3.5 ASR understand Russian.
Speaker separation labels which line belongs to the agent and which to the customer, and tells speech apart from pauses and noise. Without it a transcript is one undivided block: you cannot measure who talked more, or check whether the required script was read out. In a live conversation the same part decides when the person has finished and it is safe to reply.
The most widely used open tool for splitting a recording by speaker: who spoke and when. Usually paired with speech recognition. Weights are issued after a short form on HF.
NVIDIA models for "who is speaking": TitaNet recognizes a specific person's voice, Sortformer splits a recording into up to 4 speakers, including live during a call.
The most popular open speech detector: tells voice apart from silence and noise. Processes an audio chunk in under a millisecond on a single CPU core; trained on recordings in more than 6,000 languages.
The language model works with text by now: it pulls out the reason for the call, the topic, what was agreed and the next step, and in a live scenario writes the reply itself. Script compliance checks and flagging difficult calls for a supervisor also sit here. Drop this part and you are left with transcripts nobody reads, because nobody has time for hundreds of pages a day.
A family of language models with strong Russian language support, from small versions for a laptop to a flagship on par with commercial APIs.
The best-known Russian community fine-tune: open models (Llama, Mistral, Gemma, YandexGPT) trained to act as a Russian-speaking assistant. A convenient starting point for a Russian chatbot on your own server.
The models that started mass open source in AI. A huge ecosystem of fine-tuned versions and tools.
Speech synthesis voices the finished text. What matters here is not only how clean the voice sounds but how fast it starts: on a phone line a pause longer than a couple of seconds reads as a dropped call. Decide separately on intonation and on whether you tell the caller they are talking to a machine. Without this part you are limited to chat and notifications.
A popular model for cloning a voice from a short sample in 17 languages, including Russian. Coqui has shut down and development has stopped.
Very fast speech synthesis that runs even on a Raspberry Pi. Ready-made voices in 35+ languages, including several Russian ones.
Lightweight Russian speech synthesis that runs on a regular CPU. Version v5 added CIS languages and languages of Russia's peoples: Tatar, Bashkir, Yakut, Kazakh and others.
The main trap is expecting the models to read calls as well as an experienced human does. Start with analysing calls that already happened rather than answering customers live: a mistake in analytics is cheap, a mistake in conversation costs you the customer. Test the chain on your own phone lines, not on studio recordings. Call recording, notifying the other party and storing voice data are regulated; this is a general description only, a lawyer should review your case.
An open model can run on your own server: data stays in-house, there is no per-request fee, and the model can be fine-tuned on your documents.
