1 hour ago
22
Nvidia just made it a lot easier to figure out who’s talking. The company released Nemotron 3 Diarization, an open-weight model built specifically to solve one of the most annoying problems in audio processing: accurately tracking which speaker said what, and when they said it, in real time.
The model launched on September 23 under Nvidia’s OpenMDW 1.1 license, making it freely available on Hugging Face and through the company’s NeMo framework. It’s also accessible via inference providers like Baseten and DeepInfra, with deployment costs reportedly as low as $0.01 per audio hour in select configurations.
What Nemotron 3 Diarization actually does
Nemotron 3 tackles this with roughly 100 million parameters and what Nvidia calls its Streaming Sortformer architecture. The model handles up to eight speakers simultaneously, producing speaker-activity probabilities at 10 millisecond resolution. That’s granular enough to catch the kind of rapid-fire crosstalk that typically turns transcription software into a confused mess.
The system operates in both streaming and offline modes. In streaming mode, it can run with configurable latency profiles that go as low as approximately 320 milliseconds.
One of the more notable engineering decisions is the single-pass end-to-end design using what Nvidia calls an Arrival-Order Speaker Cache, or AOSC. Traditional diarization pipelines require multiple stages: first segmenting audio, then extracting speaker embeddings, then clustering those embeddings to figure out who’s who. Nemotron 3 collapses all of that into one forward pass, which simplifies deployment considerably.
The model assigns speaker labels based on the order they first appear in a conversation.
The accuracy jump is significant
On the AISHELL-4 benchmark using its “low” latency profile, Nemotron 3 achieved a Diarization Error Rate of 9.8%. Its predecessor, the Streaming Sortformer v2.1 released in July 2025, scored 27.2% on the same test.
Diarization Error Rate measures the proportion of time that speakers are incorrectly attributed, missed, or falsely detected.
The previous model, formally known as diar_streaming_sortformer_4spk-v2.1, also only supported four speakers. Doubling that ceiling to eight opens the door for use cases involving panel discussions, group calls, and multi-party meetings where four-speaker limits were a real constraint.
Where this fits in Nvidia’s broader strategy
Nemotron 3 Diarization isn’t arriving in isolation. It’s part of a growing family of models Nvidia has been building around agentic AI and speech recognition, aimed at creating a full stack for voice-driven applications.
The target use cases are meeting transcription, call center analytics, podcast production, and any scenario where multiple people are talking and software needs to keep track. By open-sourcing the weights under the OpenMDW 1.1 license, Nvidia has already seen early integrations surface through partnerships with MLX and FluidAudio.
The $0.01 per audio hour price point through certain inference providers is also worth noting. At that cost, processing a full eight-hour workday of meeting audio would run roughly eight cents.
Disclosure: This article was edited by Editorial Team. For more information on how we create and review content, see our Editorial Policy.
English (US) ·