TL;DR. Speaker diarization is the step in AI transcription that works out who spoke when, splitting a recording into turns labeled Speaker 1, Speaker 2 and so on. Strong systems still get about 15 to 25% of speech time wrong on hard real-world audio (pyannote). It labels voices, not people, so check names yourself.
Last reviewed: October 7, 2026. Reviewed quarterly.
Open almost any AI meeting transcript and you will see the same pattern: “Speaker 1” says something, “Speaker 2” answers, and sometimes the labels quietly swap halfway through. That labeling job has a name. On October 6, 2026, OpenAI’s help article for ChatGPT audio uploads said it plainly: “Transcripts may contain errors, and speaker identification may be unreliable.” A company selling the tool told you not to trust the who-said-what column. That sentence is the reason this page exists.
Speaker diarization is the task of working out who spoke when in a recording and splitting the audio into turns attributed to separate, anonymous speakers. In plain terms: the transcription step decides what words were said, and diarization decides when the voice changed. It does not know anyone’s name. It only knows that this voice is different from that one.
Why speaker diarization matters now
Speaker diarization matters because AI transcripts have moved from a novelty to a record people act on. A summary that says “the client agreed to the revised price” is only as good as the label saying which voice agreed. As tools like ChatGPT, Gemini and dedicated notetakers make recordings the default input for notes, follow-up emails and clinical documentation, a wrong speaker label stops being a cosmetic bug. It changes who is said to have promised, refused or reported something.
Here is what the evidence says about how reliable the labeling is:
- Standard benchmarks are friendlier than real life. According to pyannote, a diarization company, strong systems reach a diarization error rate of about 5 to 8% on standard benchmarks and about 15 to 25% on challenging real-world data (pyannote.ai, 2026). The same post shows a sample of 7.2% on the AMI meeting corpus and 18.6% on the harder DIHARD set.
- Some conversations are solved, others are not. The Third DIHARD challenge, an academic evaluation with 30 organizations submitting 499 system outputs across 11 domains, found diarization “has improved markedly… particularly for two-party interactions, but that for many domains (e.g., web video) the problem remains far from solved” (Ryant et al., 2020).
- The vendors say so too. OpenAI’s ChatGPT upload help page warns that “speaker identification may be unreliable” and tells users to “review important details against the original recording” (OpenAI Help Center, October 2026).
A “diarization error rate” is not the same as a word error rate. Word error rate counts wrong words. Diarization error rate counts the share of speaking time that was missed, invented or given to the wrong person. A transcript can have nearly perfect words and still pin them on the wrong speaker. We cover the word-accuracy side in our post on ChatGPT audio uploads.
How speaker diarization actually works
Speaker diarization works by finding the stretches of speech in a recording, giving each stretch a numeric “voice signature,” and grouping the stretches whose signatures look alike. Each group becomes a speaker label. The system never needs to know who anyone is. It only needs to decide that these 40 segments sound like one voice and those 25 sound like another.
In plain terms, think of sorting a pile of unlabeled voicemails by who sounds like whom, without knowing any of the callers. You would still end up with piles, and you could name each pile afterward.
The usual pipeline has five steps. Speech detection finds where someone is talking and where there is silence or noise. Segmentation cuts the speech into short pieces, usually at pauses or when the voice changes. Embedding turns each piece into a list of numbers that captures how the voice sounds, not what it says. Clustering groups similar embeddings, and the number of groups is the guessed number of speakers. Finally, the system lines the labeled time ranges up with the transcript’s word timestamps, so each sentence gets the label of the voice speaking at that moment.
Here is a worked example of where it goes wrong. Imagine a 40-minute call with three people: a customer, an agent and a supervisor who joins at minute 25. The system hears the first two voices clearly and labels them Speaker 1 and Speaker 2. When the supervisor joins, the system must decide whether the new voice is a third person or the agent on a worse connection. If it guesses wrong, the supervisor’s sentences are attributed to the agent, and a summary may say the agent promised a refund that the supervisor offered. The words are correct. The attribution is not. That is why a good habit is to spot-check the moments where a new person joins or two people talk over each other.
Some newer models return the transcript and speaker labels together. OpenAI’s developer model gpt-4o-transcribe-diarize, for example, returns a “diarized_json” response with segments that carry speaker, start and end fields, and accepts up to four short reference clips of known speakers (OpenAI speech-to-text guide, 2026). That is a different job from ChatGPT’s consumer audio uploads, and OpenAI documents them separately.
Speaker diarization vs naming a speaker vs voice biometrics
Three things get mixed up under the label “speaker identification,” and they carry very different accuracy and privacy consequences. Diarization separates voices inside one recording and gives them anonymous labels. Name attribution attaches a real name to a label using something you supply. Voice biometrics recognizes a person from their voice across recordings. Most consumer tools do the first, some do the second, and the third raises the most legal questions.
| What it does | What it needs | Typical output | Risk if wrong | |
|---|---|---|---|---|
| Speaker diarization | Separates voices within one recording | Audio only | “Speaker 1”, “Speaker 2” | Words attributed to the wrong turn |
| Name attribution | Puts a real name on a label | Names you type, a calendar invite, or short reference clips of known voices | “Priya”, “Dev” | Wrong person credited with a decision |
| Voice biometrics (voiceprint) | Recognizes a person across recordings | A stored sample of their voice | “This is Priya’s voice” | Privacy and legal exposure, not just a bad label |
Two examples of the middle layer. OpenAI’s ChatGPT Record feature on macOS “can distinguish multiple speakers,” and when it cannot identify someone by name “it uses a generic speaker label such as Speaker 1,” which you can rename after the recording (OpenAI Help Center, 2026). OpenAI’s developer model can take up to four reference clips of 2 to 10 seconds each so that recurring speakers get the right names.
The third layer is deliberately avoided by general chat models. OpenAI says GPT-4o was post-trained to refuse to identify a person from their voice because of privacy and surveillance risks (GPT-4o system card). In the European Union, a voice recording of an identifiable person is personal data, and it becomes biometric special-category data when it is processed to uniquely identify someone (GDPR, Article 4). In the United States, Illinois’s Biometric Information Privacy Act lists voiceprints among the biometric identifiers it covers. So “who is speaking?” answered by a stored voiceprint is a legal question, while “Speaker 1 versus Speaker 2” within one file mostly is not.
Where speaker diarization shows up in real work
Speaker diarization is built into most tools that turn conversations into text, whether or not they use the word. You meet it as the “Speaker 1” column in a transcript, the speaker-by-speaker view in a meeting recap, or the agent-versus-customer split in a call review.
| Where you meet it | What it does with speakers | Worth knowing |
|---|---|---|
| ChatGPT audio uploads (paid plans) | Transcribes and summarizes an uploaded recording | OpenAI says speaker identification “may be unreliable”; state the names in your prompt and verify |
| ChatGPT Record (macOS app) | Live capture with multiple speakers separated | Generic labels like “Speaker 1”, renameable; works best in English |
OpenAI developer API (gpt-4o-transcribe-diarize) | Returns speaker, start and end per segment | Up to four known-speaker reference clips; 25 MB file limit on the transcriptions endpoint |
| Free Gemini transcription | Interview transcripts with speaker labels | See our free Gemini transcription walkthrough |
| Meeting notetakers (bot-based, local, platform-native) | Attribute notes and action items to people | Our AI Notetaker page explains the capture types |
| Open-source toolkits such as pyannote | Build-your-own diarization for developers | Needs setup; accuracy depends on audio and tuning |
Always check each tool’s own help page. Features, limits and plan requirements in this space change within weeks, and this page is reviewed quarterly.
What this means for customer support
Support teams get the cleanest case, because most calls have exactly two voices: an agent and a customer. That is the situation where, according to the DIHARD evaluation, diarization has improved the most. A reliable agent-versus-customer split lets a manager search “what did the customer say about cancelling?” without reading the agent’s lines, and lets quality reviewers score the agent’s turns separately.
The practical workflow: record calls on two audio channels if your phone system allows it, with the agent on one channel and the customer on the other. When the channels are already separate, the system barely needs to guess. Run the transcript, check the first minute to confirm which label is the agent, then sample a few calls by hand each week. The limit to remember is three-way calls and transfers, where a new voice appears mid-call and attribution is most likely to slip. Never base a disciplinary decision on a speaker label alone.
If you want to build this into a support operation, our AI Customer Support course walks through call summaries and quality review, with the first lessons free.
What this means for healthcare workers
In clinical documentation the speaker label carries medical weight. “I’ve had chest pain for two days” attributed to the clinician instead of the patient is not a typo; it is a different record. Ambient AI scribes depend on telling clinician from patient and from family members in the room, and a 2026 five-site study reported in STAT found scribes saved clinicians modest amounts of time with inconsistent use (STAT, April 2026), which is a reminder that review is still part of the job.
The workflow for a clinician or therapist using any recording-based tool is to treat the transcript as a draft, scan every symptom, medication, dose and consent statement against who actually said it, and keep the final note in your own words. Rooms with more than two voices (a patient with family, a care team huddle) are where labels slip. Consent matters first: tell everyone in the room, and check your organization’s rules on where audio may be processed. Voiceprints and stored recordings add privacy obligations well beyond a label.
Our AI Therapy Notes HIPAA Workflow course covers a compliant documentation routine, and the Healthcare Workers course gives the wider foundation.
What this means for freelancers, interviewers and consultants
If you interview people for a living, whether as a journalist, researcher, coach or consultant, diarization decides whether the quote you pull is actually theirs. An interview is usually two voices, which is the easy case. The risk is subtler: a transcript that merges your leading question into the interviewee’s answer, or attributes your paraphrase to them.
A reliable routine is to say each person’s name at the start of the recording (“This is Dana Reyes, interviewing Sam Okafor”), because it gives you an audio anchor to check the labels against. When you ask a chat model to produce notes, tell it who the speakers are and ask it to flag any passage where it was unsure who was speaking. Before you publish or send a quote, listen to those few seconds. The limit: a transcript is not a verbatim record, and an AI summary of a transcript can smooth over a misattribution entirely.
Our Freelancers course covers client workflows with AI, and you can read how we handle the consent side in our post on Google Meet’s automatic notes for therapists, lawyers and advisors.
What this means for developers
For developers, diarization is a feature you can request from an API or run yourself. The practical choices are a hosted model that returns speaker segments, an open-source toolkit such as pyannote, or the speaker features built into a speech-to-text vendor. The decisions that matter are how you handle overlapping speech, whether you can supply known-speaker reference clips, how you chunk long audio, and how you evaluate the result.
With OpenAI’s developer model, the documented details include diarized_json output with speaker, start and end metadata, up to four reference clips of 2 to 10 seconds, a chunking_strategy setting for audio longer than 30 seconds, a 25 MB file limit on the transcriptions endpoint, no speaker labeling in Realtime transcription sessions, and no support for prompts on that model (OpenAI speech-to-text guide, 2026). Evaluate with diarization error rate on your own audio, and be careful comparing vendor numbers: a result quoted with a generous tolerance collar and overlaps ignored is not comparable to a strict one. As an independent preprint on accents shows, speech models can behave very differently across accents, so test with the voices your users actually have (GigaSpeechBench, 2026, a preprint).
Our Developers course covers building with AI APIs from the ground up.
What this means for small business owners and managers
If you run a small business or a team, you will meet diarization as the “who agreed to what” line in meeting notes, supplier calls and client check-ins. The value is simple: action items with the right owner. The risk is just as simple: an action item assigned to the wrong person, which then does not get done or causes a dispute.
The routine that works is to read the action-item list aloud at the end of the meeting, which doubles as a human check on the AI’s attribution. When you rely on a tool’s summary, make sure each action item names an owner and a date and that the owner is who you remember. Tell attendees you are recording and using AI notes before you start; in many places that is a legal requirement, not courtesy. Our AI Meeting Notes course teaches the capture, prompt and review habits in eight short lessons, with the first two free.
Common misconceptions about speaker diarization
Most mistakes about speaker diarization come from reading a speaker label as a fact about a person, when it is a machine’s grouping of similar-sounding stretches of audio. The five myths below are the ones that cause real trouble in notes, interviews and records, and each has a simple correction you can apply today.
“Diarization tells you who is speaking.” It tells you that voices differ, not who they belong to. A name only appears if you or a calendar supplied it. Treat “Speaker 2” as a placeholder, and fix the names before the transcript travels anywhere.
“If the transcript words are right, the speaker labels are right.” The two are measured separately. A transcript can score well on word accuracy and still mislabel turns, particularly with interruptions and similar voices. OpenAI’s own warning about unreliable speaker identification sits next to its separate warning about transcript errors for exactly this reason.
“More speakers just means more work for the AI.” Accuracy tends to fall as the number of speakers rises and as people talk over each other. The DIHARD results are clearest on this: two-party conversations improved the most, while other settings remain hard (Ryant et al., 2020). A four-person round table is a harder problem than a phone call.
“A speaker label is evidence.” Not on its own. It is a machine’s best grouping, and it can be wrong without any visible sign. Where an attribution matters (a legal dispute, an HR matter, a clinical record), the audio is the evidence and the transcript is an index into it.
“Diarization is a voiceprint.” Within a single recording it is not. It does not store or recognize anyone across recordings. A voiceprint is a different, more sensitive technology with its own legal rules.
How to get better speaker labels
You can improve speaker labels more by how you record than by which tool you pick, because the grouping step works best when voices are clearly separated, loud enough and not overlapping. These seven habits cost nothing and work with any transcription tool, from ChatGPT audio uploads to a dedicated meeting notetaker.
- Give each person a name on the recording. Open with “This is Priya, with Dev and Sam.” It gives you a human-checkable anchor.
- Record each person on their own channel when you can. Separate channels make the grouping step much easier than one shared microphone.
- Reduce overlap. Ask people not to talk over each other. Overlapping speech is a documented weak spot for diarization.
- Use a good microphone close to each speaker. A phone across the table makes similar voices sound alike.
- Tell the tool who is in the room. In a chat tool, write the speakers’ names and roles in the prompt. With an API, supply known-speaker reference clips if the model supports them.
- Ask for uncertainty. Add “flag any passage where you weren’t sure who was speaking” to your prompt.
- Spot-check the risky moments. Listen to the points where someone joins, two people overlap, or a decision is made.
Privacy and consent when speakers are labeled
Labeling speakers does not create new consent rules, but it raises the stakes on the ones that already apply. In the United States, federal law generally allows recording a conversation you take part in with one party’s consent, while a set of states, including California, Florida, Illinois, Maryland, Massachusetts, Washington and others, require all parties to agree for at least some conversations (Reporters Committee for Freedom of the Press). In the EU and UK, a recording of an identifiable voice is personal data, so you need a lawful basis and should tell participants what is happening (GDPR).
OpenAI’s own pages put the responsibility on you. Its Meetings plugin help says to “tell everyone in the meeting that you’ll be taking notes from the meeting and get everyone’s consent before you start,” and that the in-app reminder “does not notify other participants or obtain their consent for you.” The simplest habit is to say it out loud at the start, wait for a yes, and keep that in the recording. If you plan to store voiceprints, or to recognize people across recordings, get proper legal advice first; that is a different category from labeling voices inside one file.
Related terms
- AI Notetaker: the tools that record, transcribe and summarize meetings, and depend on speaker labels.
- AI Scribe: documentation tools that separate clinician and patient turns.
- Ambient AI: always-listening assistants where speaker separation matters most.
- AI Voice Cloning: the other side of voice AI, copying how someone sounds instead of telling voices apart.
- AI Hallucination: why a confident output, including a speaker label, can still be wrong.
See also
Courses on this and related topics
- AI Meeting Notes: capture, prompt and verify meeting notes
- AI Customer Support: call summaries and quality review with AI
- AI Therapy Notes HIPAA Workflow: a compliant documentation routine
- AI Case Notes for Social Workers: documentation with consent and review built in
- AI for Coaches: Session Notes: turn sessions into notes and follow-ups
- AI for Consultants: Client Decks: client-ready outputs from messy inputs
- Healthcare Workers: AI foundations for clinical roles
- Freelancers: AI workflows for independent professionals
- Journalists and Media: AI for reporting, interviews and verification
- Building AI Voice Agents: speech-to-text, text-to-speech and call flows
- AI Voice Cloning & Synthesis: how voices are copied, and how to stay safe
- Developers: building with AI APIs
- AI Fundamentals: the starting point for everything above
Related terms in this glossary
- AI Notetaker: meeting tools that record and summarize
- AI Scribe: clinical and call documentation drafted by AI
- Ambient AI: the always-listening assistant paradigm
- AI Voice Cloning: copying a voice from seconds of audio
- AI Hallucination: when confident AI output is wrong
- AI Text Watermark: invisible provenance marks in AI text
- SynthID: Google’s invisible watermark for AI media
AI skills (prompt templates)
- Audio Transcription with Whisper: transcribe audio with speaker detection and timestamps
- Meeting Transcript Fixer: fix names, jargon and filler words in transcripts
- Meeting Notes & Action Item Extractor: decisions and action items with owners
- Meeting Notes Generator: structured summaries from messy transcripts
- Sales Call Analyzer: objections and next steps from call transcripts
- Customer Interview Synthesizer: insights from interview transcripts
Related blog posts
- ChatGPT Audio Uploads: Turn a Recording Into Notes (2026): limits, privacy and consent for uploaded audio
- How to Transcribe an Interview Free With Gemini: a free route with speaker labels
- Turn Any Meeting Into Follow-Ups With Gemini in 5 Minutes: the prompt workflow
- Google Meet AI Notes Just Went Automatic. Now What?: checking and controlling default-on notes
- Google Meet AI Notes: What Therapists & Lawyers Must Do: regulated-profession risks
- The Free AI Scribe an Independent Clinician Can Actually Use: documentation without a hospital IT team
- AI Meeting Notes for Financial Advisors: The Compliance Line: where notetakers cross compliance lines
Profession deep-dives
- Learn AI for Freelancers: the freelancer hub
- Learn AI for Small Business: the small-business hub
The bottom line
Speaker diarization is a sorting step, not a witness. It tells you that voices differ, it is least reliable exactly when a conversation gets messy, and the people who sell it say as much. Use it to find the right moment in a recording faster, then check names, numbers and commitments against the audio before they leave your desk. FindSkill.ai’s AI Meeting Notes course builds that habit.
Frequently asked questions
What is speaker diarization? Speaker diarization is the process of working out who spoke when in a recording, so a transcript can be split into turns labeled Speaker 1, Speaker 2 and so on. It answers the question of when the voice changes, not who the person is.
Is speaker diarization the same as speaker identification? No. Diarization separates voices and gives them anonymous labels. Naming a speaker takes an extra step, such as you typing the names, a calendar invite, or a short reference clip of the person’s voice. Identifying someone from their voiceprint alone is a third, more sensitive thing again.
Why does my AI transcript mix up who said what? Speakers who interrupt each other, similar-sounding voices, a noisy room, a phone held across the table and more than a few participants all make the grouping step harder. OpenAI itself warns that speaker identification in ChatGPT audio uploads may be unreliable.
How accurate is speaker diarization? It depends on the audio. According to pyannote, strong systems score a diarization error rate of about 5 to 8% on standard benchmarks and about 15 to 25% on challenging real-world recordings. The DIHARD III evaluation found that two-party conversations improved the most, while some domains such as web video remain far from solved.
Can ChatGPT label speakers in an uploaded recording? ChatGPT can try, but OpenAI’s help article says speaker identification may be unreliable and tells you to review important details against the original recording. OpenAI’s separate developer model, gpt-4o-transcribe-diarize, returns speaker labels with start and end times, and can use up to four short reference clips of known speakers.
Is diarization the same as voice recognition or a voiceprint? No. Diarization only decides that one voice differs from another within a single recording. A voiceprint stores a person’s voice so they can be recognized across recordings, which raises extra privacy rules. In the EU, a voice becomes biometric special-category data when it is processed to uniquely identify someone.
Sources
- OpenAI, speech-to-text guide (diarization, diarized_json, known speakers)
- OpenAI Help Center, “Uploading files and audio to ChatGPT”
- OpenAI Help Center, “ChatGPT Record”
- OpenAI Help Center, “The Meetings plugin in ChatGPT”
- Ryant et al., “The Third DIHARD Diarization Challenge” (arXiv)
- pyannote.ai, “How to evaluate speaker diarization performance”
- OpenAI, GPT-4o System Card
- GDPR, Article 4 definitions (EUR-Lex)
- Reporters Committee for Freedom of the Press, First Amendment Handbook
- GigaSpeechBench, accent evaluation preprint (arXiv)
- STAT, “Large AI scribe study finds modest time savings, inconsistent use”
- Wikipedia, Speaker diarisation
