Word Error Rate: How to Measure If Your Dictation Tool Is Good Enough

Ask someone how accurate their dictation tool is and you will usually get a confident guess. Ask them how they know, and the answer is usually "it feels about right." That feeling is not a measurement, and it tends to be generous, because when you read your own transcript your brain quietly fills in what you meant to say.

Word Error Rate (WER) is the standard way to replace that feeling with a number. The formula is simple: WER = (Substitutions + Insertions + Deletions) / Total Reference Words. A WER of 5% means 5 errors for every 100 words you said. If you have never calculated your own WER, you are tuning your setup blind. Here is how to measure it, understand what drives it, and fix the problems that cost you the most time.

Why Your Sense of Accuracy Is Unreliable

When you read back a transcription, you process intent, not text. You skip past a dropped "the," read "their" as "there" without noticing, and ignore a stray inserted word. That is great for reading comprehension and terrible for measuring transcription quality.

WER exists because subjective assessment is unreliable. It is the metric speech recognition research uses to compare systems. OpenAI's Whisper paper, for example, reports results as word error rates across many datasets, and compares Whisper with commercial systems and professional human transcribers on the same recordings, concluding that "Whisper's performance is close to that of professional human transcribers" on that test set.^[1]

Why WER rather than other metrics? Character Error Rate is useful for languages without clear word boundaries, but for English dictation, word-level errors map directly to editing effort. You fix words, not characters. Sentence accuracy is too blunt: one small slip and five wrong words both count as "inaccurate." WER captures the granularity you care about: how many individual corrections will this cost me?

WER can exceed 100%, by the way. If a system inserts enough phantom words, the errors can outnumber the words you actually said.

How to Calculate Your Own WER in Three Minutes

You need a reference passage, your dictation tool, and a few minutes.

Step 1: Write or choose a 200-word passage that represents your typical dictation. If you write emails, use a real email. If you dictate clinical notes, use a de-identified note. Save it as your reference text.

Step 2: Dictate the passage at your normal pace. Do not perform or over-enunciate. Speak the way you do when working.

Step 3: Compare the output with your reference word by word. Count substitutions (wrong words), insertions (extra words), and deletions (missing words), and apply the formula.

If you are comfortable with Python, the `jiwer` library does the alignment for you:

```python

from jiwer import wer

reference = "The patient reported moderate pain in the lower back lasting three days"

hypothesis = "The patient reported moderate pain in the low back lasted three days"

error_rate = wer(reference, hypothesis)

print(f"WER: {error_rate:.1%}") # 2 substitutions / 12 words = 16.7%

```

If you do not code, paste both texts into any document comparison tool and count the differences:

```

Reference: The patient reported moderate pain in the lower back lasting three days

Hypothesis: The patient reported moderate pain in the low back lasted three days

^SUB ^SUB

Substitutions: 2 (lower -> low, lasting -> lasted)

Insertions: 0, Deletions: 0

WER = 2 / 12 = 16.7%

```

Why 200 words? With very short passages, a single error swings the result a lot. One error in 50 words is 2% WER, while the same error in 100 words is 1%. Two hundred words is long enough to be stable and short enough to repeat under different conditions.

It also helps to translate WER into the number of fixes you will make. The arithmetic is direct:

The Three Types of Errors That Inflate Your WER

Not all errors cost the same editing effort. Knowing which type dominates tells you where to focus.

Substitutions are usually the most frustrating: the system heard something and wrote the wrong word. "Affect" becomes "effect." "Patients" becomes "patience." Because they are real words, spell-check will not catch them. In legal or medical dictation, one substitution can change meaning. "The defendant was *present*" and "the defendant was *pleasant*" is a factual error, not a typo.

Insertions are extra words. They can come from breathing, lip noise, background sound, or filler words. If you say "um" 15 times in a 200-word passage and the system writes each one down, that alone adds 15 insertions, or 7.5 percentage points of WER against a reference without the fillers.

Deletions are dropped words, often short function words such as "a," "the," "in," or "of." Individually they feel minor. Together they make text choppy. In technical documentation, a dropped "not" can invert an instruction.

Filler words deserve special attention. Raw WER treats every word equally, but your editing effort depends on whether your tool removes fillers automatically. Be careful how you measure this. If your reference transcript includes the fillers you actually spoke, a tool that removes them will score worse on strict WER even though the output is cleaner. Track filler removal and grammar cleanup as a separate measure of editing burden, and keep raw WER tied to the same reference.

Why Your WER Changes From Room to Room

Run the same passage in different rooms and you will likely see different numbers. The words, voice, and software stay the same. The environment changes.

Background noise is the obvious factor. Room echo is the less obvious one. Hard surfaces such as glass, tile, and bare walls reflect your voice back into the microphone slightly delayed, which makes recognition harder. A room with soft furnishings usually sounds cleaner than a tiled kitchen at the same noise level.

Microphone distance matters more than most people expect. The farther the microphone is from your mouth, the more room sound competes with your voice. A laptop microphone on a desk while you lean back picks up much more of the room than a headset microphone near your mouth.

Sample rate is not the lever. The Whisper paper says all audio "is re-sampled to 16,000 Hz" before processing.^[1] Recording at 48 kHz does not give the model more to work with. What helps is a clean signal: a microphone close to your mouth, a sensible input level, and less room noise.

For many people, moving from a distant built-in microphone to a close headset microphone is the highest-impact change available. Test it with your reference passage rather than taking that on faith.

Model Size: A Lever Worth Testing Early

OpenAI's Whisper comes in several sizes: tiny (39M parameters), base (74M), small (244M), medium (769M), large (1550M), and turbo (809M).^[2] Larger models generally make fewer errors and run more slowly. OpenAI also notes that the English-only ".en" variants "tend to perform better, especially for the tiny.en and base.en models."^[2]

The difference between sizes tends to matter most on specialized vocabulary: medical terms, legal citations, technical jargon, and unusual proper nouns. Smaller models are more likely to substitute a common word for the one you said.

Latency is the trade-off. How long a larger model takes depends on your Mac, the runtime, and the length of the audio. Time it on your own machine. If a larger model adds a short wait but removes several corrections per paragraph, it usually saves time overall.

For more on these trade-offs, see our Whisper model comparison, our explainer on how on-device AI models run on Apple Silicon, and our look at the full cost of cloud transcription.

Building a Personal WER Benchmark You Can Reuse

A one-time test tells you where you stand today. A reusable benchmark tells you whether changes help or hurt.

Create a domain-specific reference passage. Write 200 words that contain the vocabulary you actually dictate: technical terms, names you use often, numbers, and words your tool has gotten wrong before.

Test under three conditions. Run the passage in (1) your best setup, (2) your typical daily setup, and (3) a harder condition such as background noise or the end of a long day. The gap between best and typical shows what your environment costs you.

Log results consistently. Track the date, condition, model, microphone, raw WER, and notes. Over a few weeks, patterns appear.

The rows below are example entries to show the format, not measured results:

Re-run the benchmark whenever you change hardware, update your transcription software, or switch models.

From WER Numbers to Actual Workflow Improvements

Whether dictation saves you time comes down to a simple equation: speaking time plus correction time versus typing time. Here is an illustrative example with stated assumptions you should replace with your own: you type at 60 words per minute, dictate at 150 words per minute, and each correction takes 5 seconds.

At 5% WER on a 500-word document, you have about 25 errors. At 5 seconds each, that is about 2 minutes of editing. Dictating 500 words at 150 words per minute takes about 3.3 minutes, so the total is about 5.4 minutes. Typing 500 words at 60 words per minute takes about 8.3 minutes. Under these assumptions, dictation wins by about 3 minutes.

At 15% WER, you have about 75 errors, or about 6.25 minutes of editing. Add 3.3 minutes of dictation and the total is about 9.6 minutes, longer than typing. Under these assumptions the break-even point is 12% WER. Plug in your own speeds to find yours.

Rank your fixes by effort:

1. 1.Microphone position and quality (low effort): get the microphone close to your mouth.
2. 2.Model choice (moderate effort): test the next size up on your reference passage.
3. 3.Environment (variable effort): close a door, add soft surfaces, move away from fans and vents.
4. 4.Speaking technique (ongoing effort): steady pace, clear phrase boundaries, short pauses between sentences.

Cleanup features help too, but track them with the right measure. If a verbatim reference includes spoken fillers, removing them counts as deletions in standard WER. Track filler word removal and grammar correction as editing burden or readability, and keep raw WER tied to the same reference.

Your WER Checklist: What to Do This Week

Here is a 30-minute plan:

1. 1.Write your reference passage (10 minutes): 200 words that match your daily vocabulary, including at least five domain terms and two or three names.
2. 2.Run three tests (15 minutes): dictate the passage under best, typical, and harder conditions, and record raw WER for each.
3. 3.Log your baseline (5 minutes): note microphone, model, and environment for each run.

The one metric to track weekly: your WER on your standard passage under your standard conditions. If it stays stable, your setup is consistent. If it creeps up, something changed. If it drops, your changes are working.

Frequently Asked Questions

Is 0% WER realistic?

Not consistently. Even professional human transcribers make some errors, and the Whisper paper compares models against human transcription services on the same recordings rather than against a perfect score.^[1] Aim for a WER low enough that your corrections take less time than typing.

Does accent affect WER?

It can. Recognition accuracy varies with accent, speaking style, and how well the training data represents your speech. Test with your own voice, and try a larger multilingual model if a smaller one struggles.

How often should I re-benchmark?

After any hardware change, after software updates to your transcription tool, and periodically as a routine check. It takes a few minutes.

Does speaking speed affect WER?

It can. Very fast speech blurs word boundaries, and very slow, choppy speech can break up phrases unnaturally. Find a steady, natural pace and keep it consistent between tests.

Measure Your Baseline WER Today

Your next step is concrete: open a text editor, write your 200-word reference passage, and dictate it now. In a few minutes you will have a real number instead of a guess, and a starting point for every improvement that follows.

References

[1] Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision (2022). https://arxiv.org/abs/2212.04356

[2] OpenAI, Whisper repository README. https://github.com/openai/whisper