What Are WER and WRR in Speech Recognition Systems

When working with ASR systems, one of the first questions that arises is: "How well does this model perform?" To answer objectively, we need quantitative metrics. The industry standard for evaluating ASR accuracy is Word Error Rate (WER). Let's explore what it is, how to calculate it, and how to interpret the results correctly.

What Is Word Error Rate (WER)?

Word Error Rate (WER) is a metric that measures the discrepancy between the text generated by an ASR system (the hypothesis) and the reference transcription verified by a human (the reference). The lower the WER, the more accurate the model.

WER is based on the Levenshtein algorithm, adapted to work with words instead of characters.

To calculate WER, three types of errors must be identified:

• Substitutions (S): Words the system recognized incorrectly. For example, "will" instead of "was".
• Deletions (D): Words present in the reference transcription but missed by the system.
• Insertions (I): Extra words the model "invented" that were not in the original audio.

Formula and Worked Example

The WER formula:

Where $S$ — substitutions, $D$ — deletions, $I$ — insertions, $N$ — total number of words in the reference transcription.

Worked Example

Let's calculate WER for a specific case:

• Reference: the weather will be nice today (N = 6 words)
• Hypothesis: the weather was be nice today yes

Aligning the words:

Total: $S=1$, $D=0$, $I=1$, $N=6$

What Is Word Recognition Rate (WRR)?

Word Recognition Rate (WRR), sometimes called Word Accuracy, is the "inverse" metric to WER. It indicates the proportion of correctly recognized words.

For our example: $\text{WRR}=1-0.33=0.67\to 67\%$

An alternative formula uses hits ($H$) directly:

For our example: $\text{WRR}=\frac{5-1}{6}=0.67\to 67\%$

How to Interpret Results?

WER evaluation depends heavily on context: audio quality, domain, presence of accents. However, these general benchmarks apply:

• 0–5% WER — excellent, comparable to human transcription quality.
• 5–10% WER — great quality, text requires almost no corrections. Production-ready.
• 10–20% WER — acceptable quality, some post-editing may be needed.
• 20–30% WER — fair quality, noticeable errors. Model needs tuning.
• 30% and above — poor quality, transcription is hard to use. Significant improvement required.

Limitations of WER

Despite its popularity, WER is not a perfect metric:

• All words are equal. WER treats replacing "in" with "on" the same as replacing "not" with "now", even though the second error completely changes the meaning.
• No punctuation awareness. Standard WER ignores punctuation, capitalization and formatting.
• Does not measure readability. Two texts with identical WER can have vastly different readability.

Conclusion

WER and WRR are fundamental tools for evaluating ASR system performance. They provide a fast, standardized accuracy assessment, enabling model comparison and training progress tracking.

However, a deep analysis requires looking beyond the final number. It's essential to analyze the errors themselves — are substitutions, insertions, or deletions dominant? Which specific words does the model confuse? Answering these questions is key to further improving speech recognition quality.¹