Free Your four-agent team is ready Claim Your AI Team
← Back to Blog

Scribe v3 and v3 Live: A Practical Evaluation Guide

Conceptual microphone, sound wave and approval symbol illustrating speech recognition evaluation, not a product screenshot.

Key Takeaways

  • Recorded and live recognition require different workflow evaluations.
  • Availability does not prove an existing agent has changed its default model.
  • Turn prediction requires explicit configuration and current SDK support has limits.
  • Compare critical-field accuracy and total correction effort rather than treating vendor benchmarks as business outcomes.
BLOOMIE
POWERED BY NEROVA
Author
Bloomie

Produced by Bloomie for Nerova AI using automated editorial checks. Sources used for factual claims are listed below.

ElevenLabs announced Scribe v3 and Scribe v3 Live on November 10, 2026, with recorded-audio and realtime speech-recognition paths. Its API changelog is dated November 9; the announcement was updated November 14. These are different source dates, not a reason to backdate this analysis.

The practical question for a support team is whether the system captures the right customer details and lets callers finish their thoughts. A lower benchmark error rate is useful evidence, but it is not a measured reduction in failed bookings, incorrect addresses or manual corrections in your operation.

Choose the recorded or live workflow first

The API changelog identifies scribe_v3 for recordings and scribe_v3_live for realtime transcription. It says their v2 predecessors remain available. Recorded audio can be useful for meeting archives and post-call analysis; a live conversation additionally needs reliable turn handling and timely downstream responses.

New options described there include number formatting and secondary language codes for recordings, plus turn prediction, multilingual transcription and a keyterm library for live use. Check the exact request contract for the model you choose rather than assuming batch options and live options are interchangeable.

Availability is not the same as your agent's configuration

The launch announcement says v3 Live is available in ElevenAgents and both models are available through ElevenAPI. The product guide still lists v2 Realtime as the ElevenAgents default. Availability and default configuration are different claims; we do not assume either source describes every existing deployment. Before comparing results, inspect the actual model configuration, deployment version and surface serving your traffic.

This distinction also prevents a misleading rollout: changing a test script's model ID is not proof that the production phone agent changed. Record the before-and-after configuration and verify the path with a controlled, authorized test. This guide does not claim every existing agent was automatically migrated.

Turn prediction needs an explicit setting

The model documentation says changing to scribe_v3_live leaves the default commit strategy as manual. Set commit_strategy=turn_prediction to request predictive turn completion. As checked November 15, it also says the official SDKs do not yet support turn prediction, multilingual mode or the keyterm library; those features require direct WebSocket integration. Scribe Live is speech recognition, not the separate Eleven Realtime complete voice-agent models.

Read the benchmark denominator before repeating a percentage

The updated launch article reports 27% lower word error rate for recorded Scribe v3 versus v2 across 102 languages. For v3 Live versus Scribe v2 Realtime on FLEURS, it reports 43% lower semantic word error rate across 102 languages. The November 9 changelog instead reports 44% lower semantic word error rate versus Scribe v2 Realtime on FLEURS across 73 languages. We retain those source-specific descriptions rather than combining them into one universal improvement claim.

Word error rate and semantic word error rate answer different questions. The former measures transcription differences; the latter focuses on whether meaning changed. A result on clean read speech is also not interchangeable with noisy, overlapping customer calls. Vendor-reported comparisons should inform what you test, not substitute for testing your own task.

Build a small, representative evaluation set

Use recordings you are authorized to process, with appropriate handling for personal or sensitive information. Include the languages, accents, connection quality and interruptions that actually occur in your service. Keep a held-out set so improvements to prompts or terminology lists are not evaluated only on examples used to tune them.

Write expected answers before inspecting model output. For an appointment workflow, the crucial fields might be the requested date, time, service and callback number. For order support, they may be the order reference and delivery address. Decide which errors require human review and which formatting differences are acceptable.

Keep the downstream extraction prompt and business logic fixed while comparing recognition models. Otherwise you cannot tell whether a gain came from transcription, a changed prompt or a different validation rule.

Five measurements that matter beyond a transcript score

  1. Critical-field accuracy: score the exact fields required to complete the task. Report numbers, names and dates separately so common words do not hide consequential mistakes.
  2. First-pass completion: count tasks completed correctly without asking the customer to repeat information. Define a successful outcome before the trial.
  3. Premature turn endings: include callers who pause while reading a number or thinking. Record cutoffs separately from slow responses.
  4. Correction effort: measure staff review time, clarification turns and downstream repairs, not just recognition usage.
  5. Failure handling: test timeouts, interrupted connections and ambiguous values. A clear request for clarification is preferable to silently acting on an uncertain number.

A simple cost-per-correct-task worksheet

Suppose a hypothetical evaluation costs $20 in service usage and $30 in review labor, producing 90 correctly completed tasks. The combined cost is $50 divided by 90, or approximately $0.56 per correct task. A second configuration with a higher recognition bill can still be better if it reduces review labor enough, but that must be measured.

This is an illustrative calculation, not ElevenLabs pricing or a Nerova test result. Include the actual applicable plan, usage, telephony, language-model and human-review costs in your own comparison. Do not present a speech-recognition rate as the complete cost of a voice agent.

Roll out only after the result is useful

Start with a bounded task, preserve the previous configuration and review enough varied cases to distinguish a genuine improvement from a favorable handful of calls. Do not optimize on a single hour simply because a new model is available. Correct clear failures immediately, but give uncertain comparisons an appropriate observation window.

For teams evaluating where automation belongs, Nerova's operational-improvement projects focus on a concrete business outcome. Recognition quality is one part of that outcome, alongside authorization, reliable execution and a usable handoff to a person.

Published November 15. This article reviews official documentation and proposes an evaluation method; it does not claim a Nerova integration, independent benchmark or guaranteed improvement.

Nerova context

Custom AI agents for business operations

Nerova builds custom AI agents for business operations. Companies use Nerova when they need AI support for customer intake, support, sales follow-up, research, website audits, internal handoffs, and workflow automation.

Nerova can help turn websites, business context, and operational workflows into practical AI systems: website chatbots, single-purpose agents, AI teams, audits, and automation workflows built around a clear business outcome.

Frequently Asked Questions

Does switching model IDs automatically enable turn prediction?

No. Current documentation says the default commit strategy stays manual until turn_prediction is explicitly requested.

Are the cost examples ElevenLabs prices?

No. They are hypothetical evaluation arithmetic, not provider rates or a measured Nerova result.

Find an automation worth implementing

Explore a bounded operational-improvement project with a clear customer outcome before expanding automation.

Explore operational improvements
Ask Bloomie about this article