By the LattifAI team ·

Forced Alignment Benchmark: Word-Level Timestamps on Short Utterances, Long Audio and Chinese

Forced alignment matches an existing transcript to a recording and gives every word a start and end time, with no need to transcribe again. Common uses:

  • Turn a script or a proofread transcript into SRT / VTT / ASS subtitles
  • Align lyrics to singing for word-highlight karaoke subtitles
  • Highlight audiobooks, courses and language-learning material word by word, and jump to a word on click
  • Find and cut audio or video by its text
  • Add word-level timestamps to training data for speech recognition, speech generation and full-duplex models

Most alignment benchmarks only test English utterances a few seconds long, while real subtitles and audiobooks often run for tens of minutes and mix languages. Using hand labels from TIMIT and the Biaobei Mandarin corpus, we tested English short utterances, 10- to 60-minute long audio, Chinese, and English-Chinese switching, comparing EdgeSpeak, Montreal Forced Aligner (MFA), ElevenLabs Forced Alignment and Qwen3-ForcedAligner.

On English short utterances, EdgeSpeak's error is 23.0 ms. With the utterances joined into 60 minutes and aligned in one pass, EdgeSpeak's error is 23.6 ms and ElevenLabs' is 32.0 ms. On 37 minutes of Chinese, EdgeSpeak scores 13.4 ms and ElevenLabs 54.7 ms. EdgeSpeak can be installed from the download page, or with pip install edgespeak.

The four sample recordings below were synthesized locally with EdgeSpeak. Each box is a word aligned by EdgeSpeak, and its left and right edges are the word's start and end times. Click any word to play from there:

Forced alignment takes a script you already have and finds the moment each word is spoken. Subtitles, karaoke lyrics and audiobook highlights all start from these timestamps.

Datasets:

  • TIMIT (LDC93S1): American English read by 630 speakers, with hand-labelled start and end times for every word.
  • Biaobei Mandarin female speech corpus (BZNSYP): 10,000 studio-recorded Mandarin sentences with hand-segmented initial and final boundaries. Used for Chinese and English-Chinese switching, scored per character.

Word-level timestamp error and language coverage

English, per word, TIMIT:

SystemPer-utterance errorShort F1@20Short F1@40Joined 9.7 minJoined 60 min
MFA 3.4.121.8 ms0.650.88not applicablenot applicable
EdgeSpeak23.0 ms0.620.8623.3 ms23.6 ms
ElevenLabs31.9 ms0.170.3832.3 ms32.0 ms
Qwen3-ForcedAligner32.2 ms0.400.73not supportednot supported

Error is the mean distance between predicted word boundaries and the hand labels (lower is better). F1@20 counts a boundary as correct when it falls within 20 ms of the hand label, and F1@40 widens that tolerance to 40 ms (higher is better); see method and metric definitions. One frame of 24 fps video is about 42 ms. EdgeSpeak's alignment model has a time resolution of 20 ms, so it has no advantage on F1@20, where the tolerance is also 20 ms; at 40 ms it is close to MFA.

Chinese uses 487 sentences from the Biaobei test set, about 37 minutes joined. Chinese is scored per character and English per word, so the two tables are not comparable:

SystemPer-utterance errorJoined 37 min error
EdgeSpeak13.3 ms13.4 ms
ElevenLabs61.1 ms54.7 ms
Qwen3-ForcedAligner34.2 msnot supported

MFA needs the audio split into utterances first and does not take a whole long recording; Qwen3-ForcedAligner supports up to 5 minutes.

Language coverage (per each vendor's official documentation):

SystemLanguages
EdgeSpeak346, including mixed-language text such as Chinese and English
ElevenLabs29
Qwen3-ForcedAligner11
MFAdepends on the pretrained model used

English short utterances: TIMIT core test

Five word cards above a voice waveform; two rows of tick marks for the hand labels and the alignment nearly coincide, and a caliper measures 20 ms

The 192 utterances of the TIMIT core test, clean read speech:

TIMIT core test word-boundary error (lower is better)
MFA 3.4.121.8 ms
EdgeSpeak23.0 ms
ElevenLabs31.9 ms
Qwen3-ForcedAligner32.2 ms

F1 as the tolerance widens:

SystemF1@20F1@40F1@50
MFA 3.4.10.650.880.92
EdgeSpeak0.620.860.90
Qwen3-ForcedAligner0.400.730.82
ElevenLabs0.170.380.47

Forced alignment on long audio: 10 to 60 minutes

A long paper strip unrolls from a wooden spool with a continuous voice waveform and word markers; a wooden ruler below reads 0, 30 min and 60 min

EdgeSpeak aligns 60 minutes in one pass with an error of 23.6 ms; aligned per utterance, the same set scores 23.4 ms. When the 60-minute recording is divided into 10 equal parts, the mean word-start error in each part stays between 18 and 22 ms, with no drift toward the end. Although 60 minutes is the longest test in this benchmark, EdgeSpeak supports recordings several hours long, so a whole meeting or an entire audiobook can be aligned in one pass.

The 9.7-minute test joins the 192 core-test utterances, while the 30- and 60-minute tests join training-set utterances end to end in a fixed order. The whole recording is sent at once, and the result is split back into utterances by word order before scoring.

English word-boundary error by audio length: EdgeSpeak 23.0 ms per utterance, 22.8 at 4.5 min, 23.3 at 9.7 min, 23.7 at 30 min and 23.6 at 60 min; ElevenLabs 31.9, 34.0, 32.3, 31.7 and 32.0 ms; Qwen3-ForcedAligner 32.2 per utterance and 44.2 at 4.5 min, not supported beyond; MFA 21.8 per utterance, not supported on joined audio
SystemLengthErrorF1@20F1@40F1@50
EdgeSpeak9.7 min23.3 ms0.620.860.90
EdgeSpeak30 min23.7 ms0.620.840.89
EdgeSpeak60 min23.6 ms0.620.850.89
ElevenLabs9.7 min32.3 ms0.190.380.48
ElevenLabs30 min31.7 ms0.200.400.48
ElevenLabs60 min32.0 ms0.190.400.49

Qwen3-ForcedAligner supports up to 5 minutes of audio, so we added a 4.5-minute set: the first 90 core-test utterances for English and the first 55 Biaobei test sentences for Chinese.

SystemEnglish, 4.5 minChinese, 4.5 min
EdgeSpeak22.8 ms / 0.6214.3 ms / 0.79
ElevenLabs34.0 ms / 0.1954.9 ms / 0.15
Qwen3-ForcedAligner44.2 ms / 0.4144.3 ms / 0.36

Each cell is error / F1@20. Qwen3-ForcedAligner scores 32.2 ms in English and 34.2 ms in Chinese per utterance, rising to 44.2 and 44.3 ms when joined into 4.5 minutes.

We also tested a 26-minute English launch-event recording with 4 speakers and 374 hand-labelled sentences. After removing the 28 sentences where several people talk at once, EdgeSpeak's median sentence-start error is 30 ms, and 92.2% of sentences have both start and end errors within 0.2 seconds.

Chinese forced alignment: short utterances and 37-minute long audio

Four character blocks reading 你好世界, each linked by a thin line to its own segment of the waveform below

We take the last 500 sentences of the Biaobei test set and score per character against the hand-checked initial and final boundaries. Each character runs from the start of its initial to the end of its final; characters without an initial start at the final. Removing 13 erhua sentences whose character and syllable counts do not match leaves 487.

Chinese character-boundary error by audio length: EdgeSpeak 13.3 ms per utterance, 14.3 at 4.5 min and 13.4 at 37 min; ElevenLabs 61.1, 54.9 and 54.7 ms; Qwen3-ForcedAligner 34.2 per utterance and 44.3 at 4.5 min, not supported at 37 min
SystemPer utteranceJoined 37 min37-minute run time
EdgeSpeak13.3 ms / 0.8013.4 ms / 0.8010.2 s
ElevenLabs61.1 ms / 0.1354.7 ms / 0.1716.9 s
Qwen3-ForcedAligner34.2 ms / 0.34not supported—

Each cell is error / F1@20, per character. ElevenLabs scores slightly better on joined audio than per utterance.

English-Chinese switching: 60 minutes

EN and Chinese recording segments alternate along a paper strip, and a mixing desk above brings both to the same level

English utterances (TIMIT) and Chinese utterances (Biaobei) alternate one by one, joined into 60 minutes, with no language specified. EdgeSpeak's Chinese error stays close to its per-utterance score (19.7 vs 18.9 ms), while English adds about 5 ms.

SystemAudioEnglish (per word)Chinese (per character)
EdgeSpeakper utterance (reference)23.6 ms / 0.6018.9 ms / 0.72
EdgeSpeakjoined 60 min28.2 ms / 0.6019.7 ms / 0.72
ElevenLabsjoined 60 min32.3 ms / 0.24124.8 ms / 0.20
Qwen3-ForcedAlignerjoined 60 minnot supported (5-minute limit)not supported

Lyric alignment and karaoke subtitles

EdgeSpeak's alignment model also handles singing: align lyrics to vocals with accompaniment and export word-highlight karaoke subtitles. Lyrics with [Verse] and [Chorus] markers are expanded for repeated and skipped sections to match what is actually sung; see supported inputs.

An English song generated locally by YuE2, with word-by-word karaoke subtitles from EdgeSpeak forced alignment.

Price and 60-minute run time

See pricing for EdgeSpeak. Run time is from submitting the 60-minute English recording to getting the result.

SystemPrice60-minute run time
EdgeSpeak$0.07–0.20 per hour depending on plan; Free covers about 53 minutes a month (see below)20.5 s
ElevenLabssame as transcription, $0.22 per hour, silence billed163 s

EdgeSpeak bills by input audio duration, counting 1 hour of audio as 45,000 tokens.

PlanPriceAudio includedPer hour
Free0about 53 minutes a month—
Basic, monthly$9 a monthabout 44 hours a monthabout $0.20
Basic, yearly$59 a yearabout 533 hours a yearabout $0.11
Pro, monthly$29 a monthabout 222 hours a monthabout $0.13
Pro, yearly$179 a yearabout 2,666 hours a yearabout $0.07
Alignment cost per hour of audio (lower is better)
EdgeSpeak Pro, yearly0.07 USD
EdgeSpeak Basic, monthly0.20 USD
ElevenLabs0.22 USD
EdgeSpeak shows the paid plans with the lowest and highest per-hour rates when the quota is fully used; with lower usage the per-hour cost is higher. ElevenLabs is shown at its per-hour list price. Checked 2026-11-10.

On-device processing and privacy

EdgeSpeak runs alignment on your device: audio, transcripts and results stay on your machine. See the privacy notice.

EdgeSpeak, MFA and Qwen3-ForcedAligner run locally; ElevenLabs needs the audio and transcript uploaded to the cloud.

Method and metric definitions

SystemVersion and interfaceRuns on
EdgeSpeakedgespeak-cli, Python SDK align(), default alignment model Lattice-2local machine
MFA3.4.1, english_us_arpa v3.0.0local machine
ElevenLabsForced Alignment APIcloud
Qwen3-ForcedAligner0.6B, qwen-asr 0.0.6local machine

Every system uses the same hand labels and the same scorer, one run per configuration. Scores are computed per utterance, then pooled over all boundaries; utterances with no output and words that do not match are included in the score.

  • Error: the mean absolute difference, in milliseconds, between each hand-labelled word boundary and the system's; the end of one word and the start of the next count as a single boundary, and utterance starts and ends are included.
  • F1@20: a boundary is a hit when it is within 20 ms of a hand-labelled boundary and the words on both sides match; F1 is the harmonic mean of precision and recall over boundaries, utterance edges included.

Long-audio results are first split back into utterances by word order, then passed to the same scorer. Feeding the hand labels in as output scores 0 ms and F1 1.0; short-utterance results sent through the split and scored again match direct scoring exactly.

TIMIT and Biaobei are clean read speech; singing is not scored.

Supported audio, video, transcripts and lyrics

InputDetails
Audio or video fileProcessed on the local machine, with support for recordings several hours long; the Python SDK also accepts audio bytes in memory and NumPy arrays
TranscriptPlain text, or a .txt, .json, .srt, .ass or .ssa file; existing subtitle timing is ignored and recomputed
LyricsLyrics with section markers such as [Verse] and [Chorus] are first checked against the actual singing order and expanded for repeated, skipped and reordered sections before alignment; this can be turned off (see the CLI docs)
LanguageThe default alignment model covers 346 languages. Set one by code, such as zh, en-US or haw, or leave it unset to detect it from the transcript; pass und for mixed Chinese-English text

Connect AI agents: MCP, Skills, CLI, SDK

When an agent calls EdgeSpeak, audio is processed on the local machine.

InterfaceForHow
MCPClaude Code, Cursor and other MCP clientsclaude mcp add edgespeak -- edgespeak-cli mcp, which provides 13 local tools including edgespeak_align
SkillsClaude Code, Codex, Cursornpx skills add lattifai/EdgeSpeak, with skills for alignment, karaoke subtitles, transcription, segmentation, translation and speaker naming
CLIAgents that run commands, scripts and batch jobsedgespeak-cli align …, with JSON, SRT or plain-text output
Python SDKYour own agents and data pipelinespip install edgespeak, EdgeSpeak().align(...); this benchmark calls EdgeSpeak this way

Once installed, tell the agent "use EdgeSpeak to align talk.mp4 with talk.txt and export SRT", or "make karaoke subtitles for this song".

Download and get started

Install the desktop app from the download page, import audio and a transcript on the Align page, then export SRT / VTT / ASS subtitles or karaoke subtitles.

The CLI is installed with the desktop app and can also be installed on its own; see the CLI docs.

Install the Python SDK with pip install edgespeak; usage is in the Python SDK docs.

The Python SDK's EdgeSpeak() starts an edgespeak-cli service on the local machine. Times in the result are in seconds, and errors raise EdgeSpeakError:

Python
from edgespeak import EdgeSpeak

with EdgeSpeak() as client:
    # With a language
    a = client.align("speech.wav", text="His captain was thin and haggard and his beautiful boots were worn and shabby.", language="en")

    # Without a language: detected from the transcript
    b = client.align("lecture.mp4", text="今天我们讲强制对齐。")

    # A subtitle file as the transcript; its timing is recomputed
    c = client.align("episode.m4a", text_file="episode.srt")

for segment in a["segments"]:
    for w in segment["words"]:
        print(w["start"], w["end"], w["word"])

The result (times in seconds; score is a confidence score; source_start_byte and source_end_byte are the word's byte offsets in the transcript):

JSON
{
  "task": "align",
  "duration": 4.9728125,
  "language": "en",
  "language_candidates": ["en"],
  "text": "His captain was thin and haggard and his beautiful boots were worn and shabby.",
  "segments": [
    {
      "id": 0,
      "start": 0.33,
      "end": 4.973,
      "text": "His captain was thin and haggard and his beautiful boots were worn and shabby.",
      "words": [
        { "word": "His", "start": 0.53, "end": 0.77, "score": 1.0, "source_start_byte": 0, "source_end_byte": 3 },
        { "word": "captain", "start": 0.77, "end": 1.27, "score": 1.0, "source_start_byte": 4, "source_end_byte": 11 },
        { "word": "was", "start": 1.27, "end": 1.49, "score": 1.0, "source_start_byte": 12, "source_end_byte": 15 },
        { "word": "thin", "start": 1.49, "end": 1.75, "score": 0.9742, "source_start_byte": 16, "source_end_byte": 20 },
        { "word": "and", "start": 1.75, "end": 1.85, "score": 0.7779, "source_start_byte": 21, "source_end_byte": 24 },
        { "word": "haggard", "start": 1.85, "end": 2.39, "score": 1.0, "source_start_byte": 25, "source_end_byte": 32 },
        { "word": "and", "start": 2.39, "end": 2.51, "score": 0.6644, "source_start_byte": 33, "source_end_byte": 36 },
        { "word": "his", "start": 2.51, "end": 2.73, "score": 0.9867, "source_start_byte": 37, "source_end_byte": 40 },
        { "word": "beautiful", "start": 2.73, "end": 3.19, "score": 0.9679, "source_start_byte": 41, "source_end_byte": 50 },
        { "word": "boots", "start": 3.19, "end": 3.65, "score": 0.9939, "source_start_byte": 51, "source_end_byte": 56 },
        { "word": "were", "start": 3.65, "end": 3.79, "score": 1.0, "source_start_byte": 57, "source_end_byte": 61 },
        { "word": "worn", "start": 3.79, "end": 4.19, "score": 1.0, "source_start_byte": 62, "source_end_byte": 66 },
        { "word": "and", "start": 4.19, "end": 4.33, "score": 0.67, "source_start_byte": 67, "source_end_byte": 70 },
        { "word": "shabby.", "start": 4.33, "end": 4.87, "score": 0.8857, "source_start_byte": 71, "source_end_byte": 78 }
      ]
    }
  ],
  "search_effort_used": "standard",
  "usage": { "type": "duration", "seconds": 4.9728125 }
}

The CLI works the same way; --language is optional:

Shell
edgespeak-cli align speech.wav --text-file speech.txt --language en -o speech.srt
edgespeak-cli align lecture.mp4 --text-file lecture.txt -o lecture.json

More in the docs: CLI, Python SDK, Align API.

← All posts