Multilingual Transcription Engine — Apple Music Lyrics Operations

Apple Music — Lyrics Operations

The long tail sings in languages
the current pipeline can't hear.

An ASR-drafted, LLM-corrected first pass for every track — so editors start from a draft instead of a blank page, and the languages human transcription can't reach stop falling out of coverage.

Model: Whisper / wav2vec 2.0 (singing-voice fine-tune) → LLM correction, following the LyricWhiz pattern (Zhuo et al., 2023)

listening across 5 tracks · 5 languages detecting…

The coverage gap

Human transcription doesn't scale past the top of the catalog.

Manual transcription gets applied where the ROI is obvious — global-language hits with clear commercial upside. Everything below that line competes for the same finite editor hours: non-English catalog, emerging-market artists, independent releases. It's not that those tracks matter less — it's that they're the most expensive to reach manually, so they wait the longest.

Tier 1 · Global-language, high-catalog-value

92%

Where manual transcription already concentrates. Coverage is a solved problem here.

Tier 2 · Major non-English markets

61%

Large audiences, but editor headcount per language hasn't kept pace with catalog growth.

Tier 3 · Long-tail, emerging-market, independent

18%

Where manual transcription essentially never reaches. This is the gap this tool targets.

Illustrative tiering for discussion purposes — not Apple Music catalog data.

How it works

A signal chain, not a black box.

Every stage produces something an editor could inspect. Nothing gets published without passing through the confidence gate.

1 · Vocal isolation

Source separation (e.g. Demucs) strips the vocal stem from the mix so the ASR model isn't fighting instrumentation.

2 · ASR draft

Whisper or wav2vec 2.0, fine-tuned on singing rather than speech, produces a timestamped first-pass transcript per language.

3 · LLM correction

An LLM reviews the draft using language, genre, and rhyme/meter context to resolve homophones, word boundaries, and mis-hears.

4 · Confidence routing

Low-confidence lines route to a human editor. Editors correct minutes of a track, not full manuscripts from silence.

5 · Feedback loop

Approved editor corrections re-enter fine-tuning data, so error rate on that language/genre keeps dropping over time.

Track credits — under the hood

ASR backboneWhisper large-v3 or wav2vec 2.0 XLS-R, fine-tuned on singing-voice corpora

Vocal isolationDemucs source separation, pre-ASR

Correction modelLLM post-processor, language + genre conditioned (LyricWhiz pattern)

Routing logicConfidence-threshold triage to human editor queue

Feedback loopApproved corrections re-enter fine-tuning data

Human roleVerification and edit, not first-draft transcription

Try it — sample drafts

See the draft before and after correction.

Five original demo tracks, chosen to stress-test the model across script, market tier, and genre difficulty — including the hardest case for ASR: growled metal vocals. Raw drafts below are hand-authored to mimic realistic ASR error patterns. Correction runs live against Claude.

Raw ASR draft

unpunctuated · unedited

After LLM correction

awaiting run

Click "Run correction" to send this draft through the LLM correction stage.

Why this matters

The draft is the unlock, not the transcript.

The editor bottleneck was never finding the songs — it's starting every one of them from zero. In ASR-assisted transcription workflows generally, correcting a draft is consistently faster than transcribing from silence, often by multiples rather than percentages. This tool doesn't remove the editor from the loop; it moves their time from first-draft production to verification, which is where their judgment is actually worth the most.

LyricWhiz (Zhuo et al., 2023) demonstrated that pairing Whisper with an LLM correction stage measurably reduces word error rate against prior methods, including in genres — rock and metal — where ASR historically performs worst. That result is the basis for proposing the same pattern here, tuned to Apple Music's language and genre mix.