Tiny speech-to-text models: Whistle and Phonon-2 compared

Two speech-to-text models came out within a week of each other this autumn, both small enough to run on an ordinary laptop with no graphics card and no internet connection. Whistle, from Cactus Compute, is 17 MB. Phonon-2, from Fermion Research, is 164 MB. For comparison, OpenAI's Whisper base, a common choice for local transcription, is about 145 MB, and the most accurate Whisper models are well over a gigabyte.

Small models matter for the kind of software we make. They can run inside a phone app, a browser tab or a desktop utility, so audio never has to leave the device. Below is what each one does, according to its makers' published numbers, and where those numbers need context.

The two models at a glance

Whistle: 17 MB and seven languages

Whistle runs on Cactus's own engine, which is prebuilt for macOS, Linux, Windows on Arm, Android, iOS, watchOS and the browser. It handles up to 30 seconds of audio per pass and splits longer recordings into windows. It detects the language automatically unless you name one, and it can return a start and end time for every word. The official example is two lines of Python:

import needle
print(needle.transcribe("clip.wav")["text"])

What the benchmarks say

Cactus compares Whistle with the multilingual version of Whisper base. Word error rate (WER) is the share of words a model gets wrong, so lower is better:

Whistle is better on read English and on the FLEURS multilingual set, and worse on talks, meetings and the MLS multilingual set. The English-only Whisper base.en scores 4.1 on LibriSpeech test-clean in the Whisper paper, so for English alone Whistle does not beat the best small Whisper.

The speed claim also needs reading carefully. On an Apple M4 Pro CPU, Cactus reports that Whistle produces its first word 6.6 times sooner than Whisper base and decodes about 5 times faster overall. Whistle ran at 2 to 4-bit precision in that test and Whisper at full 32-bit precision, which favours Whistle.

Two practical notes. The Python package is small, but on first use it downloads the engine and the model, so "no dependencies" applies to the engine rather than the install. And Cactus's GitHub README says the engine sends telemetry by default, switched off with the environment variables NEEDLE_TELEMETRY=0 and DO_NOT_TRACK=1, while the Hugging Face model card says the engine reads no environment variables. If you use Whistle because audio should stay private, check this yourself before shipping it.

Phonon-2: near large-model accuracy at 164 MB

Phonon-2 is NVIDIA's Parakeet TDT 0.6B v3, a 2.5 GB model, compressed to about 2.1 bits per weight and fine-tuned to keep its accuracy. Fermion says it is the most accurate open English model under 900 MB; that figure is the size class, and the download itself is 164 MB.

On the seven English test sets of Hugging Face's Open ASR Leaderboard, Fermion reports:

So it loses a quarter of a point against the model it came from, at one fifteenth of the size, and it is ahead of Whisper's large turbo model on these English sets.

The widely quoted "an hour of audio in about 20 seconds" is Fermion's figure for an M5 MacBook Air using the Mac's GPU. On the CPU alone, Fermion's own figure is 40 times real time, about 90 seconds for an hour of audio, which is still fast.

Things to know before you use it:

• It is English only, although the NVIDIA model it comes from supports 25 European languages.
• The CC-BY-4.0 licence on the weights means you must credit NVIDIA and Fermion if you redistribute them.
• It reads WAV, FLAC, OGG and AIFF but refuses MP3 and M4A, so convert those first with ffmpeg.
• It needs Python 3.10 or newer, and the base install pulls in PyTorch and Transformers, so it is a large install even though the model is small.
• Fermion Research is new, and its website and repositories name no people behind it.

Other small options

These two are not the only small models. Sizes and LibriSpeech test-clean error rates from each project's own papers and pages:

Whisper's numbers here use beam search, its more accurate decoding setting, from Table 9 of the paper.

Which one to use

• English dictation or voice notes inside an app, smallest possible size: Whistle. At 17 MB it fits where nothing else does, and it also covers six European languages.
• The best English accuracy on a laptop: Phonon-2, if you are comfortable with an anonymous publisher and a heavier install.
• The safest, best-known choice: Whisper base.en through whisper.cpp. It is older and not the fastest, but it is MIT licensed, widely used and well understood.
• Meetings with several speakers: none of the small models does well on the AMI meeting set. Use a larger model, or test on your own recordings first.

Every number in this post is the maker's own, measured on clean test sets and on their hardware. Before you choose, record a few minutes of the audio you actually have and try two or three models on it.