Whisper-Edge-Tiny

Edge fork of Whisper-Tiny with NPU-friendly attention rewriting. INT4 variant runs on the Pi alone.

Embedded ASRMITINT8INT4asrenglishtiny

412K downloads 25K deploymentsUpdated Mar 18, 2028

Headline:220ms · Raspberry Pi 5 + Hailo HAT · INT8

Overview Benchmarks8 Sim Results Deploy8 Files Discussion23

Cross-chip benchmark matrix

Every supported chip, in matched-pair runs from the Fo’c’sle HIL lab. Sortable by any column — click a header. Cells where the chip can’t run this model show Not supported.

Chip platform	Quant	Latency p50(ms)	Latency p95(ms)	Throughput	Acc. retention(%)	Power(W)	Memory(MB)	Tested by
Apple Neural Engine (M4) 38 TOPS · SoC	INT8	203.5	242.8	31 tok/s	97.7	6.8	46	Fo’c’sle HIL
Q Snapdragon 8 Gen 3 NPU 45 TOPS · SoC	INT8	216.0	260.6	30 tok/s	97.6	6.8	46	Fo’c’sle HIL
Q Qualcomm QCS8550 48 TOPS · SoC	INT8	221.7	262.4	29 tok/s	97.3	13.3	46	Publisher
H Hailo-8 26 TOPS · M.2	INT8	355.9	446.1	18 tok/s	98.1	2.1	46	Publisher
π Raspberry Pi 5 + Hailo HAT 26 TOPS · HAT	INT8	368.8	461.2	17 tok/s	97.7	6.9	46	Fo’c’sle HIL
Q Qualcomm QCS6490 12 TOPS · SoC	INT8	526.3	680.2	12 tok/s	98.2	8.0	46	Fo’c’sle HIL
R Rockchip RK3588 6 TOPS · SoC	INT8	836.5	1063.0	7.7 tok/s	97.5	9.7	46	Fo’c’sle HIL
G Google Coral Edge TPU 4 TOPS · USB	INT8	1008.9	1277.6	6.3 tok/s	98.4	1.7	46	Fo’c’sle HIL
H Hailo-10H 40 TOPS · M.2	Not supported
N NVIDIA Jetson Orin Nano 40 TOPS · SoM	Not supported
N NVIDIA Jetson AGX Orin 275 TOPS · Module	Not supported
N NVIDIA Jetson Thor 2070 TOPS · Module	Not supported
A Ambarella CV5 16 TOPS · SoC	Not supported
A Ambarella CV72 32 TOPS · SoC	Not supported
M MediaTek Genio 700 4 TOPS · SoC	Not supported
I Intel Movidius Myriad X 4 TOPS · SoC	Not supported
A AMD Versal AI Edge VE2302 22 TOPS · SoC	Not supported

Leader per columnLibriSpeech test-clean · 16 kHz streaming

HIL conditions

All numbers measured on Fo’c’sle HIL rigs in Tel Aviv (primary), Munich (secondary), and Pittsburgh (robotics). Single-stream, batch-1, real preprocessing, real downstream consumer. p50/p95 are over 10,000-frame steady-state windows after a 30-second warm-up. Power draw is package power, not wall power. Memory footprint is the resident model + activations footprint at peak — not on-disk.

Submitted publisher numbers are accepted only if they reproduce within ±8% of an HIL-lab matched run on the same chip in the same input mode. Otherwise they live separately under the Discussion tab.