ICML 2025

High-Fidelity Simultaneous Speech-To-Speech Translation
Tom Labiausse 1 Laurent Mazar´e 1 Edouard Grave 1 Alexandre D´efossez 1 Neil Zeghidour 1
Abstract
We introduce Hibiki, a decoder-only model for
simultaneous speech translation. Hibiki leverages
a multistream language model to synchronously
process source and target speech, and jointly
produces text and audio tokens to perform speech-
to-text and speech-to-speech translation.
We
furthermore address the fundamental challenge
of simultaneous interpretation, which unlike its
consecutive counterpart—where one waits for the
end of the source utterance to start translating—
adapts its flow to accumulate just enough context
to produce a correct translation in real-time,
chunk by chunk. To do so, we introduce a weakly-
supervised method that leverages the perplexity
of an off-the-shelf text translation system to
identify optimal delays on a per-word basis and
create aligned synthetic data. After supervised
training, Hibiki performs adaptive, simultaneous
speech translation with vanilla temperature
sampling.
On a French-English simultaneous
speech translation task, Hibiki demonstrates
state-of-the-art performance in translation quality,
speaker fidelity and naturalness.
Moreover,
the simplicity of its inference process makes it
compatible with batched translation and even
real-time on-device deployment.
We provide
examples1 as well as models and inference code.2
1. Introduction
We introduce Hibiki (“echo” in Japanese), a system for
streaming and expressive speech-to-speech (S2ST) and
speech-to-text (S2TT) translation. Most work in speech
translation has focused on the offline (or consecutive)
setting where the model has access to the full source
utterance before it translates, as it provides useful context
1Kyutai, Paris, France. Correspondence to:
Hibiki <hibiki@kyutai.org>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(s).
1https:/...