ICML 2025

The Brain’s Bitter Lesson:
Scaling Speech Decoding With Self-Supervised Learning
Dulhan Jayalath 1 Gilad Landau 1 Brendan Shillingford 2 Mark Woolrich3 Oiwi Parker Jones 1
Abstract
The past few years have seen remarkable progress
in the decoding of speech from brain activity,
primarily driven by large single-subject datasets.
However, due to individual variation, such as
anatomy, and differences in task design and scan-
ning hardware, leveraging data across subjects
and datasets remains challenging. In turn, the
field has not benefited from the growing number
of open neural data repositories to exploit large-
scale deep learning. To address this, we develop
neuroscience-informed self-supervised objectives,
together with an architecture, for learning from
heterogeneous brain recordings. Scaling to nearly
400 hours of MEG data and 900 subjects, our ap-
proach shows generalisation across participants,
datasets, tasks, and even to novel subjects. It
achieves improvements of 15-27% over state-of-
the-art models and matches surgical decoding
performance with non-invasive data. These ad-
vances unlock the potential for scaling speech
decoding models beyond the current frontier.
1. Introduction
In his Bitter Lesson, Richard Sutton argues that a major
conclusion of 70 years of AI research is that general methods
exploiting large-scale computation will outperform model-
based approaches as the availability of compute increases
(Sutton, 2019). The ability of deep learning to learn from
ever-larger datasets has enabled seemingly arbitrary scaling
with computation, leading to astounding advances across
a diverse set of domains (Jumper et al., 2021; Caron et al.,
2021; OpenAI, 2023; Radford et al., 2023).
In the domain of brain data, and of tasks like speech de-
coding, the bitter lesson has not yet been fully assimilated.
1PNPL
/ 3OHBA, University of Oxford 2Google DeepMind.
Correspondence to: <{dulhan, oiwi}@robots.ox.ac.uk>.
Proceedings of the 42 nd International Conference on...