  Self-Supervised Learning (SSL) using huge unlabeled data has been
successfully explored for image and natural language processing. Recent works
also investigated SSL from speech. They were notably successful to improve
performance on downstream tasks such as automatic speech recognition (ASR).
While these works suggest it is possible to reduce dependence on labeled data
for building efficient speech systems, their evaluation was mostly made on ASR
and using multiple and heterogeneous experimental settings (most of them for
English). This renders difficult the objective comparison between SSL
approaches and the evaluation of their impact on building speech systems. In
this paper, we propose LeBenchmark: a reproducible framework for assessing SSL
from speech. It not only includes ASR (high and low resource) tasks but also
spoken language understanding, speech translation and emotion recognition. We
also target speech technologies in a language different than English: French.
SSL models of different sizes are trained from carefully sourced and documented
datasets. Experiments show that SSL is beneficial for most but not all tasks
which confirms the need for exhaustive and reliable benchmarks to evaluate its
real impact. LeBenchmark is shared with the scientific community for
reproducible research in SSL from speech.
