# Comparing Speaker Adaptation Methods for Visual Speech Recognition for Continuous Spanish +
## 2 Related Work  
**Visual Speech Recognition. ** Influenced by the evolution of systems focused on auditory-based ASR, different approaches have been considered in the field [12]. Today, the current state of the art in VSR [11; 21; 22] has shown remarkable advances, achieving a WER of around 25-30% on the challenging LRS3-TED database [19]. As introduced above, this has been possible not only thanks to the availability of large-scale databases [18; 19; 20], but to the design of powerful end-to-end architectures [11; 21; 22]. **Speaker Adaptation Methods. ** Although speaker adaptation has been widely studied in traditional ASR [2; 50; 51], in this section, due to the nature of our VSR system, we only consider those works that have addressed the problem using end-to-end architectures. A simple retraining-based adaptation was adopted in [37] to fine-tune an Attention-based system. In addition, influenced by research on conventional ASR systems [51], a hybrid CTC/Attention model was adapted by incorporating speaker identity vectors [38]. On the other hand, different works [39; 40] have proposed the use of more sophisticated techniques, such as the Kullback-Leibler divergence and Linear Hidden Networks. In our work, we decided to use a retraining-based method, also known as fine-tuning. However, as previously introduced, this approach implies estimating and maintaining a separate model for each speaker to whom it needs to be adapted. Therefore, similar to other works [44; 45; 46], we contrasted this fine-tuning technique to a more recent approach based on the so-called Adapters [44]. Although it was originally proposed in the field of neural machine translation, this parameter-efficient method has recently been applied in ASR. [PERSON] et al. [46] used Adapters to address language- and domain-adaptation tasks, demonstrating that, although there was a slight deterioration in terms of performance, this lightweight method presented several advantages for deployment purposes. Reporting similar conclusions, [PERSON] et al. [45] focused on adaptation to atypical or accented speech while considering hundreds of different speakers. The authors found Adapters to be a feasible and scalable solution for personalized speaker-dependent models, as well as for domain-specific or dialect/accent-focused models. However, most of these studies were conducted in the auditory-based ASR domain. Although this research describes approaches that could be adopted to any speech modality, it is noteworthy that few works have explicitly focused on speaker adaptation for VSR systems. [PERSON] et al. [41] defined an architecture based on the CTC paradigm [23] where, after computing visual speech features, a speaker-specific identity vector was integrated as an additional input to the decoder. [PERSON] et al. [42] approached the problem indirectly, studying how to adapt the visual front end of an audiovisual recognition system. Specifically, the authors proposed an unsupervised method that allowed an audiovisual system to be adapted when only visual data were available. However, unlike our research, these works did not address natural continuous VSR, as their experiments were evaluated  
Figure 1: Scheme of a real application based on speaker-adapted visual speech recognition. on databases recorded in controlled settings. Furthermore, to the best of our knowledge, Adapters have not yet been explored in VSR. **Spanish Visual Speech Recognition. ** Although they tend to lack audiovisual resources [52], languages other than English are beginning to be considered in the field of VSR [22]. This is the case for the Spanish language, which has been the object of study on multiple occasions despite the fact that an evaluation benchmark has not yet been established. [PERSON] and [PERSON] [53] explored diverse approaches over the VLRF corpus [47], achieving around 70% WER in the best setting of their experiments. [PERSON] et al.