# Unsupervised Adaptation of Deep Speech Activity Detection Models to Unseen Domains
## 5 Conclusions and Future Lines  
In this paper, we have explored the use of unsupervised domain adaptation techniques in the context of the SAD task. An initial baseline model was trained on a variety of well-known domains with big amounts of labelled data available. Then, a study was performed on three methods that allow to perform adaptation directly on the model space with the objective of fine tuning the mentioned baseline model using only unlabelled data. We have used the data provided in the FS challenge, coming from a singular domain such as Apollo space missions, to experimentally validate in the SAD task the methods presented. Yet, the methods are general enough so that they can be easily applied to other datasets. Furthermore, no labels are required for them to be used, significantly reducing the constraints for choosing them in practical applications. Through the application of Deep CORAL based domain adaptation methods, results show a 13% relative improvement in DCF metric of the original challenge. Furthermore, the cascaded application of Deep CORAL and pseudo-labelling techniques provides the best results in this study, with a significant 24% relative improvement compared to the baseline system. These experimental results suggest that Deep CORAL and pseudo-labelling techniques are complementary. The first one providing an overall improvement in the DET curve and reducing the EER. The second one improves the AUC value by modifying the DET curve so that its slope becomes constant, specially in areas with high FPR and FNR values. The improvements in performance observed allow to substantially reduce the gap for the SAD task between a system trained using in domain data and an approach based on fully unsupervised adaptation. Even if the knowledge distillation method shows an improvement in performance compared to the unadapted baseline model, this improvement is limited compared to the one observed by CORAL based techniques. This kind of domain adaptation methods, seeking to minimise the statistical distribution shift between source and target domains, provide one of the most promising results in this paper. Some recent work has introduced the use of higher-order statistics for unsupervised domain adaptation [73], generalising the idea presented in Deep CORAL as an arbitrary order moment matching technique. Some of our future work lines may point in this direction, applying this idea to the SAD task. Conceptualisation, P.G. and A.O. ; methodology, P.G. and A.O. ; software, P.G. and A.M.; validation, P.G., D.R., A.O., A.M. and E.L.; formal analysis, P.G., D.R. and A.O. ; data curation, P.G. ; writing--original draft preparation, P.G. ; writing--review and editing, D.R., A.O., A.M. and E.L. All authors have read and agreed to the published version of the manuscript. This work was supported in part by the European Union's Horizon 2020 research and innovation programme under Marie Sklodowska-Curie Grant 101007666; in part by MCIN/AEI/10.13039/501100011033 and by the European Union \"NextGenerationEU\"/PRTR under Grant PDC2021-120846-C41, and in part by the Government of Aragon (Grant Group T36_20R). Author [PERSON] was supported in part by the Government of Aragon with a grant for predoctoral research contracts (2018-2022) co-funded by the Operative Program FSE Aragon 2014-2020. Not applicable. The Albayzin 2010 dataset is available from the corresponding authors on reasonable request. Albayzin 2018 and Albayzin 2020 RTVE datasets are available upon request through [[http://catedrature.unizar.es/albayzin.html](http://catedrature.unizar.es/albayzin.html)]([http://catedrature.unizar.es/albayzin.html](http://catedrature.unizar.es/albayzin.html)) / accessed on 26 November 2021). AMI corpus is publicly available at [[https://groups.inf.ed.ac.uk/ami/download/](https://groups.inf.ed.ac.uk/ami/download/)]([https://groups.inf.ed.ac.uk/ami/download/](https://groups.inf.ed.ac.uk/ami/download/)) (accessed on 26 November 2021).