## 2 Related Work
### Datasets  
In order to validate audiovisual ASD techniques (and train when required), some data encompassing all considered modalities must be available. Furthermore, such data is required to include some form of labeling so as to enable its application in the aforementioned techniques. Unfortunately, the multi-modal nature and challenges inherent to audiovisual ASD constitute a major setback in terms of acquiring labeled data. As such, small and constrained datasets are frequently built by research teams just to evaluate their own approaches, which of course restricts their comparability and reduces the credibility of their success. Yet very recently a few datasets have been created and made available, which address this current necessity in ASD research. In [2] the AVSpeech dataset was presented, an automatically collected large-scale corpus made up of several lecture recordings available on YouTube. These recordings, amounting to around 4700 h of audiovisual data, always show a single clearly visible face to which the ongoing speech corresponds. This dataset could be labeled for active speaker detection using some voice activity detection (VAD) or speech-silence discerning technique, and then applied for training and testing of ASD approaches. However, this may lead to model confusion later on in multi-speaker scenarios and as such AVSpeech is perhaps best left for ASD testing. A potentially better example of a suitable corpus is given in [15], where the Columbia dataset was first introduced. Here, the recording of a panel discussion between seven speakers was overlapped with upper body bounding boxes for all participants, each labeled with speak/no-speak markers. By always including more than one speaker in each frame, this dataset has the benefit over AVSpeech to enable dealing with multi-speaker situations in inference models. As for its downside, the dataset is of reduced size and its data is considerably homogeneous. This evidently means it may be harder to extrapolate relevant features only and generalize a model trained on this data, which constrains the dataset's usage in machine learning approaches. The largest and most heterogeneous active speaker detection dataset available at the moment is presumably AVA-ActiveSpeaker [25], developed by Google for the 4 th ActivityNet challenge at CVPR 2019. This hand labeled set of segments obtained from 160 YouTube videos amounts to around 38.5 h of audiovisual data, where each of the covered 3.65 million frames is detailed with bounding box locations for all detected speaker faces as well as with three types of labels for each present bounding box--speaking and audible, speaking but not audible, not speaking. Such a corpus should therefore be perfect for learning ASD as it covers all previously mentioned requirements for a suitable audiovisual dataset and presents no major drawbacks. To support this, the team behind AVA-ActiveSpeaker used it to train and test both static and recurrent prediction models, applying modality fusion and consistently achieving great performances. Unfortunately however, this dataset encompasses an extremely high amount of low quality videos in terms of image resolution, audio intelligibility and noise. The creators of the dataset even went so far as to include several voice-over videos and dubbed versions of movies not in their original language. This naturally means a copious amount of audio clips are out of sync with their supposedly respective facial image sequences. Additionally, the lengths of these clips frequently do not pair with the lengths of those facial sequences--due to phrases varying in duration for different languages. Taking this into account each of the 160 videos was manually inspected for quality assessment, as though they may be appropriate for a challenge in the same conditions, it would be senseless to use such great amounts of noisy incorrect data to train a model and expect it to perform successfully with correct data and in the real world.