## 5 Experiments
### Data Used  
The AudioSet, as described by [PERSON] et al. (2019), consists of a large-scale audiovisual dataset of manually annotated acoustic events. Starting from the literature and manual curation, the authors defined a structured hierarchical ontology of 632 audio classes to collect data from human labelers. The goal of their task was to probe the presence of specific audio classes in 10 s segments of YouTube videos. The complete dataset contains over 2 million videos and the labeled segments employ part of the AudioSet ontology. The provided data is characterized by its highly diverse audiovisual content. Dealing with such variety makes our study distinct from most previous works. Despite the fact that we use only five sound classes, the sound-image pairs are extracted from over eight thousand distinct scenes, resulting in substantial inter- and intraclass diversity. For training and testing our S2I translator, we employed a subset of AudioSet named VEGAS, made available by [PERSON] et al. [35] for their study on cross-modal image-to-sound translation. This dataset provides cleaner data, in which the starts and ends of addressed acoustic events are precisely annotated. Additionally, the tracks have been inspected to verify whether the elements/events related to the sound were present in both visual and aural modalities, and non-matching segments were removed. The complete VEGAS dataset consists of 28,109 videos of a maximum duration of 10 s, distributed across ten sound classes, among which we use five: Baby cry, Dog, Rail transport, Fireworks, and Water flowing. The original VEGAS dataset is unbalanced; thus, to prevent class biasing, an equal number of segments is used for all classes. Table 1 presents the number of original video tracks for each sound class, along with their corresponding 1 s segments designated for training, validation, and test sets.  
Regarding audio data, log-mel spectrograms were computed according to the following procedure: the signal was split into 25 ms frames, with a 15 ms overlap; a Hamming window was applied to the frames and the short-time Fourier-transform (STFT) was computed; its squared magnitude was integrated in 128 sub-bands using triangular weights according to a non-linear mel-scale, and the logarithm of those sub-band energies was computed. For a 1 s audio segment, a matrix of 100 \(\times\) 128 was obtained. The segmentation of one of these spectrograms and the respective frames from the original video are illustrated in Figure 5. As for visual data, images were extracted from the central frame of the corresponding 1 s video segment. Before being loaded into the neural network, these extracted images were square cropped at the center and then resized to 96 \(\times\) 96 pixels.  
\begin{table}
\begin{tabular}{c c c c c} \hline \multirow{2}{*}{**Sound Classes**} & \multirow{2}{*}{
\begin{tabular}{c} **No. of Original Video** \\ **Tracks (max. 10 s)** \\ \end{tabular} } & \multicolumn{3}{c}{**No. of 1 s Video Segments**} \\ \cline{3-5}  & & **Training** & **Validation** & **Test** \\ \hline Baby cry & 2059 & 9789 & 1115 & 1365 \\ Dog & 2785 & 9789 & 1115 & 1365 \\ Fireworks & 3115 & 9789 & 1115 & 1365 \\ Rail transport & 3259 & 9789 & 1115 & 1365 \\ Water flowing & 2924 & 9789 & 1115 & 1365 \\ Total & 14,142 & 48,945 & 5575 & 6825 \\ \hline \end{tabular}
\end{table}
Table 1: Sound classes and their respective number of video tracks and 1 s segments for training, validation and test sets.  
Figure 5: Images of 1 s mel spectrogram segments and respective central frames extracted from a video track of the class ‘Rail transport’.