## 2 Materials and Methods
### Feature Extraction  
There are two main DNN architectures: convolutional neural network (CNN) and recurrent neural network (RNN) ([PERSON], 1990). We determined that the CNN approach was more appropriate for the current dataset as we had a limited number of samples from which we aimed to test our model. Previous work by [PERSON], [PERSON] and [PERSON] [32] observed that more complicated neural networks (i.e., RNNs and/or hybrid models) may result in lower accuracy and/or fail to converge when trying to model \"relatively\" small datasets and [PERSON] et al. [29] observed that RNNs are more computationally expensive compared to CNNs, with potentially little increases in accuracy [33]. Therefore we implemented a \begin{table}
\begin{tabular}{c c} \hline \hline
**Name** & **Time (s)** \\ \hline [PERSON] & 95.06 \\ [PERSON] & 103.28 \\ [PERSON] & 100.93 \\ [PERSON] & 101.84 \\ [PERSON] & 97.04 \\ [PERSON] & 97.54 \\ [PERSON] & 99.17 \\ [PERSON] & 100.74 \\ [PERSON] & 105.04 \\ [PERSON] & 107.07 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The total length of audio recordings sampled for each character in the Modern Family series. Figure 2: (**A**) A representative raw audio file from the speaker, [PERSON], before thresholding. (**B**) A representative clean audio file from the speaker, [PERSON], after thresholding. convolutional neural network (CNN) model for voice samples machine learning training, which relied on image feature transformation (see [34] for a similar example). For each cleaned audio file, the spectrogram was extracted using the Mel-Frequency Cepstral Coefficient [35] algorithm, and subsequently formed the image feature used in the CNN model (see [32] for an example of this approach applied to audio data). We had a total of 1009 seconds of audio from the 300 audio files (i.e., 10 voices \(\times\) 30 files). We multiplied 1009 by 50 (the number of epochs to be run) to get 50,450 samples for this experiment (to be divided into the training set, validation set and test set, discussed below). The following steps were used for feature extraction. First, we used the random function in the Python library to pick a random audio file from 300 audio files. Second, we used the audio processor function \"scipy\" library by [PERSON] et al. [36] to read the audio file and return audio data and sample rate (in seconds). Third, we randomly sampled a section of the audio sample with a fixed window size inside the audio file. Fourth, we implemented an open-source code library, namely, \"librosa.feature.mfc\" API [PERSON] et al. [36]; source code can be found through [[https://librosa.org/doc/main/generated/librosa.feature.mfc.html](https://librosa.org/doc/main/generated/librosa.feature.mfc.html)]([https://librosa.org/doc/main/generated/librosa.feature.mfc.html](https://librosa.org/doc/main/generated/librosa.feature.mfc.html)), accessed on 2 June 2021) to extract the spectral features into a matrix. The MFCC method extracts cepstrum from the audio data and the format of cepstrum we chose was 32 pixels by 32 pixels. We set the parameters as follows: sampling rate = 44,100, hop length = 700 n_mfcc = 32, n_fft = 512 (Appendix A). Therefore, the shape of matrix X is (50,450,32,32, 1) with each of the 50,450 cepstrum representing one feature of one person. Third, the CNN model requires the addition of one color channel (gray) to our matrices, as CNN models always train image data, and for images, there are three color channels for one dimension (RGB). We used one-hot encoding [37] to deal with the classes because our classes were not pure numbers but were instead people's names. One-hot encoding aids the machine model to train and predict each class as 0 or 1. For example, if the voice is from [PERSON], and the index of [PERSON] is 5 (start from 0), the one-hot encoding matrix is [0,0,0,0,0,1,0,0,0,0].