# Beyond Spectrograms: Rethinking Audio Classification from EnCodec's Latent Space
## 2 Methodology
This approach is particularly important in scenarios where pauses or silence are treated as distinct classes for classification. Assigning artificial silence could interfere with the accurate differentiation of speech and non-speech segments, as natural pauses should be preserved to ensure the model correctly distinguishes between speech and pauses without artificial influence. The model incorporates a preprocessing layer that transforms the segmented fragments into images through the mel spectrogram representation. This representation is obtained using a 512-frames Hamming window, equivalent to 25 ms, with a hop size of 128 samples, and 128 mel bins. The final shape of these images is computed using the expression described in (1):  
\[(W,H)=\left(\left\lfloor\frac{(sample\_rate\times window)-sft\_win\_length}{ sft\_hop\_length}\right\rfloor+1,sft\_n\_mels\right) \tag{1}\]  
where \(W,H\) are the width and the height of the resulting image, respectively. Conversely, in the case of preprocessing with EnCodec, a different strategy is followed. Instead of converting the audio to mono format and generating the mel spectrogram, the audio file is first fully compressed using this system. This compression provides a more compact representation in the form of an array with dimensions \((128,75\times N)\) for EnCodec mono at 24 kHz, and \((128,150\times N)\) for EnCodec stereo at 48 kHz, where \(N\) represents the duration of the audio in seconds. However, this compressed representation also requires additional processing. Once the compression is obtained, a sliding window is applied to the resulting array, similar to the mel spectrogram approach. The new representation takes the form \((N/WINDOW,128,\)\(75\times WINDOW)\) for the EnCodec mono case and \((N/WINDOW,128,150\times WINDOW)\) for EnCodec stereo. This means that the compressed audio is segmented into fixed-size fragments, where each fragment contains \(WINDOW\) seconds of information. Unlike the mel spectrogram, the EnCodec approach leverages the compressed information directly, reducing the size of the data to be processed, which can be advantageous in scenarios where computational efficiency is a priority. However, this strategy may sacrifice some of the spectral detail present in the mel spectrogram, which could impact the model'sperformance depending on the specific task. Thus, the choice between one method or the other will depend on the balance between compression, efficiency, and the quality of the represented information. To simplify the data loading process during model training and evaluation, an efficient process has been designed to streamline the loading and organization of audio data for deep learning tasks. This process automatically assigns the folder name containing each file as its class, thereby simplifying the data structure and eliminating the need for separate label files. The method organizes audio files into folders, with each folder named after the class of the files it contains, simplifying preprocessing, reducing errors, and speeding up data loading, especially for large datasets. An automated script iterates through the folders, identifies files in formats such as WAV, MP3, or FLAC, and assigns the corresponding label, while validating file properties such as sampling rate and duration, among others. This approach simplifies audio classification experiments and is adaptable to other scenarios such as speaker identification or sound event detection. Additionally, it allows for the scalable addition of new classes by incorporating additional folders into the dataset. Figure 3 illustrates how these processes operate depending on the type of algorithm used for model creation. The output layer of the neural network is a softmax layer with a series of nodes, each representing a different audio class. This output layer will have two nodes if the classification task is binary (e.g., speech or music in the GTZAN music/speech dataset [14]).