## 2 Materials and Methods
### Proposed Model
#### 2.5.1 Implementation Detail  
Firstly, in the context of audio augmentation, AudioLDM [30] was employed, which is a diffusion-based text-to-audio model. This model takes real audio as the initial input and a text prompt as a condition to generate augmented audio. The process is illustrated in Figure 2, where different text prompts are employed to generate different variations of augmented audio. To make these textual prompts, we utilized ChatGPT [29], as outlined in Table 1. The combination of AudioLDM and ChatGPT enables the generation of diverse and customized augmented audio samples for enhanced audio data augmentation. In this research, we empirically introduce a set of hyperparameters for text-to-audio diffusion models that control the diversity of the generated audio. The specific values for these hyperparameters can be found in Table 2.  
With regard to the proposed model in this paper, when working with a pair of original audio and diffusion-based augmented audio, they were initially pre-processed to a log-scaled Mel spectrogram with a sampling frequency of 16,000 Hz, window size of 1024, hop size of 512, and Mel-spaced frequency bins 128 in the range 50-8000 Hz. Then, they were tokenized as \(a=[a_{1},a_{2},\ldots,a_{512}]\) and \(aug=[aug_{1},aug_{2},\ldots,aug_{512}]\). The next step involved masking a specific portion (50-75%) of the original and augmented audio data, denoted as \(a\) and \(aug\), respectively. In the subsequent step, we exclusively inputted \begin{table}
\begin{tabular}{c c} \hline \hline
**CharacterP Prompts: Different Factory Noises** \\ \hline “add sharp and metallic screeching sound created by metal-cutting saws in a fabrication \\ workshop as they cut through various metal materials” \\ \hline “add rhythmic and pulsating buzzing sound produced by printing presses in a publishing house \\ as they rapidly print and collate pages of books and magazines” \\ \hline “add continuous and powerful rumbling noise accompanied by the grinding and crushing of \\ rocks in a mining quarry as large crushers pulverize rocks into smaller fragments” \\ \hline “add continuous and rhythmic clicking noise produced by injection molding machines in a plastic \\ manufacturing facility as they shape molten plastic into various products” \\ \hline “add rhythmic and repetitive whirring noise accompanied by the sound of cutting tools in a \\ woodworking shop as they shape and carve wood materials” \\ \hline “add continuous and powerful whirring noise accompanied by the sound of pneumatic tools in \\ an automobile assembly line as vehicle parts are fastened” \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Samples of retrieved prompts from ChatGPT. \begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter Name** & **Value** \\ \hline The sampling step for DDIM (DDIM\_STEPS) & Random (10, 50) \\ \hline The duration of the samples (DURATION) & 10 s \\ \hline Random seed (SEED) & Random for each sample \\ \hline Text prompt for audio generation (TEXT) & add (noises) exist in (factory) \\ \hline Guidance scale (GUIDANCE\_SCALE) & 2.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: AudioLDM hyperparameters. Figure 2: Scheme of the suggested method for audio data augmentation. the unmasked tokens, represented as \(a_{unmask}\), into an audio encoder, resulting in the output \(a^{\prime}_{unmask}\). The encoder and decoders are based on ViT [14], a model for image classification that employs a Transformer-like architecture over patches of the image, Ref [50] with a depth of 12 layers and 6 heads. The proposed method comprises two branches. In the first and second branches, the outputs of the encoder \(f_{\theta}\) are mapped onto the embeddings \(R^{512}\)-dimensional space, through linear projection layers \(z_{a}=proj(f_{\theta}(a))\) and \(z_{aug}=proj(f_{\theta}(aug))\), then a Variance--Covariance regularization calculated separately on each \(z_{a}\) and \(z_{aug}\) and NCC contrastive loss is also computed based on both \(z_{a}\) and \(z_{aug}\).