## 4 Experiments
### Experiment Analysis  
To verify the effectiveness of the proposed MMGAT model, we compare our instrument recognition results using the OpenMIC-IRMAS dataset, and the instrument recognition results are shown in Table 3.  
Table 3 systematically benchmarks instrument recognition methodologies across three paradigms: feature-engineered systems, spectrogram-driven neural architectures, and synthetic data-augmented frameworks. Traditional approaches include Bosch et al. 's SVM classifier with handcrafted features and MTF-DNN's shallow neural network applied to engineered features, while modern architectures like Audio DNN demonstrate the superiority of raw audio CNNs over conventional methods. The mel-spectrogram-optimized ConvNet introduces temporal max-pooling, later extended by multi-task ConvNet with auxiliary classification to refine labels. These two models use spectrograms and enhanced spectrograms as inputs and train the model using VGG neural networks. The complexity of these models is related to the structure of the neural networks, specifically the complexity of the VGG network. Data augmentation strategies diverge into generative and latent space approaches: WaveGAN ConvNet and \begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **F1 Micro** & **F1 Macro** \\ \hline SVM [25] & 0.33 & 0.24 \\ [PERSON] et al. [26] & 0.46 & 0.40 \\ MTF-DNN (2018) [14] & 0.30 & 0.26 \\ Audio DNN [27] & 0.52 & 0.50 \\ ConvNet (2017) [15] & 0.59 & 0.51 \\ Muti-task ConvNet (2020) [10] & 0.61 & 0.56 \\ [PERSON] et al. (2022) [8] & 0.60 & 0.53 \\ WaveGAN ConvNet (2021) & 0.60 & 0.60 \\ Voting-Swin-T (2022) & 0.60 & 0.60 \\ Staged trained ConvNet & 0.60 & 0.60 \\ VAE augmentation ConvNet & 0.61 & 0.60 \\ AEDCN (2023) [28] & 0.61 & 0.61 \\ MMGAT & **0.63** & **0.62** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance of MMGAT and baselines. Voting-Swin-T both synthesize training data via WaveGAN but diverge in using CNNs versus transformer-based recognition, while VAE augmentation ConvNet leverages variational autoencoders for Melody-solos-DB latent space interpolation. Advanced hybrid systems include the staged-trained ConvNet's cross-dataset curriculum learning and AEDCN's adversarial ACEVAE architecture embedded in fully connected layers for joint data-label augmentation. These models all incorporate generative models, and their complexity is not only related to the discriminative model but also to the generative model, which typically uses multi-layer VAE and GAN structures. The discriminative models utilize VGG convolutional networks and attention-based neural networks. The input for these models is either spectrograms or enhanced spectrograms. Notably, the proposed MMGAT achieves state-of-the-art performance, surpassing all neural and augmentation-enhanced baselines in Table 3 through multi-modal polyphonic interaction modeling. MMGAT directly utilizes the VGGish model to extract features from each audio data input. The process involves loading the pre-trained VGGish model, providing the local path of our audio data, and the pre-trained VGGish model directly outputs the features corresponding to the audio (for a 10 s audio clip, a 10*128 feature matrix is generated). Therefore, the input to the MMGAT model consists of the features extracted by VGGish. VGGish maps an audio clip of a length of k seconds to a k*128-dimensional matrix and then constructs the corresponding instance graph from k*128-dimensional vectors. CLIP is used to map labels to word vectors. In the subsequent recognition process, we have built an attention model with three layers and three scales, each with only 128-dimensional hidden features. Label semantic embeddings are achieved using an autoencoder constructed with a multi-layer neural network, where both the encoder and decoder have two layers each. As a result, the overall parameter count of the network is relatively small, enabling faster training.