# Augmentation Embedded Deep Convolutional Neural Network for Predominant Instrument Recognition
## 3 The Proposed AEDCN Methods  
In many predominant instrument recognition models that rely on data augmentation, a commonly used approach involves three stages. In the first stage, the instrument representation is constructed using the original music audio. In the second stage, data augmentation methods, such as WaveGAN, are applied to generate additional instrument representations for training purposes. In the third stage, deep neural networks are built to map the augmented data to instrument labels. While these data augmentation methodsaddress issues related to low recording quality and improve the effectiveness of music instrument recognition, they may introduce inconsistencies when the data augmentation process is independent of the recognition model. Briefly, some augmented data in a particular instrument class may be redundant, while the required data for another instrument class remain insufficient. Moreover, a single data representation format limits the expression ability of the features extracted by the deep neural networks. To mitigate these issues, the proposed Augmentation Embedded Deep Convolutional neural Network (AEDCN) introduces tempogram of instruments to construct a combined two-channel representation with mel-spectrum for further capturing unique rhythm patterns of different types of instruments and incorporates an Adversarial Conditional Embedded Variational Autoencoder (ACEVAE) between the added fully connected layers within a multi-task label augmentation-based backbone instrument recognition network. The ACEVAE is designed based on an introduced adversarial loss to generate specific data according to designated labels, enhancing the data augmentation process according to the recognition results. To further improve the recognition ability, we add a pre-training process based on introduced datasets for the predominant instrument recognition task. The AEDCN also incorporates label-based data augmentation into its training process. Firstly, a multi-task deep instrument recognition neural network with two fully connected layers before the classification layer is designed as the backbone. This backbone network firstly pre-trains on predominant instrument recognition datasets and then is trained using the provided dataset for instrument recognition. Once the recognition training on the entire dataset is completed, the AEDCN addresses the issue of inconsistency based on their labels and the training results. To tackle this problem, the AEDCN utilizes the features extracted from the first fully connected layer to train the ACEVAE model. The ACEVAE is trained to generate diverse features for specific classes based on designated labels and a designed adversarial loss. By generating fully connected features using ACEVAE, the AEDCN increases the number of samples in the imbalanced class until it reaches the maximum among all classes. Next, the generated features are used to further train the sub-network from the first fully connected layer to the classification layer of the multi-task predominant instrument recognition backbone neural network. This additional training enhances the classification capability for the imbalanced instrument classes. Finally, the well-trained AEDCN model is applied to the testing dataset to verify its effectiveness in improving the overall recognition performance. This approach effectively addresses the issue of data imbalance and inconsistency to ensure that all instrument classes receive sufficient training, leading to enhanced recognition performance. The structure of the AEDCN, including the incorporation of the ACEVAE, is illustrated in Figure 1.  
In AEDCN, the backbone is a multi-task convolutional neural network and the embedded data augmentation network is labeled in blue background. This section provides a  
Figure 1: The structure of AEDCN. In AEDCN, the backbone is a multi-task convolutional neural network. The parts in blue background are the main contributions of this proposed AEGCN model.