# Verse1-Chorus-Verse2 Structure: A Stacked Ensemble Approach for Enhanced Music Emotion Recognition
## 2 Methodology
### Development of Stacked Ensemble Model
#### 2.2.2 Dual Input Stacked Ensemble Model for Datasets Verse1, Chorus, Verse2, and Whole Song  
The dual input stacked ensemble model, shown in Figure 5, was created to determine the model performance for the datasets verse1, chorus, verse2, and the whole song dataset for comparison purposes. Each network is an Artificial Neural Network that encompasses multiple hidden layers, up to six layers, with ReLu activation functions employed throughout, culminating in softmax activation at the output layer. Figure 5 depicts the architecture of the basic stacked ensemble method model, which was used as the model to run individually the datasets for verse1, chorus, verse2, and whole song. In this model, predictions from two base models serve as input for a meta-learner, aiming to surpass the performance of any individual base model. The figure showcases a dual-base model architecture, where audio and lyrics datasets are processed independently. Each base model comprises an artificial neural network (ANN) with up to six hidden layers,  
Figure 5: Dual Input Stacked Ensemble Model used for Versel, Chorus, Verse2, and Whole song Datasets. utilizing the ReLU activation function. These models culminate in an output layer with softmax activation, a common choice for multi-class classification tasks. Concatenation through Predictions of Dual Input Stacked Ensemble Model  
The predictions of the different base models are concatenated to form a new feature vector \(Z_{i,\_part}\), expressed as  
\[Z_{i,\_were1}=concat\ \Big{(}p_{i,\_audio,\_we2},\ \ P_{i,\_lyrics,\_were1}\Big{)} \tag{5}\]  
\[Z_{i,\_chorus}=concat\ \Big{(}p_{i,\_audio,\_chorus},\ \ P_{i,\_lyrics,\_chorus}\Big{)} \tag{6}\]  
\[Z_{i,\_were2}=concat\ \Big{(}p_{i,\_audio,\_we2},\ \ P_{i,\_lyrics,\_were2}\Big{)} \tag{7}\]  
\[Z_{i,\_whelosong}=concat\ \Big{(}p_{i,\_audio,\_wholesong},\ \ P_{i,\_lyrics,\_wholesong}\Big{)} \tag{8}\]  
where \(concat()\) function appends the prediction vectors for each part to form the new feature vector \(Z_{i,\_part}\), which serves as the input to the meta-model. \(Z_{i,\_part}\), generates four probability values derived from the individual predictions for each dataset: four values for each audio segment and four values for each lyrics segment. These predictions are then subsequently fed into a meta-learner. A sample of prediction for the first entry in the verse1 audio dataset generates the following values for each segment of the song: [0.0003, 0.0189, 0.0004, 0.9804]. The sequencing of class values represents the \"Angry\", \"Happy\", \"Relax\", and \"Sad\" classes in order. The high probability of 0.9804, 0.9996, 0.9849 indicates that the model predicts the song is \"Sad\". The other probabilities represent the likelihoods of the other classes. Another sample prediction for the first entry in the verse1 lyric's dataset generates the following values for each segment of the song: [0.0053, 0.0146, 0.4644, 0.5157]. Each vector contains four values representing the four emotions in this study. Concatenation then happens by appending all the above values in the verse1 segment. Therefore, the concatenated values are now [0.0003, 0.0189, 0.0004, 0.9804, 0.0053, 0.0146, 0.4644, 0.5157], which will be used as input into the meta-learner model. The rationale for concatenating the probability distributions from both audio and lyrics models in our study stems from several key advantages. First, this approach enhances classification accuracy by leveraging the complementary strengths of each modality. By considering both audio and lyrics, the model can capture nuances and contextual cues that may be missed when analyzing either modality alone, thereby improving overall classification accuracy. Second, concatenating probabilities enables comprehensive emotion detection by allowing the model to account for emotional cues present in both music and lyrics. This holistic understanding of the song's emotional content leads to more nuanced and accurate predictions.