# Verse1-Chorus-Verse2 Structure: A Stacked Ensemble Approach for Enhanced Music Emotion Recognition
## 2 Methodology
### Development of Stacked Ensemble Model
#### 2.2.3 The Six Input Stacked Ensemble Model  
Figure 2 shows a 6-input stacked ensemble model tailored to discern emotional states from music by individually utilizing the audio and lyrics datasets from various song segments--Verse 1, Chorus, and Verse 2. This sophisticated model processes each segment through dedicated audio and lyrics base models as shown in Figures 3 and 4, both being ANNs with up to six hidden layers featuring ReLU activation functions. This architecture adeptly captures complex patterns while addressing the vanishing gradient problem often encountered in deep learning tasks. The outputs from these base models undergo softmax activation, enabling multi-class classification to predict the emotional tone of the song accurately.  
Concatenation Process for Six Input Stacked Ensemble Model  
The predictions of the different base models are concatenated to form a new feature vector \(z_{i}\), expressed as  
\[Z_{i}=concat\ \left(p_{i,audio,\,\text{\it versel}},\ \ P_{i,\text{\it lyrics,\,\,\text{\it verse 1}}},\ \ P_{i,\text{\it audio,\,\,\text{\it chorus}}},\ \ P_{i,\text{\it lyrics,\,\,\text{\it chorus}}},\ \ P_{i,\text{\it audio,\,\,\text{\it verse2}}},\ \ P_{i,\text{\it lyrics,\,\,\text{\it verse2}}}\right) \tag{9}\]  
where \(concat()\) function appends these six prediction vectors to form the new feature vector \(Z_{i}\), which serves as the input to the meta-model. \(Z_{i}\), generates 24 probability values derived from the individual predictions for each dataset: four values for each audio segment (verse1, chorus, verse2) and four values for each lyrics segment (verse1, chorus, verse2), respectively. These predictions are then subsequently fed into a meta-learner.  
A sample of prediction for the first entry in the audio dataset generates the following values for each segment of the song: [0.0003, 0.0189, 0.0004, 0.9804], [0.0000, 0.0000, 0.0004, 0.9996], [0.0118, 0.0004, 0.0029, 0.9849]. The sequencing of class values represents the \"Angry\", \"Happy\", \"Relax\", and \"Sad\" classes in order. The high probability of 0.9804, 0.9996, 0.9849 indicates that the model predicts the song is \"Sad\". The other probabilities represent the likelihoods of the other classes.  
Another sample prediction for the first entry in the lyric's dataset generates the following values for each segment of the song: [0.0053, 0.0146, 0.4644, 0.5157], [0.0015, 0.0003, 0.9973, 0.0009], [0.0005, 0.9979, 0.0001, 0.0016]. Each vector contains four values representing the four emotions in this study.  
Concatenation then happens by appending all the above values in the order according to the segments verse1, chorus, verse2. Therefore, the concatenated values are now [0.0003, 0.0189, 0.0004, 0.9804, 0.0053, 0.0146, 0.4644, 0.5157, 0.0000, 0.0000, 0.0004, 0.9996, 0.0015, 0.0003, 0.9973, 0.0009, 0.0118, 0.0004, 0.0029, 0.9849, 0.0005, 0.9979, 0.0001, 0.0016], which will be used as input into the meta-learner model.  
By concatenating the probability distributions rather than raw data, we allow the meta-model to leverage the insights captured by each base model, which has been specifically trained to extract meaningful patterns from audio and lyrics data separately. This approach focuses on the most relevant information for classification.  
From a musical perspective, combining audio and lyrics data is crucial because both elements contribute uniquely to the emotional expression of a song. Audio features capture the tonal and acoustic characteristics, such as melody, harmony, and rhythm, while lyrics provide semantic content and contextual meaning. By concatenating the probability distributions from both modalities and the segments of a song, our model can leverage the strengths of each, leading to a more accurate and nuanced understanding of the song's emotional state. This integrated approach aligns with how humans perceive music, considering both the auditory and lyrical components in tandem.