# Intelligent Classification Model of Music Emotional Environment Using Convolutional Neural Networks

Authors: Feng Ke
Journal: Journal of Environmental and Public Health
 (2022)

---

## Abstract

The majority of traditional text sentiment classification techniques rely on machine learning or sentiment dictionaries, but these approaches have the drawback of sparse data and ignore word semantics and word order information. A convolutional neural network- (CNN-) based music emotion classification model is proposed in this paper to address the aforementioned issues. The model in this paper has clear advantages in every way. On the same dataset, the model in this study has an average accuracy of 91.4 percent, while LeNet, AlexNet, and VGGNet have accuracy averages of 75.3 percent, 72.2 percent, and 79.4 percent, respectively. The error value of the other three algorithms is higher than the cost function value because people’s emotions in the cognitive field are divided into different categories. However, in the field of music emotion retrieval, we can only extract the features of the known melody and then search for the same emotion, so we need to build a computerized music emotion classifier if we want to find emotions that are similar to a particular melody. This study examines musical emotion models that already exist, then extracts musical emotion features, and builds a musical emotion classifier using a neural network. The classifier is then further trained until the error classification rate of the training samples is within a certain error range, after which the classification results are marked by pertinent feedback.


---

## Body

## 1. Introduction

Different users have varying levels of love for music for each emotion, and music emotion classification is a crucial component of music information retrieval. No matter how intensely felt, a song’s composition is very intricate, accompanied by a variety of instruments, the variety of vocals is also extremely important, and the harmony of the various elements is constantly shifting. The user experience can be enhanced, and the retrieval time for users’ favorite music can be effectively decreased by building a good music classification system [1]. The majority of early music emotion classification relied on expert listening annotations, which was undoubtedly labor- and time-intensive. The acoustic features that can be used after the invention of computer-aided technology are initially decided by artificial judgement, and these features are extracted from music to train a classifier in order to achieve music emotion classification. It is challenging to increase accuracy with this kind of method because it is unstable and necessitates manually designing feature sets, which relies to some extent on professional expertise and personal experience. Given that CNN has the advantages of directly ingesting data and automatically extracting features without manual intervention, and the multilayer CNN has stronger fitting ability, it can offer helpful theoretical guidance for the challenging problem of feature extraction [2, 3] in music emotion classification. This will help improve the limitations of traditional music emotion classification methods. In order to address the issues of error, omission, incorrect classification, slow processing speed, low efficiency, and other issues, a music emotion classification method [4] based on CNN is proposed in this paper.Theoretically, harmonic composition primarily determines the timbre of sound waves, and various musical instruments have varying timbre characteristics. For instance, the cello’s deep, heavy timbre makes it simple to stifle the emotional influence of sadness; the flute’s loud, crisp timbre makes it simple to convey a cheerful emotional impression. The tempo of music is referred to as speed. If a different tempo is used, even if it is the same piece of music, people will experience the music differently emotionally. The weights of the neurons in each layer are changed in this study to improve the accuracy of the music emotion classification through the learning process of forward propagation and backpropagation. The happier the emotion, the faster the beat; the sadder the emotion, the slower the beat. Numerous traditional machine learning techniques have shown success in classifying the emotions present in music using standard data sets, but these techniques rely heavily on artificial features created by specialists in the field, which raises the bar for nonspecialists. Some features cannot be easily transferred to other fields because they are not universal. The method of using a deep learning model has started to show up in music emotion classification tasks due to the widespread use of deep learning models in other fields [5, 6]. However, the current approaches still have issues with model complexity, accuracy, and model training.As the foundation for searching and recommending music, the systematic project of music emotion classification is extremely significant. In order to describe music information, we must first extract music features based on the classification model of musical emotion. It can be broken down into a variety of categories, including features in the energy, time, and frequency domains. Music can also be categorized based on the combination of various features. To a certain extent, these methods can categorize music, but each has flaws of its own. Only simple audio files can be classified using rules. A standard pattern must be established in order to match a pattern, which requires a lot of calculation and is not very accurate. In any case, since the process of pattern recognition is at the core of music classification, it is possible to use this method. Data should first be collected from audio files, followed by the selection of extracted features and models based on the characteristics of the data, the interception of some data to train the classifier, and finally, the adjusting and determining of the classifier’s parameters based on the test results. The classifier is one of them and is essential to the classification outcome. In the areas of speech analysis and image recognition, CNN clearly outperforms the conventional deep neural network structure. Convolution, pooling, and partially connected network properties in its primary structure enable it to perform exceptionally well during training.This paper’s chapters are organized as follows: a music emotion classification model based on the CNN model is established in the third section of this paper. The fourth section compares this algorithm with LeNet, AlexNet, and VGGNet, and the fifth section is the full text summary. The first section of this paper introduces the related research of related scholars on music emotion classification.The originality of this paper is as follows: this study designs an experiment and contrasts the effects of various audio clips on classification. The experiment demonstrates that this model outperforms many classification models that combine traditional machine learning models with artificial features created by domain experts. The final emotional classification task of lyrics is carried out by designing various layers of structures in CNN, which is fed a matrix of generated feature vectors as its input.


## 2. Related Work

Music is a form of art and a cultural activity that uses sound as its medium. The traditional classification of musical works is based on musical emotion. Although there is not a clear line separating the traits of various musical emotions, songs that share the same musical emotion have traits in common. Humans are able to categorize a large number of musical works emotionally by analyzing these traits. The professional bar for producing various artistic works, such as literature, photography, and music, has gradually lowered in recent years as Internet and multimedia technology has become more widely used. As a result, an increasing number of people are now involved in the production, appreciation, and dissemination of various artistic works. The amount of music is increasing rapidly, making it increasingly difficult for professionals to analyze and categorize it using the conventional methods. The workload of professionals can be significantly reduced, and classification accuracy can be increased, by using computer programs to automatically categorize musical emotions.Gong et al. put forward an emotion analysis method of cross-media word-wrapping model, which can achieve better results than Bayesian and SVM without the assumption of conditional dependence, but cannot deal with the problem that the text information contains the same emotion words as class labels. This kind of sentiment analysis method is based on a single classifier. When the text record is short, and colloquial phenomenon is serious, the recognition rate of sentiment polarity is low [7]. Ya-Nan et al. proposed SVM emotion recognition and classification method based on ensemble learning, which better avoided the problem of low accuracy of single recognizer. Because of the high complexity of SVM itself, its complexity as the base classifier of ensemble learning is higher, so the training time is longer [8]. Wang et al. put forward a sentiment analysis method based on phrases. By extracting phrases from sentences for sentiment recognition, we can accurately judge whether expressions are neutral, polar, or ambiguous, but this algorithm tends to ignore sentence details [9]. Xiang proposed using principal component analysis (PCA) and grammar-based model to extract and select the features of texts and then combined it with the learning framework of support vector machine to classify them. They verified and evaluated the models on corpus data sets [10]. Lin et al. put forward a research method of text emotion classification based on semantic understanding. They consider the emotional semantics of emotional words and redefine the similarity of conceptual emotional words to get the sum and semantic values of emotional words [11]. Zhang and Sun extracted spectrum features from audio signals to train multiple one-to-many SVMs and then built a classification tree. Then, the scores of leaf nodes were used as feature vectors to train KNN classifier, and the final classification accuracy was obtained. The test shows that the classification accuracy of this hierarchical classification model is improved by 7% [12]. Nag et al. use Gaussian process and support vector machine to study different features, including MFCC, linear prediction coefficient, timbre features, and their various combination features and then use them for music genre classification and Valence-Arousal sentiment estimation. From their experiments, it can be seen that the classification result of GP method is indeed better than that of SVM method, but the algorithm complexity of GP method is higher than that of SVM method, so it is difficult to apply it in large-scale tasks [13]. Lü et al. spliced the features related to rhythm, dynamics, timbre, pitch, and tone into 38-dimensional musical features and used the method based on depth Gaussian process for musical emotion recognition. They build a GP regression for each emotion category and use regression to classify musical emotions. Although this method achieves a good effect of emotion classification, it cannot expand the music sample after the model training is completed [14]. Liu et al. put forward a deep learning model, which uses music signal feature spectrogram as music feature input and uses CNN and recurrent neural network to extract features and classify emotions [15]. Donglim and Kim used analytic hierarchy process (AHP) to study the influence of the weights of feature words in different positions on the classification of musical emotions. Through analytic hierarchy process (AHP), this paper studies the influence of the position information of characteristic words in lyrics on the classification of musical emotions. By mixing and fusing with audio information, it is imported into the deep confidence network for supervised training, thus realizing the intelligent classification algorithm of music emotion [16]. Qiang and Liu put forward a music emotion classification model based on improved voting mechanism. Their method is improved on Max Vote, and a voting mechanism based on the emotional distribution characteristics of song fragments is put forward for emotion discrimination. This method effectively avoids the shortcomings of low accuracy and unobvious features of the classifier for emotion classification of song fragments [17].The majority of the current research is conducted by combining artificial features based on subject-matter expertise with classifiers, and it is discovered in this paper by consulting pertinent literature. Artificial feature design based on expert knowledge is difficult, the threshold is high, and such features require a lot of experience and practice in the relevant field. There are numerous drawbacks in practical use because it frequently works well only for a particular kind of task. In this study, a novel CNN architecture is created for the task of music emotion classification, and the benefits and drawbacks of existing models in the field of music information retrieval are analyzed and summarized.


## 3. CNN Technology

CNN is a unique type of artificial neural network. Each neuron represents a distinct nonlinear output function, and as a signal passes through a neuron, it experiences nonlinear transformation. The biological visual system’s architecture served as inspiration for CNN. The visual cortex’s neurons only react to localized stimulation of particular regions, meaning that they receive information locally. The linkage between two neurons symbolizes the signal’s weight as it travels through the linkage. While there is no connection between the neurons in each layer, all the neurons in the adjacent layers are interconnected, meaning that the information that the neurons in the current layer receive is related to all the information of the neurons in the layer before. In CNN, the neurons between adjacent layers are only partially connected, unlike in a typical ANN, in accordance with this theory. With the help of this feature, the weight transfer matrix, which depicts the connections between network layers, can be made smaller. CNN also has a feature known as weight sharing, which means that each neuron’s information in the current layer is calculated using the same weight matrix-convolution kernel and submatrix of the same size as in the previous layer. The weight sharing feature considerably reduces the size of the weight transfer matrix and speeds up neural network training. The structure of CNN is shown in Figure1.Figure 1
CNN structure.Convolution layer is an important component of CNN, and its two key ideas are local connection and weight sharing. Through the local connection and weight sharing of the convolution layer and the downsampling process of the pool layer, CNN greatly reduces the number of parameters and increases the translation invariance to a certain extent. At the same time, because CNN supports the high-dimensional matrix input, which is difficult for neural networks to support, the space vector of the convolution layer is shown in the following formula:(1)a1:d=ai1,ai2,ai3...,ai4,i∈1,d.Transform musical emotional features into a matrix of linear space, as shown in the following formula:(2)A=a11a12...a1va21a22...a2v.........ad1ad1...adv.The feature matrix is input to CNN, and then features are extracted by convolution layer, and the preliminary emotional classification of music is obtained. As a weight matrix, the convolution kernel moves according to the designed step size, and the data at the corresponding position of the output is weighted and summed as the output value in the characteristic graph. Its calculation formula is shown in the following formula:(3)fij=s∑m=0F−1∑n=0F−mxm,nwi+m,i+n+b.The width and height of convolution kernel can be simplified as shown in the following formula:(4)Y=cosvW,A+X.The pool layer consists of downsampling, dimensionality reduction, invariance, perceptual field expansion, etc. The size and step size of the pool area must be determined for the pool process, just like for convolution, and the values must be aggregated. In the field of image recognition, CNN is frequently used, and the input data is frequently in the form of images. There is only one convolution kernel per convolution layer, and it can move in both the row and column directions of the pixel matrix. The generated feature matrix should be used as input data when CNN is used to classify text sentiment. If another architecture uses the output layer as its input, the function G serves as the activation function. The function is a classification function when used as the output layer directly. This paper uses the softmax function as its classification function. The distribution probability for each class label is determined by the softmax function. After the final layer went through convolution or was pooled, all of the feature maps are combined into a single vector known as a global feature. This global feature is then mapped with additional fully connected layers as probability vectors for classification judgement. The equation displays the music feature normalization formula:(5)f=fij−mj4∂i,1≤m≤j.At the same time of CNN training, the gradient of loss function to each parameter is calculated, and the parameters are updated by backpropagation algorithm. The reverse transfer process is mainly to calculate the error of each layer in the return process, so as to correct each weight. Train the neural network classifier to train the classifier by using the features of music pieces marked with similar emotions in the library as training samples. The output definition of output layer nodes is shown in the following equation:(6)kix=exp1c−xi22∂i3.The trained classifier is used to classify the unlabeled segments in the library, and the same segments are returned to the user. The user judges these results, and if they are all segments with similar emotions, they are marked in the library, and the marked music is added to another music library until all songs are classified. The convolution layer of CNN is actually a process of feature extraction. A convolution kernel extracts a feature to obtain a feature matrix. When CNN extracts a certain feature, different regions of the original input are transformed with the same convolution kernel, and the local features are generalized, while the whole features are retained. In this way, the original semantic and word order features that can best represent the emotional tendency of music can be preserved.Since the audio signal is a one-dimensional time-domain signal with a complex frequency distribution, it is not a good candidate for direct input to CNN. The more generalized time-frequency analysis is more understandable than spectrum analysis. It is necessary to multiply each sampling time point by a window function in order to analyze audio signals using the short-time Fourier transform. In order to obtain the time-frequency distribution diagram of the signal, which is a type of spectrum diagram, during training, first framing and windowing are performed, then discrete Fourier transform, and finally the results produced by the entire segment of signals are stacked. As a result of the entire process, one-dimensional sound signals are transformed into two-dimensional signals that can reflect the frequency distribution and change over time. Although the number of parameters significantly decreased after the application of local connection, it remained high. The idea of weight sharing was developed to address this issue. Weight sharing is based on the presumption that if a feature is valuable in the interval, the implicit principle is that the statistical features of some areas of an image are consistent with those of other areas. Its operation is similar to convolution operation, with the exception that the pool operation does not contain parameters, and it typically takes place between adjacent convolution layers. Keep the maximum value (maximum pool) in each pool window, or take the average of all values in the window, and move the window on the input to repeat the operation. Its working principle is shown in Figure2.Figure 2
Working principle of pool layer.Common pooling methods include maximum pooling, average pooling, and random pooling. The first two are to take the maximum and average values in the pooled area as outputs, while the latter is to assign probability values in the pooled area according to their values and randomly sample according to the probability values. In this paper, the effects of the above different pooling methods on the classification accuracy of the model are compared through experiments. The input spectrum is the Mel spectrum with the best experimental effect before, and the experimental results are shown in Table1.Table 1
Influence of table pooling mode on classification accuracy.
Pool modeACC (%)Max pooling73.15Mean pooling76.17Stochastic pooling72.18A large convolution kernel can better capture global information, while a small convolution kernel can more easily discover local information. On the other hand, deep CNN is prone to overfitting, and gradient information is difficult to be transmitted to the front of the network. When working, the convolution kernel moves according to a predetermined step length, and the swept area is subjected to matrix dot multiplication, and the offset value is superimposed, as shown in Figure2 for 2D input.(7)aj,k=∂∑m=0f1−1wi+xk2∂i.The feature map produced by the convolution layer is typically sent to the pool layer for feature selection and data filtering. This downsampling process does not require any parameter updates and aids in avoiding overfitting. The output size of a pool is determined by the pool size, step size, and filling. A pool is similar to a convolution kernel scanning feature map. Mel frequency cepstrum, which is based on the linear cosine transform of the logarithmic power spectrum in the nonlinear Mel frequency range, is a representation of the short-term power spectrum of sound in sound processing. Mel cepstrum coefficient, which can describe the spectral envelope and details, is a coefficient that makes up MFC. As feature vectors, the extracted audio features are then used to train a variety of classifiers. The classification accuracy obtained is compared with the previously developed music emotion classification model using ten-fold cross-validation. The final result is shown in Table2.Table 2
Comparison of different classification model algorithms.
Classification modelACC (%)Decision tree49.18Logistic regression60.31Random forest68.24SVM(RBF)64.19CNN76.28There are many shortcomings in neural network, such as huge number of parameters and limitation of input structure, and the above cumbersome processes cause a lot of loss in information circulation, and when the most primitive audio signal enters the network, many important details are lost, resulting in insufficient feature vectors for sentiment classification, which affects the final classification result and generalization performance of the model. If we can simplify the steps of inputting audio signals into the network, we can not only save a lot of time, but also reduce unnecessary information loss [18, 19]. This section illustrates that CNN is superior to ordinary neural networks in specific tasks by analyzing the characteristics of local connection, weight sharing, and pool operation.


## 4. Construction of Classification Model of Music Emotion

### 4.1. CNN Classification Model Based on Audio Signal

Audio signal is a one-dimensional time domain signal, it is difficult for us to judge its frequency distribution by observation, and it is not suitable for direct input to CNN. The method of spectrum analysis can be used to convert the audio frequency from the time domain to the frequency domain through discrete Fourier transform to reflect the frequency domain distribution of the signal, but at the same time, the time domain information will be lost, and it is difficult to obtain the frequency change rule with time. In order to better solve this problem, the method of time-frequency analysis should be used.Intuitively, Mel spectrogram can be seen as obtained by applying nonlinear transformation to the frequency axis of short-time Fourier spectrogram; that is, the spectrogram with Hz as frequency scale is converted into Mel scale. This auditory frequency scale has the effect of highlighting low-frequency details, because people are more sensitive to low-frequency sound changes, while weakening high-frequency details, mainly fricative and other sudden noises, and usually do not need too high model fidelity. To analyze audio signals based on short-time Fourier transform, a window function should be added at each sampling time point. This process is called framing and windowing, then discrete Fourier transform is carried out, and finally the results generated by the whole segment of signals are stacked, so that the time-frequency distribution diagram of the signal can be obtained, which is a kind of spectrum diagram. In the whole process, one-dimensional audio signals are converted into two-dimensional signals that can reflect the frequency distribution and change with time. The spectrum diagram generated based on a certain segment of music is shown in Figure3.Figure 3
Short-time Fourier spectrum.The information presented in the spectrogram is the changing law of sound signals with time. It can be seen that the frequency axis of the spectrogram is linearly related to Hz, which is very balanced for the display of low, medium, and high frequencies of audio signals. In order to further simplify the model size, reduce the amount of calculation, and prevent overfitting, this paper uses a transition layer consistent with Dense Net to reduce the dimension of features, which essentially consists of a convolution unit and a pool layer. The former is used for channel dimension reduction of features, and the latter is used for width and height dimension reduction of features. The prediction method based on audio clips can make the model more flexible to deal with music samples with different time lengths, which is of great significance in practical use. The audio clips are selected by randomly selecting the starting point and intercepting the audio clips in sequence [20]. There is 50% overlap between the audio clips, which can avoid the loss of information that partially lies near the interception point of the audio clips and spans two audio clips due to nonoverlapping cutting. This section tests the performance of the model when different numbers of segments from 1 to 18 are used in the prediction stage, and the specific classification accuracy is shown in Figure 4.Figure 4
Classification is accurate when different numbers of segments are used.It can be seen from Figure4 that, theoretically, the accuracy rate can continue to be improved by increasing the number of audio clips used for prediction. However, with the increase of the number of audio clips used, each audio clip will bring less and less additional effective information, and its contribution to the improvement of accuracy rate will be weaker and weaker, and the final accuracy rate will converge to a stable range. Therefore, it is meaningless to blindly increase the number of audio clips used. In order to ensure high accuracy and avoid meaningless increase in the number of audio clips, the characteristics of GTZAN data set should be combined.


### 4.2. CNN Classification Model Based on Music Lyrics

The majority of conventional methods for classifying musical emotions are based on lengthy texts. Music, on the other hand, is a brief text that emerged from social contexts and possesses the traits of a large amount of data, brief content, sparse features, rich in new words, and mixed information. In the past, it was challenging to ensure that analysis would be effective when dealing with short texts. This article introduces it as a method for categorizing the emotional content of music, which not only addresses the traditional bag-of-words model’s data sparseness issue, but also takes into account word semantics and word order relationships between words [21]. By using a convolution operation and a pool operation, respectively, CNN’s convolution layer and pool layer extract the features of the sentence expression of music and produce generalized feature vectors. The sentence expression of music is the feature vector that the pool layer transmits, that is, CNN’s classifier. The global average pool operation rather than the maximum pool operation of the final convolution control block’s output is used in the convolution control block model. The model receives the global feature average as a result of the global average pooling, and the output features can subsequently be directly trained with the classifier. The correlation change of emotion classification results of convolution control block model on the training set is shown in Figure 5.Figure 5
Relationship of emotion classification in model.The MFCC-based feature extraction algorithm is selected to extract the features of audio information of various musical emotions, and the principal component analysis method is used to reduce the dimension of feature sets. It shows that the classification performance after feature fusion is better than that based on a single feature, and the classification emotion has high relevance and good stability. A song with various emotions can be randomly selected from the data source, and different audio characteristic maps can be obtained.When classifying emotions, the feature extraction step actually involves filtering out words that have no bearing on the process and removing those that do. The reference for extracting the feature items of lyrics is the statistical analysis of the number of words, the types of words, and the number of lines in the lyrics. The emotion of lyrics is classified in this study using CNN, so it is possible to determine how many feature items were extracted from each song’s lyrics. Some of the music lyrics in this study contain more than 100 feature items. The specific operation method used in this paper involves calculating the TF-IDF value corresponding to each music lyric, selecting the first 20 words with the highest TF-IDF value as the filling objects, and then backfilling the feature words with the highest TF-IDF value for music with lyrics that are less than 100 words in length. The input layer’s specific job is to connect to the next layer—the convolution layer, which is responsible for feature extraction—by connecting the feature words and the corresponding feature vectors to CNN as input data. Local features are extracted using three filters of varying sizes, and a feature map is produced by computing the ReLU activation function. The local feature map that is generated is sampled twice in the following layer of sampling in order to extract the feature items with the strongest emotion. The maximum sampling approach is used in this paper. The final result of the emotion classification is output after the full connection layer has finished mapping each category. In this study, the model’s input data consists of the feature vectors that correspond to the extracted feature items. The model’s input data consists of the following feature vectors that correspond to the extracted feature items. Concurrently, the input data are handled differently, and the feature objects are drawn from the word- and sentence-level data. The emotional classification results of lyrics by CNN are shown in Figure6.Figure 6
Classification performance of feature objects.The accuracy of emotion classification results obtained by this method of extracting features at word level and sentence level is less different. Compared with the result of emotion classification at sentence level, the effect of emotion classification based on words is better. Although the text of lyrics is short, it is also different from ordinary short texts, in which all complete sentences appear, while sometimes incomplete sentences appear in lyrics, which will affect the final classification results. Adding TF-IDF value to the generated feature vector is helpful in distinguishing different kinds of music.


## 4.1. CNN Classification Model Based on Audio Signal

Audio signal is a one-dimensional time domain signal, it is difficult for us to judge its frequency distribution by observation, and it is not suitable for direct input to CNN. The method of spectrum analysis can be used to convert the audio frequency from the time domain to the frequency domain through discrete Fourier transform to reflect the frequency domain distribution of the signal, but at the same time, the time domain information will be lost, and it is difficult to obtain the frequency change rule with time. In order to better solve this problem, the method of time-frequency analysis should be used.Intuitively, Mel spectrogram can be seen as obtained by applying nonlinear transformation to the frequency axis of short-time Fourier spectrogram; that is, the spectrogram with Hz as frequency scale is converted into Mel scale. This auditory frequency scale has the effect of highlighting low-frequency details, because people are more sensitive to low-frequency sound changes, while weakening high-frequency details, mainly fricative and other sudden noises, and usually do not need too high model fidelity. To analyze audio signals based on short-time Fourier transform, a window function should be added at each sampling time point. This process is called framing and windowing, then discrete Fourier transform is carried out, and finally the results generated by the whole segment of signals are stacked, so that the time-frequency distribution diagram of the signal can be obtained, which is a kind of spectrum diagram. In the whole process, one-dimensional audio signals are converted into two-dimensional signals that can reflect the frequency distribution and change with time. The spectrum diagram generated based on a certain segment of music is shown in Figure3.Figure 3
Short-time Fourier spectrum.The information presented in the spectrogram is the changing law of sound signals with time. It can be seen that the frequency axis of the spectrogram is linearly related to Hz, which is very balanced for the display of low, medium, and high frequencies of audio signals. In order to further simplify the model size, reduce the amount of calculation, and prevent overfitting, this paper uses a transition layer consistent with Dense Net to reduce the dimension of features, which essentially consists of a convolution unit and a pool layer. The former is used for channel dimension reduction of features, and the latter is used for width and height dimension reduction of features. The prediction method based on audio clips can make the model more flexible to deal with music samples with different time lengths, which is of great significance in practical use. The audio clips are selected by randomly selecting the starting point and intercepting the audio clips in sequence [20]. There is 50% overlap between the audio clips, which can avoid the loss of information that partially lies near the interception point of the audio clips and spans two audio clips due to nonoverlapping cutting. This section tests the performance of the model when different numbers of segments from 1 to 18 are used in the prediction stage, and the specific classification accuracy is shown in Figure 4.Figure 4
Classification is accurate when different numbers of segments are used.It can be seen from Figure4 that, theoretically, the accuracy rate can continue to be improved by increasing the number of audio clips used for prediction. However, with the increase of the number of audio clips used, each audio clip will bring less and less additional effective information, and its contribution to the improvement of accuracy rate will be weaker and weaker, and the final accuracy rate will converge to a stable range. Therefore, it is meaningless to blindly increase the number of audio clips used. In order to ensure high accuracy and avoid meaningless increase in the number of audio clips, the characteristics of GTZAN data set should be combined.


## 4.2. CNN Classification Model Based on Music Lyrics

The majority of conventional methods for classifying musical emotions are based on lengthy texts. Music, on the other hand, is a brief text that emerged from social contexts and possesses the traits of a large amount of data, brief content, sparse features, rich in new words, and mixed information. In the past, it was challenging to ensure that analysis would be effective when dealing with short texts. This article introduces it as a method for categorizing the emotional content of music, which not only addresses the traditional bag-of-words model’s data sparseness issue, but also takes into account word semantics and word order relationships between words [21]. By using a convolution operation and a pool operation, respectively, CNN’s convolution layer and pool layer extract the features of the sentence expression of music and produce generalized feature vectors. The sentence expression of music is the feature vector that the pool layer transmits, that is, CNN’s classifier. The global average pool operation rather than the maximum pool operation of the final convolution control block’s output is used in the convolution control block model. The model receives the global feature average as a result of the global average pooling, and the output features can subsequently be directly trained with the classifier. The correlation change of emotion classification results of convolution control block model on the training set is shown in Figure 5.Figure 5
Relationship of emotion classification in model.The MFCC-based feature extraction algorithm is selected to extract the features of audio information of various musical emotions, and the principal component analysis method is used to reduce the dimension of feature sets. It shows that the classification performance after feature fusion is better than that based on a single feature, and the classification emotion has high relevance and good stability. A song with various emotions can be randomly selected from the data source, and different audio characteristic maps can be obtained.When classifying emotions, the feature extraction step actually involves filtering out words that have no bearing on the process and removing those that do. The reference for extracting the feature items of lyrics is the statistical analysis of the number of words, the types of words, and the number of lines in the lyrics. The emotion of lyrics is classified in this study using CNN, so it is possible to determine how many feature items were extracted from each song’s lyrics. Some of the music lyrics in this study contain more than 100 feature items. The specific operation method used in this paper involves calculating the TF-IDF value corresponding to each music lyric, selecting the first 20 words with the highest TF-IDF value as the filling objects, and then backfilling the feature words with the highest TF-IDF value for music with lyrics that are less than 100 words in length. The input layer’s specific job is to connect to the next layer—the convolution layer, which is responsible for feature extraction—by connecting the feature words and the corresponding feature vectors to CNN as input data. Local features are extracted using three filters of varying sizes, and a feature map is produced by computing the ReLU activation function. The local feature map that is generated is sampled twice in the following layer of sampling in order to extract the feature items with the strongest emotion. The maximum sampling approach is used in this paper. The final result of the emotion classification is output after the full connection layer has finished mapping each category. In this study, the model’s input data consists of the feature vectors that correspond to the extracted feature items. The model’s input data consists of the following feature vectors that correspond to the extracted feature items. Concurrently, the input data are handled differently, and the feature objects are drawn from the word- and sentence-level data. The emotional classification results of lyrics by CNN are shown in Figure6.Figure 6
Classification performance of feature objects.The accuracy of emotion classification results obtained by this method of extracting features at word level and sentence level is less different. Compared with the result of emotion classification at sentence level, the effect of emotion classification based on words is better. Although the text of lyrics is short, it is also different from ordinary short texts, in which all complete sentences appear, while sometimes incomplete sentences appear in lyrics, which will affect the final classification results. Adding TF-IDF value to the generated feature vector is helpful in distinguishing different kinds of music.


## 5. Comparison of Experimental Results

In order to better evaluate the proposed CNN emotion classification model, the experimental data in this chapter are SST, IMBD, SemEval, and Douban, and ICSA network is set up for the comparison with the traditional LeNet, AlexNet, and VGGNet. The performance of CNN emotion classification model is verified by cost function and accuracy index. In this paper, we compare the performance of CNN emotion classification model with the traditional LeNet, AlexNet, and VGGNet algorithms on four standard data sets, as shown in Figure7.Figure 7
Comparison of CNN emotion classification models.As can be seen from Figure7, this model has obvious advantages in all aspects. On the same data set, the average accuracy of this model is 91.4%, while the average accuracy of LeNet, AlexNet, and VGGNet is 75.3%, 72.2%, and 79.4%, respectively, which shows that the cost function value of this model is one order of magnitude smaller than the other three algorithms, and the error value is smaller.


## 6. Conclusions

The number of music works is growing rapidly along with Internet and multimedia technology’s popularity, and it is challenging to categorize and manage a large number of music works. Because users’ preferences are becoming more and more demanding, and the traditional method of music analysis and classification, which relies primarily on professionals, is progressively becoming unworkable, there is an increasing need for computer programs that automatically classify the emotions of music. Learning and categorizing the characteristics of music are the foundation of a system that can accurately and effectively satisfy the aforementioned requirements. In order to categorize music emotions, this paper primarily examines the CNN music emotion classification model based on frequency spectrum, which uses CNN to extract the bottom and top acoustic features of music signals. This paper examines the shortcomings of the old model in terms of input data in order to enhance the built CNN model: small amounts of training data for the deep learning model make it challenging to increase accuracy; the time-consuming process of creating frequency spectra makes it simple to miss important information and may even introduce noise. The split spectrograms are not correlated with one another. The network architecture’s main drawbacks are that the depth is insufficient, the structure is a little too simple, and it is challenging to extract the high-level abstract features of the entire musical composition. Due to the aforementioned flaws, changes have been made: the method of sliding window is used to segment audio, greatly increasing the correlation between training data and data. The input is the intermediate data of the generated spectrogram, which reduces the loss caused by information conversion. The network’s depth is increased, and a residual module is added on top of the base network to create the new CNN as a collection of modules. According to the experimental findings, LeNet, AlexNet, and VGGNet are all less accurate than the CNN emotion classification model used in this paper. CNN has been successfully applied to the field of computer vision, and the depth of its recently proposed model has reached hundreds of layers of network architecture. For natural language processing tasks, the depth of the CNN model is still extremely shallow, and the deepest model is only a dozen layers. Therefore, how to make full use of CNN to automatically extract higher-level feature values and build deeper models to solve complex natural language processing tasks is still an important research direction.



---

*Source: 7221064-2022-08-30.xml*