# Forecasting of Musical Equipment Demand Based on a Deep Neural Network

Authors: Jiemin Zhang
Journal: Mobile Information Systems
 (2022)

---

## Abstract

With the popularity of music, the music equipment market has ushered in a new round of explosion. But at the same time, the market environment is changing rapidly, and music equipment companies are greatly affected by market changes. This type of enterprise has a wide variety of products, fast update, short delivery time and urgent time, which is a severe challenge to its production and operation. In this environment, in order to quickly respond to changes in the market environment, and to formulate plans for corporate procurement, production, and sales in an orderly manner, music equipment companies must make accurate predictions of market demand. At present, the extended research based on LSTM is a relatively mainstream deep neural network in the research methods of time series problems. This article is based on LSTM model to conduct an in-depth study on the prediction of music equipment demand. In order to solve the problems of overfitting, disappearance of gradient, model collapse and other problems in previous experimental studies, this paper proposes an improved LSTM prediction model. In terms of model structure selection, Dropout mechanism is used, and L2 regular term is introduced. In the selection of the activation function, MReLU function is proposed, which can improve the prediction effect of the model and enhance the applicability of the model. To measure the prediction effect of the improved LSTM model established in this paper, this paper selects RMSE and MAE as evaluation indicators, and compares experiments with other mainstream prediction models. The research results show that the improved LSTM network prediction model is superior to other models in the prediction of music equipment demand, which verifies the effectiveness.


---

## Body

## 1. Introduction

Since entering the twenty-first century, the music industry has developed rapidly, and music equipment has become an indispensable part of people's lives. However, with the development of market economy and economic globalization, the world economic situation and the living environment of enterprises have undergone tremendous changes. The globalization of economic activities is accelerating, and the needs of customers are becoming increasingly diversified and individualized. Music equipment companies are facing increasingly cruel market competition. If you want to win competition and customers, win in fierce competition. Units or individuals engaged in the production and sales of musical equipment must provide products to customers in the fastest time and at the lowest cost. This makes estimating the changes in the music equipment market and making accurate and timely product demand forecasts become the key elements for the success of modern enterprises. Because the key to management is decision-making, and the premise of decision-making is prediction. In the process of decision-making implementation, in order to make the decision-making goals to be achieved smoothly, it is necessary to reduce uncertainty through prediction and enhance the foresight of the future [1–5].Music equipment demand forecasting is a key link in the supply chain of this type of enterprise. Based on the results of demand forecasts, companies can formulate reasonable raw material procurement plans, production plans, staffing plans, and inventory plans. Equipment demand forecasting is based on systematic investigation and research of various factors affecting demand, using scientific methods to analyze, foresee, estimate and judge the development trend of future product demand and changes in various related factors. Without accurate equipment demand forecasting, it is impossible to have correct business decisions and scientific plans. However, accurate forecasting is difficult, on the one hand because the demand for equipment is affected by many factors. The analysis of multiple factors and large amounts of data is beyond the reach of the human brain, and information technology must be used. On the other hand, because demand forecasts are based on a large amount of data, most of these data come from market forecasts and production records in the production process. Therefore, it has strong timeliness and complexity, so the effective collection of production demand data, processing and conversion are the prerequisites for its analysis. This also has higher requirements for the timeliness of data processing and the diversity of generation rules. Traditional statistical analysis methods can no longer meet the information processing needs of demand forecasting. Therefore, absorbing new data processing methods has become a requirement for demand forecasting [6–10].If music equipment companies or businesses want to formulate reasonable production and sales plans, they need to use scientific methods to accurately predict the demand for music equipment in the market. At present, the forecasting ability of these enterprises is very poor. Using experience and some basic mathematical statistics to make judgments, the final forecast results are quite different from the actual situation. The main reason is that it generally uses traditional linear prediction or a single prediction method. However, the current market conditions have led to demand that is no longer a single data that can be met by simple linear fitting. This requires a more scientific and accurate forecasting model to forecast demand. With the advent of the information age, international and domestic information science has made continuous development. Many new results and theories have been produced, especially machine learning and deep learning are widely used in various prediction problems. These results and theories have broad guiding significance and value. How to organically combine these theories with practice has also become a very meaningful research topic [11–15].This article is based on a deep neural network to accurately predict the demand for music equipment. The contributions of this paper are concluded as the following two aspects: (1) Combining Adaboost and LSTM, an improved LSTM prediction model is proposed, which is used to forecast the demand of music equipment. (2) In terms of model structure selection, the Dropout mechanism is used and the L2 regular term is introduced. In the selection of the activation function, the MReLU function is proposed, which can improve the prediction effect of the model and enhance the applicability of the model.


## 2. Related Work

At present, the research on demand forecasting mainly focused on the innovation of research methods and the establishment of combined forecasting models. Many forecasting methods and models had been produced. Literature [16] studied the operation of neural network based on the premise that the study of neural network was only qualitative analysis and the accuracy cannot be guaranteed. It used the neural network model to actually predict, and the time series was highly accurate, which was better than traditional quantitative methods, and it studied how the neural network mechanism could make good predictions. Literature [17] proposed GRNN, which was applied to problems that cannot be solved by linear calculation. As an extension of the neural network, this method was smoother in multi-criteria prediction, and could converge to the optimum faster when there were fewer samples. Literature [18] used data mining technology to study consumer reviews and consumption history, used artificial intelligence to predict inventory demand, and optimized inventory for forecasted demand. Literature [19] proposed a combined forecasting method. The idea of combined forecasting was to build a model by integrating multiple forecasting methods. The way of combination was usually to assign different weights to each method, and used related methods to calculate the weights. Because it avoided the shortcomings of a single method, considered all factors. Its prediction accuracy was good, and it had achieved success in various fields. Literature [20] optimized the combination forecasting, considering the fuzziness of information in the combination forecasting, and constructing a fuzzy combination model to predict the demand for wind power. Literature [21] proposed an induced ordered weighted average operator. Its main idea was to sort the elements and then weight them together. The sorting had nothing to do with the size of the elements, and it was arranged according to the size of the induced value. Literature [22] verified that the combined method has better prediction accuracy than a single method through a case study. Experiments showed that the combined prediction error of the two methods is reduced by 7.3%, and the combined prediction error of the five methods was reduced by 16.5%. Literature [23] combined prediction method predicts Hong Kong's inbound tourists. Compared with the single method, the prediction effect was better, which proved the practicality of combined prediction. Literature [24] studied the fuzzy combination model, and proposed linear and non-linear fuzzy combination forecasts. The linear model was based on least squares prediction, and the nonlinearity is based on fuzzy integral.With the development of computer science, deep learning technology was widely used in prediction. This article built a music equipment demand prediction model based on LSTM, so the following was a brief introduction to the development of LSTM. LSTM promoted the development of recurrent neural networks. Especially in today's wide application of deep learning [25], LSTM achieved certain results [26]. With the maturity of the overall technical environment and the improvement of improvement methods, LSTM had become more popular in the near future. Today, with the rapid development of science and technology, LSTM had a wide application. The LSTM-based system could complete tasks [27]. According to the introduction of the LSTM proponent, Google's CTC-based LSTM program had greatly improved the speech recognition capabilities of Android phones and other devices. And Google used LSTM in a wide range, and it had applications in generating image captions, automatically replying to emails, and the new smart assistant Allo. In addition, the application of LSTM had significantly improved the quality of Google Translate. At present, a large part of the computing resources of the Google data center were now performing LSTM tasks [28]. Apple's iPhone used LSTM in QucikType and Siri. Microsoft not only used LSTM for speech recognition, but also used this technology for virtual dialogue image generation and programming code, and so on. Not only business giants such as Google, but also scholars at home and abroad in recent years had continuously studied and improved long-term and short-term memory neural networks [29] and applied them in various fields. Literature [30] proposed to increase peephole connection, which meaned that the gate layer would also accept the input of the cell state. In the absence of external reset or loss of previous reports, the improved LSTM could learn to generate highly non-linear, accurate and stable sequences. Literature [31] based on the combination of LSTM model and various generalized autoregressive conditional heteroscedasticity models, a new hybrid LSTM model was proposed to predict stock price fluctuations. The hybrid LSTM model, as an integrated model combining time series model and neural network model, significantly improved the prediction performance. There were many researches on similar LSTM models, and the proposed method could be extended to various fields [32].


## 3. Methods

To solve the problems of large model complexity, experimental overfitting, sample input data, and weak learning ability of experimental models caused by the purpose of improving accuracy in previous experiments. In this chapter, the following aspects are introduced respectively, and the model structure and prediction process are described.


### 3.1. Adaboost

The enhanced algorithm is currently one of the most popular and successful neural network algorithms. Adaboost is a machine learning algorithm to promote performance. It is very sensitive to noisy data and outliers. It is less susceptible to overfitting. After research by researchers, the Adaboost algorithm has improved the generalization ability and predictive ability of previous models, and has also been strengthened in terms of model fitting speed. The Adaboost algorithm first generates a weak classifier to obtain some features, and updates the weights so that the latter round of weak classifiers gets more attention. The Adaboost algorithm enhances the role of this feature in the voting process by increasing the weight of the weak predictorfor a small prediction error. Finally, a strong influential feature is obtained by integrating multiple weak classifiers into a strong classifier. This process can effectively improve the generalization and prediction capabilities of the model. The final strong classifier is the weighted average of multiple weak classifiers.The learning process of the Boosting algorithm is regarded as the enhancement of the sample data, enhancing the influence of the data with a large influence rate so as to make the prediction result more accurate. Adaptive is reflected in self-regulation. It continuously strengthens the samples that were misclassified by the previous weak classifier, and trains the next weak classifier again through all the weighted samples. This operation is repeated once in each round, and a new weak classifier is added in each round until the maximum number of iterations is finally reached. The mechanism for Boosting is initializing the sample weight first, training a weak learner, and update the weight of the training sample to obtain the new weight. This process makes the weight of training samples with high learning error of weak learner in the previous step become lower, and its purpose is to continue to strengthen the influence of the samples that have a large impact on the result. The next step is to use the adjusted new weights to train the samples, and it is repeated until weak learners reach the set number. Last, all the obtained weak learners are integrated into a strong learner. The Adaboost integrated learning algorithm uses Boosting to enhance data and uses the concept of Adaptive self-adjustment to combine the two into a stronger iterative algorithm.The advantage of the Adaboost algorithm is that it can make the classification accurate enough so that the features will not be missed. At the same time, the Adaboost algorithm is very compatible and provides users with a framework for integrated learning. The corresponding disadvantage is that the training time is too long, and the training results are overly dependent on the choice of weak classifiers. For abnormal samples, a larger weight may be obtained, which affects the prediction accuracy of the final strong classifier. Therefore, this experiment will perform multiple preprocessing on the input samples to avoid errors such as sample abnormalities.In the Adaboost algorithm, the data set is first given an initial weight. After passing through a classifier, the weighted prediction result will be weighted by the alpha value to obtain a weak classifier, and the error rate of the obtained weak classifier is guaranteed to be the smallest. When the weight is updated and the next iteration is performed, the Adaboost algorithm will continue to perform repeated training to update and adjust the weight of the feature value and the learning rate. When the iteration is completed, the final strong classifier is obtained by summing the weighted results. This article will use LSTM neural network as the weak classifier for prediction research, and use the Adaboost algorithm for ensemble learning of the weak classifier. After learning a variety of different weak classifiers through the model ensemble, a strong classifier is obtained, so that the model can be considered more comprehensively in the training process. Finally, an improved LSTM (ILSTM) model used for music equipment demand forecasting is constructed.


### 3.2. Structure

The experiment selects LSTM neural network and uses Adaboost algorithm to optimize and improve the network structure for prediction research. Figure1 shows the structure of the ILSTM network model. The generation of the weak classifier is determined by the setting of the internal structure of the LSTM in the experiment. First initialize the weights, after the weak classifier training, adjust the sample weight to Wn, and generate the weak classifier GW from the minimum error rate obtained in each iteration of training. During the second iteration of training, the updated sample weights of the first iteration of training will affect the adjustment of the second iteration of sample weights. Increase the weight of samples for a small prediction error rate, so as to enhance the influence of this feature in the voting process. After all the weak classifiers are trained, the strong classifier is obtained by weighting and averaging all the weak classifiers, and the optimal convergence parameters of the model are obtained. Finally, the optimal prediction model is obtained and the prediction result is obtained.Figure 1
Structure of ILSTM.In the experimental process of this research, the experimental data is collected and preprocessed and then input into the network model. The Adaboost framework is used in the network structure to update the weights and optimize the LSTM model parameters. When the data is input to the LSTM model, the sample weight is first initialized, and the sample weight is updated every time the data undergoes LSTM training. Increase the weight of the weak classifier for a small prediction error, and reduce the influence of weak classifier for a large error in the final voting stage. Each time the data passes through the LSTM and the weight is updated, an optimal weak classifier is obtained, and so on until the end ofN iterations. When the iteration is over, we get N weak classifiers with the smallest error rate, and combine them to get a strong classifier. The iteration stops when the training error rate is 0 or the number of weak classifiers N reaches the peak of the computing power of the computer.The experiment uses a stacked LSTM neural network to increase the depth. Compare the weights adjusted by LSTMs of different layers to obtain the final strong predictor. According to the hardware and software conditions, the experiment sets the number of LSTMs, that is, the number of iterationsT, to 0, and the LSTM model with more than 2 layers has good predictive ability. To take into account the low complexity of the model, 2-layer, 3-layer, 4-layer, 5-layer, and 6-layer LSTM were used for training. The specific settings of the internal structure of the ILSTM model are shown in Table 1.Table 1
Internal structure of ILSTM.
ItemLayerActivationHidden layerLSTM-12Tanh3LSTM-23Tanh3LSTM-34Tanh3LSTM-45MReLU3LSTM-56MReLU3LSTM-65MReLU3LSTM-74MReLU3LSTM-83MReLU3LSTM-92MReLU3


### 3.3. Forecasting Process

According to the mixed model structure, the prediction process of this research model is formulated. First, the data set needed for experimental prediction is obtained by screening and adjusting the collected initial music equipment data. The data set is given as input into the ILSTM model. In the process of continuous training, the model adjusts the sample weight, learning rate, and feature influence rate. When the model training is over, the optimal prediction model and prediction results and related prediction indicators are obtained. The input of the data set and the training process of the model will be introduced below.Model data set input. First, divide the data set according to the prediction method, and perform cross-validation when the experimental model starts to train. Cross-validation can make the model achieve good results under a smaller experimental data set and improve the prediction performance. In the process of cross-validation, the model continuously adjusts the model parameters through continuous training. For the weak classifiers in the Adaboost algorithm, each weak classifier is input with full cross-validated data. That is, in each weak classifier, a full round of cross-validation data input is required to obtain more accurate feature classification. When all samples in the data set have been trained, the cross-validation input ends and the training of a weak classifier is completed.The training process of ILSTM. According to the Adaboost integrated learning idea, the LSTM neural network is embedded in the Adaboost framework. The number of weak classifiers is selected according to the experimental hardware and software conditions, and the network depth of weak classifier is set. The network depth of weak classifiers in this study is based on the number of layers and activation functions of LSTM, the classification of weak classifiers is more comprehensive according to different LSTM stacked layers and different activation functions. Adaboost combines and trains weak classifiers with different LSTM blocks. In the process of training the model, Adaboost continuously adjusts the sample weight according to the error of weak classifiers. The purpose is to make the samples with a high impact rate have a stronger impact on the prediction results, and adjust the samples with a low impact rate to have a lower impact on the prediction results. When all the weak classifiers are trained, the Adaboost algorithm weights and averages multiple weak classifiers to form a strong classifier to obtain the prediction result and output the experimental prediction error. The detailed steps are:Step 1.
Calculation of sampling weight of training samples:(1)Wmt=1M,where Wnt is the weight of samples in the t-th iteration, M is the number of LSTM predictors, and t is the number of training samples.Step 2.
LSTM predictorfm is trained by training samples, and the training samples are sampled according to the weights.Step 3.
Prediction error and overall weight of LSTM predictor are calculated as follows:(2)emt=xi−x^ixi,Wm=0.5ln1−∑t=1Memt∑t=1Memt.Step 4.
Update the sample weight of the training sample:(3)Wm+1t=Wmtαmt∑t=1MWmtαmt,αmt=expemt.Step 5.
Repeat steps2 to step 4 until all LSTM predictors are obtained.Step 6.
Combine the prediction results of all LSTM predictors according to the overall weight to generate the final strong classifier and get the model prediction results.
Through the continuous adjustment of the weight of the weak classifier, the final strong classifier can be affected by more and influential feature values. This can make the prediction experiment performed by the final model more accurate.


### 3.4. Loss

According to the prediction goals of this experimental study and the research experience of past researchers, more experiments have chosen the mean square error (MSE) as loss function. The mean square error is sensitive to outliers, and a more stable closed-form solution can be given by setting its derivative to 0. The mean square error is the most commonly used regression loss function. Its algorithm is to find the sum of the squares of the distance between the predicted value and the true value. The mathematical expression is as follows:(4)MSE=1N∑i=1Nyi−y^i2,where N is the number of samples, yi is the truth label, and y^i is the predicted value.


### 3.5. Optimizer

As the neural network continues to deepen, the model structure becomes more and more complex, the amount of data training is also increasing, and the time required to train the network is also increasing. In order to deal with complex neural network topics, these problems are unavoidable. Therefore, in the neural network model, researchers have found some methods to accelerate the training speed and shorten the training time. The more common methods for neural network optimization are SGD, AdaGrad, and RMSProp.SGD updates the sample gradient by calculating the partial derivative of the loss function of each sample. However, SGD has many unavoidable shortcomings while optimizing the model due to its feature that each sample needs to be updated for each update. Although to a certain extent, the training speed is fast, but the accuracy of the obtained results is reduced, and the training results are not optimal in the world. In terms of the gradient update of the optimizer, the update method of SGD is relatively complicated and not easy to implement in parallel.The AdaGrad optimizer can automatically adjust the learning rate. At the same time, it can update and adjust the low-frequency parameters in a larger range, and the high-frequency parameters can be updated and adjusted in a smaller range. The adjustments made to AdaGrad enable the data to have good sparsity, and therefore improve the robustness of SGD, and improve the optimization defects of SGD.The RMSProp algorithm first calculates the average of the squared gradients of the previousn times. Then, by dividing the nth gradient by this average to find the relationship between the gradients, the update ratio of the learning step is obtained. According to the calculated ratio, a new learning step length is obtained. If the gradient at this time is negative, then the learning step size will be smaller. If the gradient is positive, then the learning step size will be increased a bit. Using the RMSProp optimization algorithm can speed up the convergence speed much faster than other optimizers such as SGD, and can save a lot of model training time. In this paper, the RMSProp optimizer is selected to optimize the model.


### 3.6. Activation Function

In neural networks, it is important and necessary for the selection and use of activation functions. In order to make the model reflect the ability of training non-linear data, researchers often use activation functions to first non-linearly map the output value, and then output the mapping result through the neuron. In the training process of the model, the activation function not only reflects the neuron's ability to process data signals, but also directly relates to the efficiency of the gradient backpropagation.In the traditional LSTM model, the sigmoid function and tanh function are usually used as the activation function of the neuron. However, for very large positive numbers, the output will be infinitely close to 1. Especially, with small negative numbers, the output will infinitely approach 0. When the sigmoid activation function is used, the gradient disappears. At the same time, the value of each output of the sigmoid function is not the mean value, which will cause the neurons in the next layer to get all non-zero values in the previous layer to a certain extent. After the average value is output, the output is all positive. Many people currently use the tanh function to make up for some of the shortcomings of the sigmoid function. The tanh function changes the output of the sigmoid function, that is, the output value under the action of the tanh function is the mean value centered at zero. However, both the sigmoid function and the tanh function have relatively limited values during the training process, and the output function value is prone to the phenomenon of gradient disappearance, which is also very difficult for the training of the deep network model.Many researchers have used the ReLU activation function for experiments. Although the expression of the ReLU function is simple, the calculation speed is very fast, and the convergence speed is much faster than that of the sigmoid function and the tanh function. Also, it solves the problem that the output value was limited in the positive interval in the past. However, when the input value is negative, the ReLU function will be necrotic and cause the model to crash. Both the ELU function and the PReLU function are improvements to the ReLU function. In contrast, the PReLU function does better when the input value is negative. In terms of the learning rate setting of the model, the PReLU function is much easier than the ReLU function. In the network model under the ReLU activation function, once the learning rate setting is not accurate enough, there will be many dead neurons that are faulty and useless, causing the model parameters to not be updated. In terms of model input, once the input is negative, the PReLU function will handle it better than the ReLU function. The PReLU function not only solves the problem of gradient disappearance on the output but also improves the training speed during the training process and avoids the model collapse caused by the occurrence of some extreme phenomena.This work combines these different activation functions to propose a mixed ReLU (MReLU) function. This loss function is composed of ReLU, ELU, and PReLU. The principle is as follows:(5)MReLU=ReLUx+ELUx+PReLUx.


### 3.7. Strategies to Prevent Overfitting

Most of the traditional overfitting phenomenon processing uses parameter regularization, and the idea is to restrict the norm of the parameter to make it not too large and control it within a certain range. Therefore, the overfitting phenomenon can be reduced to a certain extent. In practical applications, in most cases, L2 can avoid parameter sparsity better than L1, and the regularization is better. However, the method of using this penalty added to the cost function may cause underfitting and it is also difficult to calibrate during the training process.The dropout algorithm can effectively suppress overfitting, and its core content is random drop or drop. In this algorithm, neural network units are randomly discarded according to a certain probability, and the discarded neurons are inactivated. So as to achieve the effect of reducing the complexity of the model and enhancing the robustness of the network.This article finally combines these two different methods. The dropout mechanism is selected and the L2 regular term is also introduced to reduce the complexity of the model, prevent the overfitting problem of the model, and improve the predictive ability of the model. Dropout adjusts settings based on experience and the specific implementation of experiments, and the regularization item is added to the Adaboost algorithm.The iterative results of the previously trained weak learner are as follows:(6)fnx=fn−1x+βtGnx.The iterative result of the weak classifier after adding the regularization term is as follows:(7)fnx=fn−1x+pβtGnx,where p is the regularization term, smaller p values correspond to larger iterations.


## 3.1. Adaboost

The enhanced algorithm is currently one of the most popular and successful neural network algorithms. Adaboost is a machine learning algorithm to promote performance. It is very sensitive to noisy data and outliers. It is less susceptible to overfitting. After research by researchers, the Adaboost algorithm has improved the generalization ability and predictive ability of previous models, and has also been strengthened in terms of model fitting speed. The Adaboost algorithm first generates a weak classifier to obtain some features, and updates the weights so that the latter round of weak classifiers gets more attention. The Adaboost algorithm enhances the role of this feature in the voting process by increasing the weight of the weak predictorfor a small prediction error. Finally, a strong influential feature is obtained by integrating multiple weak classifiers into a strong classifier. This process can effectively improve the generalization and prediction capabilities of the model. The final strong classifier is the weighted average of multiple weak classifiers.The learning process of the Boosting algorithm is regarded as the enhancement of the sample data, enhancing the influence of the data with a large influence rate so as to make the prediction result more accurate. Adaptive is reflected in self-regulation. It continuously strengthens the samples that were misclassified by the previous weak classifier, and trains the next weak classifier again through all the weighted samples. This operation is repeated once in each round, and a new weak classifier is added in each round until the maximum number of iterations is finally reached. The mechanism for Boosting is initializing the sample weight first, training a weak learner, and update the weight of the training sample to obtain the new weight. This process makes the weight of training samples with high learning error of weak learner in the previous step become lower, and its purpose is to continue to strengthen the influence of the samples that have a large impact on the result. The next step is to use the adjusted new weights to train the samples, and it is repeated until weak learners reach the set number. Last, all the obtained weak learners are integrated into a strong learner. The Adaboost integrated learning algorithm uses Boosting to enhance data and uses the concept of Adaptive self-adjustment to combine the two into a stronger iterative algorithm.The advantage of the Adaboost algorithm is that it can make the classification accurate enough so that the features will not be missed. At the same time, the Adaboost algorithm is very compatible and provides users with a framework for integrated learning. The corresponding disadvantage is that the training time is too long, and the training results are overly dependent on the choice of weak classifiers. For abnormal samples, a larger weight may be obtained, which affects the prediction accuracy of the final strong classifier. Therefore, this experiment will perform multiple preprocessing on the input samples to avoid errors such as sample abnormalities.In the Adaboost algorithm, the data set is first given an initial weight. After passing through a classifier, the weighted prediction result will be weighted by the alpha value to obtain a weak classifier, and the error rate of the obtained weak classifier is guaranteed to be the smallest. When the weight is updated and the next iteration is performed, the Adaboost algorithm will continue to perform repeated training to update and adjust the weight of the feature value and the learning rate. When the iteration is completed, the final strong classifier is obtained by summing the weighted results. This article will use LSTM neural network as the weak classifier for prediction research, and use the Adaboost algorithm for ensemble learning of the weak classifier. After learning a variety of different weak classifiers through the model ensemble, a strong classifier is obtained, so that the model can be considered more comprehensively in the training process. Finally, an improved LSTM (ILSTM) model used for music equipment demand forecasting is constructed.


## 3.2. Structure

The experiment selects LSTM neural network and uses Adaboost algorithm to optimize and improve the network structure for prediction research. Figure1 shows the structure of the ILSTM network model. The generation of the weak classifier is determined by the setting of the internal structure of the LSTM in the experiment. First initialize the weights, after the weak classifier training, adjust the sample weight to Wn, and generate the weak classifier GW from the minimum error rate obtained in each iteration of training. During the second iteration of training, the updated sample weights of the first iteration of training will affect the adjustment of the second iteration of sample weights. Increase the weight of samples for a small prediction error rate, so as to enhance the influence of this feature in the voting process. After all the weak classifiers are trained, the strong classifier is obtained by weighting and averaging all the weak classifiers, and the optimal convergence parameters of the model are obtained. Finally, the optimal prediction model is obtained and the prediction result is obtained.Figure 1
Structure of ILSTM.In the experimental process of this research, the experimental data is collected and preprocessed and then input into the network model. The Adaboost framework is used in the network structure to update the weights and optimize the LSTM model parameters. When the data is input to the LSTM model, the sample weight is first initialized, and the sample weight is updated every time the data undergoes LSTM training. Increase the weight of the weak classifier for a small prediction error, and reduce the influence of weak classifier for a large error in the final voting stage. Each time the data passes through the LSTM and the weight is updated, an optimal weak classifier is obtained, and so on until the end ofN iterations. When the iteration is over, we get N weak classifiers with the smallest error rate, and combine them to get a strong classifier. The iteration stops when the training error rate is 0 or the number of weak classifiers N reaches the peak of the computing power of the computer.The experiment uses a stacked LSTM neural network to increase the depth. Compare the weights adjusted by LSTMs of different layers to obtain the final strong predictor. According to the hardware and software conditions, the experiment sets the number of LSTMs, that is, the number of iterationsT, to 0, and the LSTM model with more than 2 layers has good predictive ability. To take into account the low complexity of the model, 2-layer, 3-layer, 4-layer, 5-layer, and 6-layer LSTM were used for training. The specific settings of the internal structure of the ILSTM model are shown in Table 1.Table 1
Internal structure of ILSTM.
ItemLayerActivationHidden layerLSTM-12Tanh3LSTM-23Tanh3LSTM-34Tanh3LSTM-45MReLU3LSTM-56MReLU3LSTM-65MReLU3LSTM-74MReLU3LSTM-83MReLU3LSTM-92MReLU3


## 3.3. Forecasting Process

According to the mixed model structure, the prediction process of this research model is formulated. First, the data set needed for experimental prediction is obtained by screening and adjusting the collected initial music equipment data. The data set is given as input into the ILSTM model. In the process of continuous training, the model adjusts the sample weight, learning rate, and feature influence rate. When the model training is over, the optimal prediction model and prediction results and related prediction indicators are obtained. The input of the data set and the training process of the model will be introduced below.Model data set input. First, divide the data set according to the prediction method, and perform cross-validation when the experimental model starts to train. Cross-validation can make the model achieve good results under a smaller experimental data set and improve the prediction performance. In the process of cross-validation, the model continuously adjusts the model parameters through continuous training. For the weak classifiers in the Adaboost algorithm, each weak classifier is input with full cross-validated data. That is, in each weak classifier, a full round of cross-validation data input is required to obtain more accurate feature classification. When all samples in the data set have been trained, the cross-validation input ends and the training of a weak classifier is completed.The training process of ILSTM. According to the Adaboost integrated learning idea, the LSTM neural network is embedded in the Adaboost framework. The number of weak classifiers is selected according to the experimental hardware and software conditions, and the network depth of weak classifier is set. The network depth of weak classifiers in this study is based on the number of layers and activation functions of LSTM, the classification of weak classifiers is more comprehensive according to different LSTM stacked layers and different activation functions. Adaboost combines and trains weak classifiers with different LSTM blocks. In the process of training the model, Adaboost continuously adjusts the sample weight according to the error of weak classifiers. The purpose is to make the samples with a high impact rate have a stronger impact on the prediction results, and adjust the samples with a low impact rate to have a lower impact on the prediction results. When all the weak classifiers are trained, the Adaboost algorithm weights and averages multiple weak classifiers to form a strong classifier to obtain the prediction result and output the experimental prediction error. The detailed steps are:Step 1.
Calculation of sampling weight of training samples:(1)Wmt=1M,where Wnt is the weight of samples in the t-th iteration, M is the number of LSTM predictors, and t is the number of training samples.Step 2.
LSTM predictorfm is trained by training samples, and the training samples are sampled according to the weights.Step 3.
Prediction error and overall weight of LSTM predictor are calculated as follows:(2)emt=xi−x^ixi,Wm=0.5ln1−∑t=1Memt∑t=1Memt.Step 4.
Update the sample weight of the training sample:(3)Wm+1t=Wmtαmt∑t=1MWmtαmt,αmt=expemt.Step 5.
Repeat steps2 to step 4 until all LSTM predictors are obtained.Step 6.
Combine the prediction results of all LSTM predictors according to the overall weight to generate the final strong classifier and get the model prediction results.
Through the continuous adjustment of the weight of the weak classifier, the final strong classifier can be affected by more and influential feature values. This can make the prediction experiment performed by the final model more accurate.


## 3.4. Loss

According to the prediction goals of this experimental study and the research experience of past researchers, more experiments have chosen the mean square error (MSE) as loss function. The mean square error is sensitive to outliers, and a more stable closed-form solution can be given by setting its derivative to 0. The mean square error is the most commonly used regression loss function. Its algorithm is to find the sum of the squares of the distance between the predicted value and the true value. The mathematical expression is as follows:(4)MSE=1N∑i=1Nyi−y^i2,where N is the number of samples, yi is the truth label, and y^i is the predicted value.


## 3.5. Optimizer

As the neural network continues to deepen, the model structure becomes more and more complex, the amount of data training is also increasing, and the time required to train the network is also increasing. In order to deal with complex neural network topics, these problems are unavoidable. Therefore, in the neural network model, researchers have found some methods to accelerate the training speed and shorten the training time. The more common methods for neural network optimization are SGD, AdaGrad, and RMSProp.SGD updates the sample gradient by calculating the partial derivative of the loss function of each sample. However, SGD has many unavoidable shortcomings while optimizing the model due to its feature that each sample needs to be updated for each update. Although to a certain extent, the training speed is fast, but the accuracy of the obtained results is reduced, and the training results are not optimal in the world. In terms of the gradient update of the optimizer, the update method of SGD is relatively complicated and not easy to implement in parallel.The AdaGrad optimizer can automatically adjust the learning rate. At the same time, it can update and adjust the low-frequency parameters in a larger range, and the high-frequency parameters can be updated and adjusted in a smaller range. The adjustments made to AdaGrad enable the data to have good sparsity, and therefore improve the robustness of SGD, and improve the optimization defects of SGD.The RMSProp algorithm first calculates the average of the squared gradients of the previousn times. Then, by dividing the nth gradient by this average to find the relationship between the gradients, the update ratio of the learning step is obtained. According to the calculated ratio, a new learning step length is obtained. If the gradient at this time is negative, then the learning step size will be smaller. If the gradient is positive, then the learning step size will be increased a bit. Using the RMSProp optimization algorithm can speed up the convergence speed much faster than other optimizers such as SGD, and can save a lot of model training time. In this paper, the RMSProp optimizer is selected to optimize the model.


## 3.6. Activation Function

In neural networks, it is important and necessary for the selection and use of activation functions. In order to make the model reflect the ability of training non-linear data, researchers often use activation functions to first non-linearly map the output value, and then output the mapping result through the neuron. In the training process of the model, the activation function not only reflects the neuron's ability to process data signals, but also directly relates to the efficiency of the gradient backpropagation.In the traditional LSTM model, the sigmoid function and tanh function are usually used as the activation function of the neuron. However, for very large positive numbers, the output will be infinitely close to 1. Especially, with small negative numbers, the output will infinitely approach 0. When the sigmoid activation function is used, the gradient disappears. At the same time, the value of each output of the sigmoid function is not the mean value, which will cause the neurons in the next layer to get all non-zero values in the previous layer to a certain extent. After the average value is output, the output is all positive. Many people currently use the tanh function to make up for some of the shortcomings of the sigmoid function. The tanh function changes the output of the sigmoid function, that is, the output value under the action of the tanh function is the mean value centered at zero. However, both the sigmoid function and the tanh function have relatively limited values during the training process, and the output function value is prone to the phenomenon of gradient disappearance, which is also very difficult for the training of the deep network model.Many researchers have used the ReLU activation function for experiments. Although the expression of the ReLU function is simple, the calculation speed is very fast, and the convergence speed is much faster than that of the sigmoid function and the tanh function. Also, it solves the problem that the output value was limited in the positive interval in the past. However, when the input value is negative, the ReLU function will be necrotic and cause the model to crash. Both the ELU function and the PReLU function are improvements to the ReLU function. In contrast, the PReLU function does better when the input value is negative. In terms of the learning rate setting of the model, the PReLU function is much easier than the ReLU function. In the network model under the ReLU activation function, once the learning rate setting is not accurate enough, there will be many dead neurons that are faulty and useless, causing the model parameters to not be updated. In terms of model input, once the input is negative, the PReLU function will handle it better than the ReLU function. The PReLU function not only solves the problem of gradient disappearance on the output but also improves the training speed during the training process and avoids the model collapse caused by the occurrence of some extreme phenomena.This work combines these different activation functions to propose a mixed ReLU (MReLU) function. This loss function is composed of ReLU, ELU, and PReLU. The principle is as follows:(5)MReLU=ReLUx+ELUx+PReLUx.


## 3.7. Strategies to Prevent Overfitting

Most of the traditional overfitting phenomenon processing uses parameter regularization, and the idea is to restrict the norm of the parameter to make it not too large and control it within a certain range. Therefore, the overfitting phenomenon can be reduced to a certain extent. In practical applications, in most cases, L2 can avoid parameter sparsity better than L1, and the regularization is better. However, the method of using this penalty added to the cost function may cause underfitting and it is also difficult to calibrate during the training process.The dropout algorithm can effectively suppress overfitting, and its core content is random drop or drop. In this algorithm, neural network units are randomly discarded according to a certain probability, and the discarded neurons are inactivated. So as to achieve the effect of reducing the complexity of the model and enhancing the robustness of the network.This article finally combines these two different methods. The dropout mechanism is selected and the L2 regular term is also introduced to reduce the complexity of the model, prevent the overfitting problem of the model, and improve the predictive ability of the model. Dropout adjusts settings based on experience and the specific implementation of experiments, and the regularization item is added to the Adaboost algorithm.The iterative results of the previously trained weak learner are as follows:(6)fnx=fn−1x+βtGnx.The iterative result of the weak classifier after adding the regularization term is as follows:(7)fnx=fn−1x+pβtGnx,where p is the regularization term, smaller p values correspond to larger iterations.


## 4. Experiment

### 4.1. Data set

Taking into account the many unstable factors in the music equipment market, this article selects more mature and stable music equipment sales and demand data from two provinces and cities, and self-made two data sets, namely, MED1 and MED2. The time span of the data set is from January 2015 to December 2019, which contains a relatively complete period of fluctuations in the demand for music equipment, which can be better used for the research and analysis of this article. The proportions of training data and testing data are 70% and 30%, respectively.In this work, RMSE and MAE are two evaluation metrics to evaluate the performance of the proposed model. The detailed calculation formulas are given below:(8)RMSE=1N∑i=1Nyi−y^i2,MAE=1N∑i=1Nyi−y^i,where N is the total number of the samples, yi is the truth value, and y^i is the value which is predicted by model.


### 4.2. Comparison with Other Methods

This work compares the proposed ILSTM method with other prediction methods including SVR, BP, RNN, and LSTM. The result on two data sets is illustrated in Table2.Table 2
Comparison with other methods.
MethodMED2MED2RMSEMAERMSEMAESVR30.524.727.821.2BP25.619.822.518.2RNN19.316.617.415.3LSTM15.612.914.811.7ILSTM12.310.410.99.1Obviously, compared with the other methods listed in the table, ILSTM can achieve the best performance. The RMSE and MAE on the MED1 data set are 12.3 and 10.4, respectively. The RMSE and MAE on the MED2 data set are 10.9 and 9.1, respectively. Compared to the best method of LSTM, RMSE decreased by 3.3 and 3.9, respectively, MAE decreased by 2.5 and 2.6, respectively. These data can verify the effectiveness of our method.


### 4.3. Evaluation on an Optimizer

As mentioned early, this work selects RMSProp as an optimizer of the network. To verify the correctness of this selection, a comparative experiment comparing other optimizers is conducted. The experimental result is illustrated in Figure2.Figure 2
Evaluation on an optimizer.Compared with the SGD and AdaGrad optimizers, the RMSProp optimizer used in this article corresponds to the lowest RMSE and MAE. Compared to the AdaGrad, RMSE decreased by 3.4 and 3.3, respectively, MAE decreased by 2.7 and 2.3, respectively. These data can verify the effectiveness of our selection for optimizers.


### 4.4. Evaluation on Activation Function

This work proposes a mixed ReLU (MReLU) activation function to make the model reflect the ability to train non-linear data. To verify the effectiveness of our designed MReLU, a comparative experiment comparing other activation functions is conducted. The experimental result is illustrated in Figure3.Figure 3
Evaluation on an activation function.Obviously, compared to the traditional ReLU activation function, the performance of ELU and PReLU is better. However, it should be noted that after combining ELU and PReLU, the proposed MReLU activation function can obtain lower RMSE and MAE, and the corresponding prediction performance is also the best. It can be proved that the MReLU activation function designed in this paper is effective and correct and can promote the improvement of network performance.


### 4.5. Evaluation on Preventing Overfitting

As mentioned early, this work combines a dropout mechanism and L2 regular term to prevent the overfitting of the model. To verify the effectiveness and reliability of this strategy, this paper conducts a corresponding comparative experiment. It compares the performance of a single dropout, a single L2 regular term, and a combination of the two. The experimental result is illustrated in Figure4.Figure 4
Evaluation on preventing overfitting.It can be seen that a single dropout or L2 regular term can effectively prevent overfitting, but the performance improvement is very limited. By combining the two and then constraining network training, the problem of network overfitting can be alleviated to the greatest extent, thereby reducing RMSE and MAE as much as possible and improving performance. This also further verifies the effectiveness of the combination of a dropout and L2 regular term in this paper to prevent network overfitting.


### 4.6. Evaluation on Time Lag Length

For music equipment demand forecasting, generally, the demand data are compared with time series data to make future time series forecasts. Usually, the data from a time period in the past are selected to predict the demand data at a certain point in the future. Therefore, how many historical moments of data are selected is naturally an aspect worth exploring. Here, the selected historical time data is generally called time lag. For choosing different lag time steps, the performance of the model is different. Table3 shows the predicted performance under different lag time steps.Table 3
Evaluation on time lag length.
Time lagMED1MED2RMSEMAERMSEMAE518.116.517.214.51014.612.813.511.71512.310.410.99.12013.911.912.810.92516.413.214.811.6Obviously, as the length of the time lag increases, the performance of the network gradually rises. However, after reaching the peak value, RMSE and MAE will gradually rise again, which will cause performance degradation. On the two data sets, when the time lag step is 15, the corresponding model performance is optimal.


## 4.1. Data set

Taking into account the many unstable factors in the music equipment market, this article selects more mature and stable music equipment sales and demand data from two provinces and cities, and self-made two data sets, namely, MED1 and MED2. The time span of the data set is from January 2015 to December 2019, which contains a relatively complete period of fluctuations in the demand for music equipment, which can be better used for the research and analysis of this article. The proportions of training data and testing data are 70% and 30%, respectively.In this work, RMSE and MAE are two evaluation metrics to evaluate the performance of the proposed model. The detailed calculation formulas are given below:(8)RMSE=1N∑i=1Nyi−y^i2,MAE=1N∑i=1Nyi−y^i,where N is the total number of the samples, yi is the truth value, and y^i is the value which is predicted by model.


## 4.2. Comparison with Other Methods

This work compares the proposed ILSTM method with other prediction methods including SVR, BP, RNN, and LSTM. The result on two data sets is illustrated in Table2.Table 2
Comparison with other methods.
MethodMED2MED2RMSEMAERMSEMAESVR30.524.727.821.2BP25.619.822.518.2RNN19.316.617.415.3LSTM15.612.914.811.7ILSTM12.310.410.99.1Obviously, compared with the other methods listed in the table, ILSTM can achieve the best performance. The RMSE and MAE on the MED1 data set are 12.3 and 10.4, respectively. The RMSE and MAE on the MED2 data set are 10.9 and 9.1, respectively. Compared to the best method of LSTM, RMSE decreased by 3.3 and 3.9, respectively, MAE decreased by 2.5 and 2.6, respectively. These data can verify the effectiveness of our method.


## 4.3. Evaluation on an Optimizer

As mentioned early, this work selects RMSProp as an optimizer of the network. To verify the correctness of this selection, a comparative experiment comparing other optimizers is conducted. The experimental result is illustrated in Figure2.Figure 2
Evaluation on an optimizer.Compared with the SGD and AdaGrad optimizers, the RMSProp optimizer used in this article corresponds to the lowest RMSE and MAE. Compared to the AdaGrad, RMSE decreased by 3.4 and 3.3, respectively, MAE decreased by 2.7 and 2.3, respectively. These data can verify the effectiveness of our selection for optimizers.


## 4.4. Evaluation on Activation Function

This work proposes a mixed ReLU (MReLU) activation function to make the model reflect the ability to train non-linear data. To verify the effectiveness of our designed MReLU, a comparative experiment comparing other activation functions is conducted. The experimental result is illustrated in Figure3.Figure 3
Evaluation on an activation function.Obviously, compared to the traditional ReLU activation function, the performance of ELU and PReLU is better. However, it should be noted that after combining ELU and PReLU, the proposed MReLU activation function can obtain lower RMSE and MAE, and the corresponding prediction performance is also the best. It can be proved that the MReLU activation function designed in this paper is effective and correct and can promote the improvement of network performance.


## 4.5. Evaluation on Preventing Overfitting

As mentioned early, this work combines a dropout mechanism and L2 regular term to prevent the overfitting of the model. To verify the effectiveness and reliability of this strategy, this paper conducts a corresponding comparative experiment. It compares the performance of a single dropout, a single L2 regular term, and a combination of the two. The experimental result is illustrated in Figure4.Figure 4
Evaluation on preventing overfitting.It can be seen that a single dropout or L2 regular term can effectively prevent overfitting, but the performance improvement is very limited. By combining the two and then constraining network training, the problem of network overfitting can be alleviated to the greatest extent, thereby reducing RMSE and MAE as much as possible and improving performance. This also further verifies the effectiveness of the combination of a dropout and L2 regular term in this paper to prevent network overfitting.


## 4.6. Evaluation on Time Lag Length

For music equipment demand forecasting, generally, the demand data are compared with time series data to make future time series forecasts. Usually, the data from a time period in the past are selected to predict the demand data at a certain point in the future. Therefore, how many historical moments of data are selected is naturally an aspect worth exploring. Here, the selected historical time data is generally called time lag. For choosing different lag time steps, the performance of the model is different. Table3 shows the predicted performance under different lag time steps.Table 3
Evaluation on time lag length.
Time lagMED1MED2RMSEMAERMSEMAE518.116.517.214.51014.612.813.511.71512.310.410.99.12013.911.912.810.92516.413.214.811.6Obviously, as the length of the time lag increases, the performance of the network gradually rises. However, after reaching the peak value, RMSE and MAE will gradually rise again, which will cause performance degradation. On the two data sets, when the time lag step is 15, the corresponding model performance is optimal.


## 5. Conclusion

With the progress of society and the popularity of music, people's demand for music equipment is also greater. However, the music equipment market is changing rapidly, and music equipment companies are greatly affected by market changes. In this environment, in order to quickly respond to market changes, music equipment manufacturers and sales companies must make accurate predictions of market demand. The method based on LSTM is the mainstream deep neural network in the research methods of time series problems. Based on the LSTM model, this paper conducts an in-depth study on the demand forecast of music equipment. In order to solve the problems of overfitting, gradient disappearance, model collapse, and other problems in previous experimental studies, this paper proposes an ILSTM prediction model, which is an improvement of the traditional LSTM model. In terms of model structure selection, the dropout mechanism is adopted and the L2 regular term is introduced. In the selection of the activation function, the MReLU function is proposed, which can improve the prediction effect of the model and enhance the applicability of the model. The research results show that the improved LSTM network prediction model is superior to other models in the prediction of music equipment demand, which verifies its effectiveness.



---

*Source: 6580742-2022-03-14.xml*