# Time Series Forecasting Method Based on Multi-Scale Feature Fusion and Autoformer
## 3 Results
### Bayesian Hyperparametric Optimization Experiments  
In hyperparameter optimization, genetic algorithms have the advantage of global search in high-dimensional parameter spaces and avoiding local optima. However, they require generating and evaluating many individuals each generation, consuming significant computing resources and increasing optimization time. In time-series forecasting research, many scholars have enhanced forecasting accuracy using Bayesian hyperparameter optimization. For example, [PERSON] et al. [40] improved BNN-based electricity load forecasting accuracy with this algorithm. [PERSON] et al. [41] used it to boost the accuracy of the Attention-LSTM method for room temperature forecasting. [PERSON] et al. [42] enhanced the accuracy of the BODE in multiple fields through Bayesian optimization. These studies show the algorithm's good performance in time-series forecasting. After evaluation, we chose the TPE-based Bayesian optimization algorithm. It uses computing resources more efficiently, quickly exploring the hyperparameter space with less consumption. Especially with large hyperparameter spaces, TPE converges to good solutions quickly, greatly improving optimization efficiency. Bayesian optimization algorithm (BOA) is a model-based method that builds conditional probability models of validation performance via surrogate functions [43]. Its core includes five elements: hyperparameter space, objective function, acquisition function, evaluation history, and surrogate function. The TPE-based method avoids re-evaluating poor hyperparameters by tracking historical assessments, unlike grid or random search. The core innovation of the TPE method is the use of non-parametric density estimation technology, whose mathematical expression is shown in Formula (15). Here, \(\theta\) represents the hyperparameters to tune, \(\mathcal{H}\) is the hyperparameter space, \(l(\theta)\) and \(g(\theta)\) are kernel density estimates (KED) based on historical good and ordinary samples, and \(\mathcal{D}\in\{(\theta_{t},\mathcal{L}_{\cup})\}\) is the observed data from the search process. The algorithm splits historical data into a good subset \(\mathcal{D}_{\text{good}}\) and a bad subset \(\mathcal{D}_{\text{bad}}\), building separate KED models. This dual-density approach better guides the exploration and exploitation of the hyperparameter space. \[\theta_{t}=\underset{\theta\in\mathcal{H}}{\text{argmax}}\frac{l\Big{(}\theta \Bigm{|}\mathcal{D}_{\text{good}}\Big{)}}{g(\theta\Bigm{|}\mathcal{D}_{\text {bad}})} \tag{15}\]  
In this paper, to fully exploit MD-Autoformer's potential in time series forecasting, we conduct experiments using the TPE-based Bayesian hyperparameter optimization algorithm, targeting MSE minimization. Algorithm 1 outlines the simplified TPE-based Bayesian optimization algorithm (BOA). Table 12 lists the key optimization parameters and their semantic interpretations for this study. The Dropout parameter controls the ratio of randomly dropped neurons during training to prevent overfitting. The Learning rate determines the step size for parameter updates. The train epoch indicates the number of training iterations. The MSFN factor is the scaling coefficient for channel grouping in the multi-scale feature fusion network. During experiments, models are trained and evaluated using hyperparameter combinations generated by the BOA, which dynamically adjusts the sampling strategy based on previous results to approach the optimal solution. The hyperparameter optimization results of different datasets are presented in Table 13. Judging from the experimental results, in terms of Dropout settings, the larger the scale of the dataset, the richer the information it contains. An excessively high Dropout rate will lead to the loss of data information. For relatively small-scale datasets such as Apparel Sales, appropriately increasing the Dropout rate can effectively prevent the model from over-fitting.