# EL-MTSA: Stock Prediction Model Based on Ensemble Learning and Multimodal Time Series Analysis
## 3 Model Construction
### Hyperparameter Optimization Based on Bayesian Inference  
As the complexity of the model increases, the selection of hyperparameters becomes a non-trivial challenge, and the appropriate configuration of hyperparameters is crucial to ensuring optimal model performance. In this study, we employ a Bayesian optimization algorithm (BOA) [45] for hyperparameter tuning. The BOA is a robust global optimization technique that iteratively refines results by leveraging an objective optimization function throughout the optimization process. The objective optimization function can be formulated as follows:  
\[f(\omega)=\sqrt{\frac{\sum_{i=1}^{m}\left(\hat{y}_{i}(\omega)-y_{i}\right)^{2} }{m}} \tag{30}\]  
where \(y_{i}\) represents the true value, \(\hat{y}_{i}(\omega)\) represents the predicted value, and \(m\) represents the length of the input time series. The objective function is minimized as follows:  
\[\omega^{*}=\underset{\omega\in W}{\text{argmin}}f(\omega) \tag{31}\]where \(W\) is the set of all parameters, \(\omega^{*}\) denotes the best obtained parameter, and \(\omega\) is the set of hyperparameter combinations.  
During the parameter tuning process, the Gaussian function is adopted as the assumed distribution of the prior function. Subsequently, the next point for evaluation is determined in the posterior process based on the acquisition function. Gaussian processes represent an extension of multivariate Gaussian distributions and can be characterized by their mean and covariance functions as follows:  
\[f(\omega)\sim GP(\mu(\omega),K(\omega,\omega^{\prime})) \tag{32}\]  
where \(\mu(\omega)\) is the mean of \(f(\omega)\) and \(K(\omega,\omega^{\prime})\) is the covariance matrix of \(f(\omega)\). Initially, it can be expressed in the following way:  
\[K=\begin{pmatrix}k(\omega_{1},\omega_{1})&\cdots&k(\omega_{1},\omega_{i})\\ \vdots&\ddots&\vdots\\ k(\omega_{i},\omega_{1})&\cdots&k(\omega_{i},\omega_{i})\end{pmatrix} \tag{33}\]  
During the optimization process, the covariance matrix is continuously updated with each iteration as new samples are incorporated into the dataset. The specific update rule for the covariance matrix is as follows:  
\[K^{\prime}=\begin{pmatrix}K&k^{T}\\ k&k(\omega_{i+1},\omega_{i+1})\end{pmatrix} \tag{34}\]  
\[k=[k(\omega_{i+1},\omega_{1}),k(\omega_{i+1},\omega_{2}),\cdots,k(\omega_{i+ 1},\omega_{i})] \tag{35}\]  
The posterior probabilities can be obtained from the updated covariance matrix.  
\[P(f_{i+1}|D_{i+1},\omega_{i+1})\sim N(\mu_{i+1}(\omega),\sigma_{i+1}^{2}( \omega)) \tag{36}\]  
where \(D\) is the observed data, \(\mu_{i+1}(\omega)\) is the mean of \(f(\omega)\) at step \(i+1\), and of \(\sigma_{i+1}^{2}(\omega)\) is the variance of \(f(\omega)\) at step \(i+1\).  
Through the computation of the mean matrix and covariance matrix, we observe that the sampling function derived from the joint posterior distribution can significantly accelerate the determination of parameters and minimize the wastage of resources. For this purpose, we select the upper confidence bound (UCB) as the sampling function, which is formulated as follows:  
\[UCB(\omega)=\mu(\omega)+\lambda\sigma(\omega) \tag{37}\]  
\[\omega_{i+1}=\text{argmax}UCB=\text{argmax}\mu(\omega)+\lambda_{i+1}\sigma_{i }(\omega) \tag{38}\]  
where \(\omega_{i+1}\) is the hyperparameter chosen in step \(i+1\), \(\mu(\omega)\) and \(\sigma_{i}(\omega)\) are the mean and covariance of the joint posterior distribution of the objective function obtained in the Gaussian process, respectively.