## 2 Materials and Methods
### Hyperparameter Optimization  
Tuning hyperparameters is an optimization problem with an unknown objective function. As can be seen in Equation (5), where \(x\) denotes a hyperparameter setting, \(x_{opt}\) means the best choice and \(f\) represents performance. \[x_{opt}=\underset{x\in X}{argmax}f(x) \tag{5}\]  
There is a range of methods that can be used to accomplish this. Grid Search [35], for example, is a method for searching exhaustively over a portion of a hyperparameter space for a certain algorithm. The number of parameter combinations will grow exponentially as additional hyperparameters are introduced. As a result, the procedure will take a long time to complete. Random Search [35] is another method for locating valuable hyperparameters. Random Search, unlike Grid Search, tries out different combinations of parameters at random. When computing, this causes a high variance. These two approaches are unable to learn anything from the assessed hyperparameter sets during the optimization procedure. In this experiment, an efficient strategy known as BO [36] is adopted to select the optimum hyperparameters. BO incorporates previous knowledge about the objective function and updates posterior knowledge, increasing accuracy and reducing model loss. The optimization is carried out in accordance with Bayesian Theorem [37], which is enshrined in Equation (6), where \(M\) denotes the model and \(O\) denotes the observation. \[P(M|O)=(P(O|M)P(M)/P(M)\]where \(P(M)\) describes the prior probability of \(M\), \(P(M|O)\) is \(M\)'s posterior probability given \(O\), and \(P(O|M)\) is \(O\)'s likelihood given \(M\). By using BO, one can find the objective function's minimum on a bounded set. The CNN model with additional evaluations for the objective function is adequate for making the right choices. By reducing the error in validation, BO leads to optimal CNNs in the current study. The first critical step in the BO process is to use a Posterior distribution with a GP to update the prior Function F results. GP provides a set of observed previous possible values transformed directly into posterior probabilities on the functions. This scenario allows us to apply GP in our multi-classification problem. The next step is to use an acquisition function to pick the optimum point for F. Following that, BO identifies the suggested sampling points as determined by the acquisition function. The Expected Improvement (EI) acquisition function is used at the points where the objective function is maximum and the location where the uncertainty of the prediction is high. The mathematical representation of EI function is shown in Equation (7), where \(MSP\) denotes the sample points' mean, \(OFH\) means the Optimization Function at the highest point. The optimization function has a search space limitation represented as \(L\). The Cumulative Distribution, denoted by \(CD\), takes into account all of the points up to the point sample under Gaussian distribution. \[Expected\ Improvement=((MSP-OFH)-L)(CD) \tag{7}\]  
In order to obtain results in the validation set, it is essential to use an objective function. The critical step after that is to update the Gaussian distribution network. Several iterations of this process are completed in order to tune the validation set for creating optimized hyperparameters that work best for our classification problem. In this paper, 60 iterations were performed. The flow of BO algorithm related to our proposed CNN network is summarized in Figure 4.  
The hyperparameters chosen for optimization in this experiment are:  
The activation function for the convolutional layers and the first three dense layers. The dropout rate. The number of nodes in the first three dense layers. The optimizer. The batch size. For each of these hyperparameters, the valid search ranges are predefined. \"ReLU, SELU, Tanh, ELU, and Sigmoid\" are the activation functions specified for the convolutional layers and the first three dense layers. ReLU is a popular activation function in CNN.