# NewApproach to Predict the Motion Characteristics of Single Bubbles in Still Water
## Materials and Methods
### Machine Learning Approaches
#### 2.3.2 Classification Problem-Solving  
K-nearest neighbours (KNN), random forest, and logistic regression are the common algorithms for classification problem. KNN algorithm achieves the final classification purpose by measuring the distances between the eigenvalues and calculating the difference between the distances. The principle of the KNN algorithm is to find the k distance points closest to the training set and then select the k-nearest neighbours category as its own type. Basically, k is an integer not larger than 20 [33; 34]. The basic parameters of the KNN algorithm include distance metrics, k-values, and decision rules of classification.  
Entering the dataset \(X\) as follows:  
\[X=\{(x_{1},y_{1}),(x_{2},y_{2}),(x_{3},y_{3}),\cdots,(x_{n},y_{n})\} \tag{6}\]  
where \(x\) is the variable and \(y\) is the label.  
Euclidean distance, as shown in Equation (7), is the generally taken distance:  
\[d(x,y)=\sqrt{(x_{1}-x_{2})^{2}+(y_{1}-y_{2})^{2}} \tag{7}\]  
Meanwhile, the classification law is as follows:  
\[y=\mathrm{argmax}_{cj}\sum\
olimits_{x_{i}\in\mathrm{N}_{k}(x)}I(y_{i}=c_{i}) \tag{8}\]  
where i = 1, 2,..., N, j = 1, 2,..., K; for the indication function I, when \(y_{i}=c_{i}\), \(I=1\), otherwise I equal to 0.  
Figure 4: Architecture for the drag coefficient prediction model.  
Figure 3: Architecture for the final velocity prediction model.  
On the other hand, the random forest algorithm selects each feature as a classification criterion. The error of the classification depends on the degree of differentiation of each independent feature and the connection among them.  
By combining the multiple weak classifiers, the final classification result can be obtained through the voting or averaging method. The weak classifier usually chooses the classification and regression tree (CART) decision tree which uses the Gini index to determine the most appropriate branching feature. The definition is shown in Equation (9):  
\[Gini(x)=\sum{}_{k}p_{k}(1-p_{k})=1-\sum{}_{k}p_{k}^{2} \tag{9}\]  
where \(p_{k}\) is the probability of the occurrence of the \(k\)-category, the smaller the Gini, the smaller the uncertainty, and the better the data classification.  
The CART decision tree avoids the over-fitting through pruning, and the loss function (as shown in Equation (10)):  
\[C_{a}(T)=C(T)+a|T| \tag{10}\]  
where \(C(T)\) is the error of the trunk \(T\) in the training data and \(a\) is a trade-off parameter.  
Logistic regression algorithm uses the idea of selection probability to classify the sample. In this study, the softmax regression algorithm was chosen to classify the bubble shape. Softmax regression algorithm converts the features into probabilities. The function relationship is as follows:  
\[P\Big{(}y^{i}=j\big{|}X^{i};\theta\Big{)}=\frac{e^{\theta_{j}^{T}x^{i}}}{\sum _{l=1}^{k}e^{\theta_{l}^{T}x^{i}}} \tag{11}\]  
Meanwhile, the cost function is  
\[J(\theta)=-\frac{1}{m}\Bigg{[}\sum{}_{i=1}^{m}\sum{}_{j=1}^{k}I\Big{\{}y^{i}=j \Big{\}}\log\frac{e^{\theta_{j}^{T}x^{i}}_{j}}{\sum_{i=1}^{k}e^{T}_{i}x^{i}} \Bigg{]} \tag{12}\]  
where the specific form of \(I(x)\) is  
\[I(x)=\left\{\begin{array}{l}0,x=\textit{false}\\ 1,x=\textit{true}\end{array}\right. \tag{13}\]  
Softmax regression algorithm uses the gradient descent method to solve the model parameters. The calculated gradient result is as follows:  
\[\frac{1}{m}\sum_{i=1}^{m}X^{i}\cdot\Big{(}I\Big{\{}y^{i}=j\Big{\}}-P\Big{(}y^{ i}=j\big{|}X^{i};\theta\Big{)}\Big{)} \tag{14}\]  
All of the above-mentioned algorithms were used to classify the bubble shapes under different dimensionless parameters (Re number, Eo number, and We number). Figure 5 shows the overall flow chart for the machine learning implementation in this study.