# Automatic Optimization of Deep Learning Training through Feature-Aware-Based Dataset Splitting
## 3 Materials and Methods
### Proposed Diversity-Oriented Data Split Approaches
#### 3.6.2 Feature Clustering Distance-Based Image Selection (FCDIS)  
The primary step in the FCDIS method is centered on the clustering technique, where features extracted from each data class are structured using the K-means clustering algorithm. Figure 9 illustrates the process that serves as a foundation for the FCDIS splitting method, accompanied by a plot designed to provide an informative schematic representation of the divisions resulting from the application of this process. Figure 8: Visualization of centroid point of features for the class of _“absence of joint cover plate”_ for \(D_{T-Val}^{BDI}\), for exemplification purposes. The blue dots represent the distribution of features, while the red cross is the global centroid. Figure 7: Key steps of PCA. K-means partitions the feature space into \(K\) clusters based on the similarity of the deep features, aiming at the creation of distinct groups of related data points. To optimize the clustering process, a crucial step involves fine-tuning the parameter \(K\), using a precomputed elbow value determination step (Figure 10a), aiding in the identification of the optimal number of clusters necessary to effectively segment the data within a specific class [54,55]. Then, an iterative process is employed for each class to select images for training. The essence of this selection lies in the distance ratio, wherein images positioned in farther clusters from the global centroid are accorded higher consideration. More specifically, a computation step of the distance between each cluster's center and the center of the entire features space is carried out (Figure 10b). The latter center corresponds to the global one, denoted as \(C\), which is the mean of the top-2 principal components in the reduced feature space. This distance is computed using the Euclidean distance formula:  
\[Distance(c_{i},\;C)=\sqrt{\left(c_{ix}-C_{x}\right)^{2}+\left(c_{iy}-C_{y} \right)^{2}} \tag{3}\]  
where \((c_{ix},c_{iy})\) are the coordinates of the \(i\)-th cluster center and correspond to \((C_{x},C_{y})\), the coordinates of the global center. Then, a metric-based distance is used to determine the images' selection size allocation for each cluster, according to the following formula:  
Figure 9: FCDIS process for setting up training and validation subsets: (**a**) presents the main pipeline of the process; (**b**) depicts the pipeline’s general result-oriented concept using 3 abstract clusters as examples, where in red, yellow, green represents the farthest, intermediate, and closer distance to centroid, respectively. Considering the distance of each cluster centroid to a global centroid, the \(D_{T}\) and \(D_{Val}\) are assembled, grounded in the following diversity rule: the farthest the cluster centroid is, the more images are used to compose the \(D_{T}\). \[Image\;Selection\;Size(c_{i,})=\frac{Distance(c_{i,}\;C)}{\sum\limits_{j=1}^{n} Distance(c_{j,}\;C)} \tag{4}\]  
where \(n\) represents the total number of clusters in the class. This ratio is used to determine the relative importance of each cluster in contributing images used for setting up training/validation sets. It is based on the distance of their respective center from the global center, with direct impact. More intuitive results of the FCDIS approach can be seen in Figure 11, considering the bridge defects _\"absence of joint cover plate\"_ class as an example. Upon closer examination, a noticeable visual similarity becomes apparent among images clustered together. This visual consistency proves that the clustering algorithm successfully identified and grouped images that share common features related to the _\"absence of joint cover plate\"_ class. In addition, in Figure 11b, a summary of the images collected for constituting \(D_{\text{Tr}}\), dependent on the distances and sizes of the clusters, is provided. Figure 10: Example of K-means-based clustering process plot perspective, using a computational method for the determination of K value.