## 3 Materials and Methods
### Presented CAD
#### 3.3.5 Multi-Layer Feature Combination and Selection  
In this step, the attribute vectors from three different deep neural networks are sequentially combined for each layer, resulting in three unique feature combinations for Layers 1, 2, and 3. This fusion procedure seeks to investigate the possibility of boosts in performance attainable via multi-architecture feature incorporation and comparative combination sets assessment. Subsequent to the production of the feature combination sets of Layer 1, 2, and 3 separately, a thorough concatenation step is executed to merge these three combined sets of the three deep neural networks, yielding an extensive multidimensional representation. Given the computational challenges associated with high-dimensional data, an advanced FS technique is essential. FS is a vital computational method for recognizing and keeping the most valuable variables while progressively discarding redundant or unimportant ones [60]. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**CNN Model** & **Layer 3 Number of Features** & **Layer 2 Number of Features** & **Layer 1 (Before Reduction) Number of Features** & **Layer 1 (After Reduction) Number of Features** \\ \hline
**MobileNet** & 2 & 1280 & 62,720 & 980 \\ \hline
**ShuffleNet** & 2 & 544 & 26,656 & 417 \\ \hline
**EfficientNetB0** & 2 & 1280 & 62,720 & 980 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The dimensions of the feature vectors obtained from the multiple layers of the three deep neural networks. The minimum redundancy and maximum relevance (mRMR) methodology [61] is utilized to perform this advanced FS procedure, using mutual information as the main analytical metric. The mRMR method systematically assesses feature replication and relevance to selectively identify feature subsets that exhibit maximal class correlation and minimal redundancy among features using mutual information (MI). This FS approach expedites model training, reduces overfitting risks [62], and improves computational effectiveness and predictive accuracy within the machine learning model. MI measures the association between two variables, indicating the extent to which knowledge of one variable diminishes uncertainty regarding the other. The calculation employs Equation (5), which depends on the marginal probabilities \(P(A)\) and \(P(B)\), in addition to the joint probability \(P(A,B)\) for the associated attributes. \[I(A;B)=\sum_{b\in B}\sum_{a\in A}p(a,b)\text{log}\bigg{(}\frac{p(a,b)}{p(a)p(b )}\bigg{)} \tag{5}\]  
A different technique emphasizes the identification of features most pertinent to the target class, as delineated by the maximum relevance criterion in Equation (6). This approach aims to identify features \(X_{i}\) that demonstrate the highest MI \(I\) (\(X\) ; \(C\)) with the class \(C\). Although effective, dependence on maximum relevance alone can lead to significant redundancy among chosen features, potentially impairing model performance. To resolve this, the minimum redundancy criteria, as delineated in Equation (7), is integrated. It reduces redundancy by evaluating pairwise MI \(I(X_{i},X_{j})\) between variables in the specified set, making sure the attributes of choice are maximally apart from each other [63]. \[\text{max}D(X,C);D=\frac{1}{|X|}\sum_{X_{i}\in X}I(X_{i},\ C) \tag{6}\]  
\[\text{min}R(X);R=\frac{1}{|X|}\sum_{X_{i},\ X_{j}\in X}I(X_{i},\ X_{j}) \tag{7}\]  
The integration of these two principles--maximum relevance and minimum redundancy--constitutes the mRMR framework. Equation (8) consolidates both criteria and employs a greedy algorithm for the iterative selection of features. In this methodology, \(S\) denotes the collection of previously selected features, and at each iteration, the algorithm seeks to optimize the disparity between the significance of a prospective feature \(I\) (\(X\) ; \(C\)) and its mean redundancy in relation to the features within \(S\).