# Deep-Learning-Based Computer-Aided Systems for Breast Cancer Imaging: A Critical Review
## 2 Materials and Methods
### Deep Learning Models
#### 2.2.6 Evaluation Metrics  
Different quantitative metrics are used to evaluate the classifier performance of a DL-CAD system. These include accuracy (Acc), Sensitivity (Sen), Specificity (Spe), area under the curve (AUC), F1 score, and a confusion matrix. The statistical equations are shown in Tables 3 and 4.  
Figure 6: An ANN learns by processing images, where each of which contains an input, hidden, and result layer.  
Figure 7: A feed-forward CNN network, where the convolutional layers are the main components, followed by a nonlinear layer (rectified linear unit (ReLU)), pooling layer for feature map extraction, loss function (softmax), and the fully connected layers for classification. The output can be either benign or malignant classes.  
The receiver operating characteristic curve (ROC) is a graph for plotting the true positive rate (TPR) versus a false positive rate (FPR) and is derived from the AUC. The TPR and the FPR are also called sensitivity (recall) and specificity, respectively, as shown in Figure 8.  
The AUC provides the area under the ROC curve and a perfect score has a range from 0.5 to 1. A 100% correct classified version will have an AUC value of 1 and it will be 0 if there is a 100% wrong classification [168].  
Cross-validation is a statistical technique that is used to evaluate predictive models by partitioning the original samples into training, validation, and testing sets. There are three types of validation: (1) hold-out splits (training 80% and testing 20%), (2) three-way data split (training 60%, validation 20%, and testing 20%), and (3) K-fold cross-validation (from 3 to 5 k-fold for a large data set, 10 k-fold for a small dataset), where the data are split into k different subsets depending on their size [65].  
\begin{table}
\begin{tabular}{l c} \hline \hline
**Model** & **Equation** \\ \hline Accuracy & Acc = \(\left(\frac{\text{TP }+\text{ TN}}{\text{TP }+\text{ TN }+\text{ FP }+\text{ FN}}\right)\) \\ \hline Sensitivity & TPR \\ \hline Specificity & \(\text{TNR }=\left(\frac{\text{TN}}{\text{TN }+\text{ FN}}\right)\) \\ \hline Precision & \(\text{Precision }=\left(\frac{\text{TP }}{\text{TP }+\text{ FP}}\right)\) \\ \hline F1 Score & \(\text{F1 Score }=2\times\left(\frac{\text{precision }\times\text{ recall}}{\text{precision }+\text{ recall}}\right)\) \\ \hline MCC & \(\text{ MCC }=\frac{\text{TP }\times\text{ TN }-\text{ FP }\times\text{ FN}}{\sqrt{ \left(\text{TP }+\text{ FP}\right)\left(\text{TP }+\text{ FN}\right)\left(\text{TN }+\text{ FP}\right)\left(\text{TN }+\text{ FN}\right)}}\) \\ \hline \hline \end{tabular}
\end{table}
Table 4: Validation assessment measures.  
Figure 8: The confusion matrix for the ROC. The number of images correctly predicted by the classifier is located on the diagonal. The ROC curve utilizes the TPR on the \(y\)-axis and the FPR on the \(x\)-axis.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Classes**} & \multicolumn{2}{c}{**Predicted Classes**} & \multirow{2}{*}{**Equation**} \\ \cline{2-3} \cline{5-5}  & \(\mathbf{C_{1}}\) & & \(\mathbf{C_{2}}\) \\ \hline \multirow{2}{*}{\(\text{C}_{1}\) (Benign)} & \multirow{2}{*}{TP} & \multirow{2}{*}{FN} & \multirow{2}{*}{TPR = \(\left(\frac{\text{TP}}{\text{TP }+\text{ FN}}\right)\)} \\ \hline \multirow{2}{*}{\(\text{C}_{2}\) (Malignant)} & \multirow{2}{*}{FP} & \multirow{2}{*}{TN} & \multirow{2}{*}{FPR = \(\left(\frac{\text{FP}}{\text{FP }+\text{ TN}}\right)\)} \\ \hline \hline \end{tabular}
\end{table}
Table 3: Confusion matrix for a binary classifier that is used to distinguish between two classes, namely, benign and malignant. TP: true positive; FN: false negative, FP: false positive, TN: true negative, TPR: true positive rate, FPR: false positive rate.