# A Novel Computer-Aided-Diagnosis System for Breast Ultrasound Images Based on BI-RADS Categories
## 5 Experimental Results and Discussion
### Performance Evaluation with Confusion Matrix  
As mentioned previously, in order to validate the detection performance of the proposed CAD, we compared the segmentation results with Equation (1). Figure 13 shows the contour detection results using different approaches on different ultrasound instruments, including: Model B, Model B, and Model C.  
Figure 13: Performance comparison of the contour detection with the proposed CAD. The first row lists original images; second row lists the processing results with the method defined in Equation (1); and third row lists the contour detection with the proposed CAD system. (**a**,**d**,**g**) Images fetched with Model B; (**b**,**e**,**h**) images fetched with Model B; (**c**,**f**,**i**) images fetched with Model C. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Layer** & **Type** & **Input Size** & **Detail** \\ \hline Layer 0 & Input & 99 \(\times\) 99 & 1 image \\ Layer 1 & Convolution & 47 \(\times\) 47 & K = 7 \(\times\) 7, N = 25 \\ Layer 2 & Rectified Linear Unit & 47 \(\times\) 47 & K = 3 \(\times\) 3, N = 25 \\ Layer 3 & Max Pooling & 23 \(\times\) 23 & K = 3 \(\times\) 3, N = 25 \\ Layer 4 & Convolution & 11 \(\times\) 11 & K = 3 \(\times\) 3, N = 50 \\ Layer 5 & Rectified Linear Unit & 11 \(\times\) 11 & K = 3 \(\times\) 3, N = 50 \\ Layer 6 & Max Pooling & 5 \(\times\) 5 & N = 50 \\ Layer 7 & Fully connected & 100 \(\times\) 1 & 100 node \\ Layer 8 & Fully connected & 5 \(\times\) 1 & 5 class \\ \hline \hline \end{tabular}  
K = mask size = kernel size; N = number of mask.  
\end{table} 
Table 5: Details of the proposed CNN layer detail. ### Performance Evaluation with Confusion Matrix  
In general, confusion matrix is a commonly used index to evaluate the efficiency of the machine learning. In order to validate the performance of the multi-class identification, the predicted and actual BI-RADS categories (ground truth) were compared to generate a confusion matrix [25]. As shown in Figure 14, a 2D confusion matrix is generated in terms of the predicted and actual classification results using different machine learning approaches. Assume that \(K_{1}\), \(K_{2}\),..., \(K_{3}\) are the multiclass label; \(A_{ij}\) denotes that the actual class is \(K_{i}\) for each sample, but the predicted class is \(K_{j}\). According to the confusion matrix, various indices for the performance evaluation can be measured so as to evaluate the classification efficiency. Some popular indicators will be expressed further in terms of the following definitions. Accuracy is used to evaluate the correction percentage for all categories, and the measure is calculated by Equation (4):  
\[Accuracy=\frac{\sum\limits_{i=1}^{n}A_{ii}}{\sum\limits_{i=1}^{n}\sum\limits_{j =1}^{n}A_{ij}} \tag{4}\]  
Precision is its prediction accuracy for a specific class. In other words, it is the measurement accuracy of successful prediction for a sample, and it is calculated by Equation (5):  
\[Precision_{i}=\frac{A_{ii}}{\sum\limits_{k=1}^{n}A_{ki}} \tag{5}\]  
Recall is an accuracy measure of the prediction model (the test model of the classifier) in the measurement of a particular class. In other words, the probability for a certain category is not misclassified, and its definition is denoted as Equation (6):  
\[Recall_{i}=\frac{A_{ii}}{\sum\limits_{k=1}^{n}A_{ik}} \tag{6}\]  
The statistical _F-score_ (_F1-score_) is the harmonic mean calculated by the precision rate and the recall rate. When the _F-score_ is over 70%, the test method will be considered to be more effective, and the measure is calculated by Equation (7):  
\[F\text{-}score_{i}=\frac{2\times Precision_{i}\times Recall_{i}}{Precision_{i}+ Recall_{i}} \tag{7}\]  
In our experiments, there are a total of 106 training samples and 45 testing samples which were selected and identified by an experienced radiologist.