# An Intelligent Radomic Approach for Lung Cancer Screening
## 3 Results
### Model Verification  
In order to statistically evaluate the reproducibility our system, we have conformed an independent set of test patients from our database and the LIDC-IDRI public database. From our database we used one benign and eight malignant nodules. Regarding the LIDC-IDRI database, since it was not collected to evaluate malignancy, scans are, in general, of a too low quality to assess malignancy. Following [19, 20], we selected cases fulfilling the minimum acquisition requirements that allow radiological assessment of malignancy, which are slice thickness \(<=2.5\), resolution \(<=0.71\), except in case thickness is \(<=1.5\), that resolution can be \(<=0.86\) and only taking in consideration those nodules that have been diagnosed through a biopsy as benign or malign. After this filtering, 18 cases with diagnosis (five benign and 13 malign) were selected. In this way, the independent set of test patients is conformed by a total amount of 27 nodules with 6 benign and 21 malign.  
We have compared Model3 with state of the art methods which include the three type of approaches: radiomics [5], machine learning [6] and deep CNN [8, 9, 10, 11, 12]. In order to compare to the results reported for each of them, we have computed the following metrics from true positive, \(TP\), true negative, \(TN\), false negative, \(FN\), and false positive, \(FP\) diagnosis at nodule level:  
\[\text{Sensitivity}=100\cdot\frac{TP}{TP+FN} \tag{12}\]  
Sensitivity measures the percentage of correctly diagnosed malignant nodules:  
\[\text{Specificity}=100\cdot\frac{TN}{TN+FP} \tag{13}\]  
Specificity measures the percentage of benign nodules correctly identified:  
\[\text{Accuracy}=100\cdot\frac{TP+TN}{Number\ of\ Nodules} \tag{14}\]  
for _Number of Nodules_ denoting the total amount of nodules. The accuracy measures the percentage of correctly diagnosed nodules (both malign and benign nodules) among the total number of nodules in the dataset:  
\[\text{F1 Score}=100\cdot\frac{2\cdot Prec\cdot Rec}{Prec+Rec} \tag{15}\]  
for _Rec_, _Prec_ denoting, respectively, the precision and recall at diagnosis level:  
\[Rec=100\cdot\frac{TP}{TP+FN}\ \ \ \ \ Prec=100\cdot\frac{TP}{TP+FP} \tag{16}\]  
The metric (15) measures the trade-off between recall and precision, and in general, a higher F1-score means a better performance. We also computed the receiver operating characteristic (ROC) curves and the area under the curve (AUC).  
Table 9 shows the metrics for state of the art methods grouped according to the type of approach (Radiomics, Machine Learning, Deep CNN) and our method. As in Table 8, best performances are in boldface. Table 9 reports the metrics obtained by Model3 in our test set together with the results reported by each state of art method using their datasets. We also report the number of parameters of each method as indicator of its complexity and computational and data cost for training. Our method outperforms in Accuracy, Sensitivity and F1 Score. In computer-aided diagnose, sensitivity is significant because correctly finding out patients with malignant nodules is crucial. Besides, the highest F1 Score implies that our method achieves the best trade-off between precision and recall. Our method has a splendid compromise between the performance of the system and the number of trainable parameters. A remarkable point compared to Deep CNN approaches, is that, our method needs strongly less samples to train the model, which is a must in medical imaging.