# Empowering Healthcare: TinyML for Precise Lung Disease Classification
## 4 Results and Discussion  
In this section, we compare the performance of the three models by using metrics such as accuracy, precision, recall, F1-score, and Area Under the Curve (AUC), as well as model size, inference time, and peak RAM usage, which are critical for deployment on TinyML devices. The dataset was split into 72% training, 10% validation, and 18% testing. Each model was trained and quantized by Edge Impulse for deployment suitability. Table 8 shows the results of our experiments. the custom CNN model achieved the highest performance, with an accuracy of 96% on the test set and an AUC of 0.96. In contrast, the CNN Edge Impulse model, while faster and less resource-intensive, demonstrated lower accuracy, 85%, and an AUC of 0.85. This difference can be attributed to the simpler architecture of the Edge Impulse model. The custom LSTM model, however, achieved lower accuracy, 90%, compared with the CNN. The confusion matrix in Table 9 provides a detailed evaluation of the classification performance of the CNN, LSTM, and CNN-EDGE-IMPULSE models on the test set data. The custom CNN outperformed the other models, correctly classifying 94.1% of \"Asthma\" cases and 98.3% of \"Normal\" cases, while the LSTM achieved 82.4% and 98.3%, respectively. CNN Edge Impulse, while more resource-efficient, had the lowest performance, with accuracy of 76.5% for \"Asthma\" and 88.1% for \"Normal\". \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1-Score** & **Loss Value** &
\begin{tabular}{c} **Area Under** \\ **ROC Curve** \\ \end{tabular} \\ \hline
**Custom CNN** & & & & & & \\ Validation & 100\% & 100\% & 100\% & 100\% & 0.03 & 1.00 \\ Testing & 96\% & 97\% & 97\% & 97\% & - & 0.96 \\ \hline
**CNN Edge Impulse** & & & & & & \\ Validation & 92\% & 92\% & 92\% & 92\% & 0.24 & 0.91 \\ Testing & 85\% & 86\% & 86\% & 86\% & - & 0.85 \\ \hline
**Custom LSTM** & & & & & & \\ Validation & 93\% & 93\% & 93\% & 93\% & 0.28 & 0.93 \\ Testing & 90\% & 93\% & 92\% & 92\% & - & 0.90 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Model evaluation on validation and testing sets. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & **Asthma** & **Normal** & **Asthma** & **Normal** & **Asthma** & **Normal** & **Uncertain** \\  & **(CNN)** & **(CNN)** & **(LSTM)** & **(CNN-EDGE-IMPULSE)** & **IMPULSE)** & \\ \hline
**Asthma (CNN)** & 24.1\% & 5.9\% & 0\% & 0\% & 0\% & 0\% \\
**Normal (CNN)** & 17.7\% & 0.85\% & 0\% & 0\% & 0\% & 0\% \\
**Asthma (LSTM)** & & & & & & \\
**Normal** & 0\% & 0\% & 17.6\% & 0\% & 0\% & 0\% \\
**LSTM** & & & & & & \\
**Asthma** & 0\% & 0\% & 0\% & 0\% & 76.5\% & 20.6\% & 29\% \\
**(CNN-EDGE-IMPULSE)** & & & & & & \\
**Normal** & 0\% & 0\% & 0\% & 0\% & 10.2\% & 33.9\% & 1.2\% \\
**(CNN-EDGE-IMPULSE)** & & & & & & \\
**F1-score** & 0.96 & 0.97 & 0.89 & 0.94 & 0.79 & 0.88 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Confusion matrix of test set data. Green: correct classifications, Red: misclassifications. Table 10 compares the models based on resource consumption, as estimated by the Edge Impulse cloud platform on the Arduino Nano 33. The CNN Edge Impulse model demonstrates its clear advantage for low-power, resource-constrained environments, requiring only 4.5 KB of RAM. However, this comes at the cost of lower classification performance. The custom CNN, which uses more RAM (12 KB) and has a longer inference time, strikes a balance between resource usage and accuracy, making it a more suitable option when both high classification performance and moderate resource usage are required. On the other hand, the LSTM model, while providing good accuracy (90%) and the ability to process sequential data, has the highest resource demands, consuming 23.2 KB of RAM. This makes it less practical for highly resource-constrained devices. Table 11 presents a comparative analysis between our proposed CNN model and similar works that utilize Edge ML models.