# Efficient Learning of Healthcare Data from IoT Devices by Edge Convolution Neural Networks +
## 5 Evaluation
### Evaluation of EdgeCNN
#### 5.1.1 Performance of the ECG Diagnosis Model Deployed on the Edge Devices  
We evaluate the overall performance of the ECG diagnosis model that is deploy on the edge devices of processor snapdragon 660. After testing the prediction accuracy of normal rhythm (N), AF rhythm (A), other rhythm (O) and noisy recordings (\(\sim\)), the result is shown in Figure 9, which shows the prediction accuracy rate of \"Normal\" is 87%, \"AF\" is 84%, \"Other\" is 81% and \"Noisy\" is 60%.  
The figure describes that there are three different categories of ECG which are easily misjudged as \"Other\". In addition, the ratio of \"AF\" to \"Other\" is the same, 11%. \"Noisy\" has the lowest prediction accuracy, because the majority of them are classified as \"Other\" and \"Normal\". In the true production environment, to avoid the transmission of numerous amount of data in the network environment, most of the predictions of the hybrid architecture are completed at the edge devices. This strategy can decline the ratio of noise generated by data transmission and decrease the predicted error to some degree.  
Then, we compare the accuracy of the model in predicting with \(F_{1}\)-score, the cost of storage and memory and predict time. \(F_{1}\)-score is a measure of the accuracy of the model in predicting atrial fibrillation. Its calculation formula is Equation (6):  
\[F_{1}=\frac{1}{4}\times\left(\frac{2\times Nn}{\Sigma N+\Sigma n}+\frac{2\times Aa }{\Sigma A+\Sigma a}+\frac{2\times Oo}{\Sigma O+\Sigma o}+\frac{2\times Pp}{ \Sigma P+\Sigma p}\right). \tag{6}\]  
The definition of each variable is shown in Figure 10.  
Figure 10: Counting rules for the numbers of the variables.  
Figure 9: The prediction accuracy for each category.  
The results of the ECG diagnosis model and the optimal model in 2017 CinC [35] is shown in Table 2. [PERSON] et al. [35] used a CNN with a 16-layer convolutional layer and obtained a final score of 0.83. However, this complex model structure needs a larger volume and causes higher computational complexity. Although our streamlined and efficient model acquires a score of 0.78, which is a little bit lower than the optimal model, the optimal model's physical storage space reached 4.95 MB after being persisted, which is almost 1.5-fold larger than our model. Moreover, the memory at runtime reached 74 MB, which is 23 MB more than our model.  
The definition of prediction time is the time which is required for the ECG diagnosis model from receiving data to output the final prediction result. It means that the faster diagnosis speed can decrease the general delay of the system. Hence, we should ensure a shorter prediction time and acquire the basic accuracy which is higher than cardiologists. The complex calculations make the prediction time of 2017 CinC optimal model about 100 ms longer than our model. Such a long delay is definitely catastrophic in the field of smart healthcare.  
Therefore, our model absolutely occupies an advantage from the perspective of the size of the storage space occupied by the model, the memory occupied by the runtime and the inferred time, which makes the model easily deploy on resource-constrained edge devices. it can fit the new architecture of the cloud-edge-integrated home smart healthcare proposed by us.