# Smart Embedded System for Skin Cancer Classification
## 3 Dual-Model Design  
The objective of using two models incrementally during inference is to have a system with a performance and energy consumption close to that of the smaller model and an accuracy close to that of the larger model. The pair of models determines the performance, energy and accuracy of the final system.  
An important component of the dual-model inference is the confidence predictor, which determines the confidence of image classification. An image classified with low confidence needs to be classified again with the more accurate model. The confidence predictor requires a function to determine the confidence of a classification and a threshold that establishes the border between confident and non-confident.  
Some works consider the softmax output probabilities and calculate the difference between the highest probability and the accumulation of a subset of the remaining probabilities to assess the confidence of the model outcome [6]. This metric, however, is relatively weak in determining the robustness of the model. Instead, entropy is considered to be a more robust metric [33]. Both methods were analyzed and tested in the context of this work and the entropy metric provided better results. In addition, calculating the entropy for a small set of classes is quick.  
In this work, the confidence of a classification, \(conf_{top1}\), was calculated as the absolute value of the entropy of the probability output array of the final layer as follows:  
\[Conf_{top1}=|-\sum_{i}p_{i}\times log(p_{i})| \tag{5}\]  
where \(p_{i}\) is the probability associated with a class. The classification of an image is considered confident if \(Conf_{top1}\leq th_{entropy}\), where \(th_{entropy}\) is the entropy threshold of the confidence. The threshold determines the ratio between the accuracy and inference runtime. As the threshold is reduced, the number of images that need to be classified with the more accurate model increases, which increases the accuracy of the dual model. However, since the more accurate and slower model is executed more times, the average runtime of the dual-model inference increases.  
Given a pair of models and an accuracy tolerance, Tacy (relative to the accuracy of the most accurate model), a tool was developed to find the confidence threshold with an accuracy within the accuracy error and a lower inference runtime. The final dual model should be faster and within the accuracy tolerance.  
The tool developed in Python receives two trained models with different accuracy levels and the accuracy range. It then finds the fastest multi-model configuration with an accuracy within the accuracy range, according to the design flow illustrated in Figure 2.  
The design flow has the following steps:  
1. Trained Models--The models are previously trained;
2. Set minimum entropy threshold--The minimum initial entropy threshold and the entropy increment are defined. As explained above, this threshold is used by the confidence predictor. In this work, both parameters were initialized at 0.1;
3. Run dual inference--The dual-model inference is run with the entropy threshold set previously. The task runs the inference with the smaller model and the training dataset. For each sample, it finds the entropy. If the entropy is higher than the entropy threshold, it runs the second model. This process allows us to determine the accuracy of the dual model;
4. Determine accuracy--The accuracy corresponds to the number of samples correctly classified by the first model and by the second model, whenever it runs, divided by the total number of samples. If the accuracy is within the accuracy tolerance, the entropy threshold is increased by 0.1 and the process repeats;
5. Estimate speedup and accuracy--The speedup is estimated as follows: \[\frac{M2_{OPS}}{M1_{OPS}+M2_{OPS}\times S2}\] where \(MX_{OPS}\) is the number of operations of the model, and \(S_{X}\) is the percentage of inputs executed by model \(X\). The accuracy is given by the previous step.