# A Real-Time Nut-Type Classifier Application Using Transfer Learning
## 2 Materials and Methods
### Performance Evaluation  
The evaluation environment for the performance investigation included a set of development tools and a desktop application developed in this study using Python programming language. The object-oriented application development approach was used during the development process. Five different learning models, including four TL models and a custom CNN model, were embedded into the application. The learning models evaluated in the study were developed using Jupyter Notebook and Spyder IDE via the Anaconda software. The TensorFlow, Pandas, NumPy, and Keras libraries were utilized to train the models. After training the models, the performance metrics were calculated and plotted using the Matplotlib library. The OpenCV library was also used to capture the real-time image data from the camera hardware. The desktop application consisted of the models with the optimal hyperparameters among the models trained on Jupyter Notebook. The interaction between the development tools included in the evaluation environment is shown in Figure 3.  
The application was able to recognize and classify only one type of nut at a time. In other words, the classification of a mixture of different nuts was not considered in this study. An example of the desktop application execution can be found in Figure 4. The application window displays the streaming captured via the camera in real time. The OpenCV codes developed in the application recognized the nuts, and the learning models trained in the study recognized the type of nuts. The accuracy of the proposed recognition was also displayed at the top of the video streaming. For example, as shown in the figure, the application detected that the nuts were roasted chickpeas with 95.5% accuracy. The application was tested on a computer with a 64-bit Windows 10 operating system and a speed of 3.7 GHz CPU. For the detection of nuts, an embedded camera with the ability to record video in real time (1080p) was used. The application was ready to be deployed on a kiosk machine, a terminal hardware, or another type of smart devices that support Python run-time specifications. A customization of the source codes may also be performed to use the application in a nut shop. The performance of different learning models was evaluated with respect to a set of parameters, including loss, accuracy, validation loss, validation accuracy, and _F1-score._ For this purpose, two different strategies were established to compute the performance metrics. Firstly, all data were used for both training and testing to calculate loss and accuracy. Secondly, 50 samples from each dataset were randomly selected to obtain a test set to evaluate the model trained with the remaining 200 samples and calculate the validation loss and validation accuracy. In addition, the F1-score values for the test data are calculated with respect to different learning models trained. The loss values given in [31] are calculated according to Equation (1), as given below  
\[L_{CE}=-\sum_{i=1}^{n}t_{i}\log\left(p_{i}\right) \tag{1}\]  
for \(n\) classes, where \(t_{i}\) is the truth label, and \(p_{i}\) is the maximum probability value for belonging to the \(i\)th class. Figure 3: The interaction of development tools used in the evaluation environment. The accuracy parameters were calculated with respect to true-positive (_TP_), true-negative (_TN_), false-positive (_FP_) and false-negative (_FN_) predictions. The concrete expression is given in [31] and formulated by Equation (2) as follows:  
\[Accuracy=\frac{(TP+TN)}{TP+TN+FP+FN} \tag{2}\]  
The _F1-score_ parameter is calculated with respect to the precision value and recall value, as introduced in [31] and stated in Equation (3):  
\[F1-score=2\times\frac{Precision\times Recall}{Precision+Recall} \tag{3}\]  
where the precision value is calculated as \(\frac{\sum TP}{\sum TP+\sum FP}\), and the recall value is calculated as \(\frac{\sum TP}{\sum TP+\sum FN}\), according to [31].