# Tomato Maturity Estimation Using Deep Neural Network
###### Abstract  
In this study, we propose a tomato maturity estimation approach based on a deep neural network. Tomato images were obtained using an RGB camera installed on a monitoring robot and samples were cropped to generate a dataset with which to train the classification model. The classification model is trained using cross-entropy loss and mean-variance loss, which can implicitly provide label distribution knowledge. For continuous maturity estimation in the test stage, the output probability distribution of four maturity classes is calculated as an expected (normalized) value. Our results demonstrate that the F1 score was approximately 0.91 on average, with a range of 0.85-0.97. Furthermore, comparison with the hue value--which is correlated with tomato growth--showed no significant differences between estimated maturity and hue values, except in the pink stage. From the overall results, we found that our approach can not only classify the discrete maturation stages of tomatoes but can also continuously estimate their maturity. Furthermore, it is expected that with higher accuracy data labeling, more precise classification and higher accuracy may be achieved. tomato maturity; convolutional neural networks; deep learning; mean-variance loss; robot harvesting +
Footnote †: journal: Medical  
detection have been demonstrated for various crops. Tomato is one of the main target crops for fruit detection in robotic harvesting, as it is an economically significant horticultural crop worldwide with steadily increasing production. For fruit detection in robotic harvesting, it is necessary to determine not only the position of the target fruit, but also whether it is ripe. The maturity of a tomato is visually expressed in terms of its fruit size and color as it grows. It is easier to qualitatively determine how mature a fruit is from the color change of its surface, which gradually changes from green to red [2]. For this reason, tomato maturity is commonly classified into 4-6 stages based on red color occupancy; thus, tomato maturity can be estimated or classified based on color features, which various studies have implemented simultaneously with fruit detection or segmentation. Most studies have conducted detection using supervised learning methods, in which the model was trained using a labeled dataset. There are several limitations associated with learning tomato maturity using manually labeled images, due to the inaccuracy caused by ambiguity. In the case of tomato maturity, it is difficult to determine the specific class and maturity level for the classification task, due to the continuous change in the color of a tomato skin. There is a color occupancy-based guide for determining tomato maturity level; however, it is hard to obtain consistent results, as they vary depending on labeling quality [3]. Therefore, rather than being limited to a set class through classification learning, a method that can evaluate maturity in terms of a continuous value is expected to be effective for use in various environments. The purpose of this study was to estimate tomato maturity continuously, based on the use of a deep neural network. The maturity distribution between samples within a specific class was learned intrinsically using a label distribution learning method and mean-variance loss, which has been demonstrated [4] as being capable of estimating the age of humans from facial images. The CNN structure was used to implement the classification model, and the model was trained using images collected in a greenhouse. Tomato maturity was evaluated using the expected value of the probability distribution from the model output. The novelty of this study is that the proposed approach can be used to estimate the continuous maturity of tomatoes from images, although the model was trained in a supervisory manner using specific maturity level classes. We validated the continuous maturity results by comparing the distribution with color information by class.