# Tomato Maturity Estimation Using Deep Neural Network
## 3 Materials and Methods
### Deep Neural Network Model  
For this study, tomato maturity was estimated using a deep neural network (DNN). The maturity was evaluated continuously based on the implicitly learned feature distribution in each specific class during DNN training for the classification of the four maturity stages. The maturity stages of the tomato can be divided into several classes, and the results depend on the data type and quality, as well as the characteristics of the annotator who conducts the image labeling. For this reason, there is some variety within each class, which is called the label distribution [22]. This challenge is also being actively dealt with for human age estimation from facial images [23], due to the great diversity in faces even at the same age. In this case, the label in the classification task is only a clue, and it is necessary to learn where the sample is positioned in the internal distribution of the specific class. Mean-variance loss is an approach for label distribution learning (LDL), which can be implemented easily by adding the mean and variance differences to the probability between the ground truth and estimated values [4]. In this way, the system can learn a label distribution that has a mean value close to the ground truth label. Figure 3 shows the architecture of the DNN used to estimate tomato maturity in this study. The model has a convolutional neural network (CNN)-based structure for a simple classification task, consisting of four layers to extract the features. With this structure, it is not difficult to classify the four categories that are likely to depend on color features. Figure 2: The tomato object annotation and background elimination process. The architecture consists of four convolutional layers, one fully connected layer, and a classifier (softmax), with each conv-net (convolutional layer) consisting of max pooling and rectified linear unit (ReLU) functions. The output of the final convolutional layer includes 256 feature maps, where each feature map has a size of \(8\times 8\) pixels. The fully connected layer for classification has \(256\times 8\times 8\) input neurons with four output neurons. The output values are then converted into probabilities with regard to the four maturity stages. Back-propagation  
In the training stage, tomato images were used as inputs for the classification model and the features were represented through hierarchical convolutions. The output, expressed as probability distributions between classes (maturity stages), was then compared with one-hot encoded labels. The loss, that is, the numerical value obtained from this comparison, was backpropagated into the DNN in order to update the weights. If tomato maturity is evaluated using only the results of general classification learning, only one of the four stages is selected, which makes it difficult to reflect the various intermediate maturity stages. In order to implicitly learn the distribution within a class during classification, the weights in the DNN were updated considering not only cross-entropy loss (softmax loss) but also mean and variance losses, as shown in Equations (1) and (2), respectively [4]. The mean-variance loss has the advantage of being able to learn the probabilistic distribution of the class and can reflect the inherent ambiguity when the class has an inaccurate label due to the ambiguous selection of classes, which may be the case for tomato maturity. \[L_{m}=\frac{1}{2N}\sum_{i=1}^{N}(m_{i}-y_{i})^{2} \tag{1}\]  
\[L_{v}=\frac{1}{N}\sum_{i=1}^{N}v_{i} \tag{2}\]  
where \(L_{m}\) is the mean loss, \(N\) is the batch size, \(m_{i}\) is the mean of the \(i\) sample, \(y_{i}\) is the label of the \(i\) sample, \(L_{v}\) is the variance loss, and \(v_{i}\) is the variance of the \(i\) sample. In the test stage, the model weights were fixed, and the final inference of maturity from the tomato image, expressed as the probability distributions for the four maturity stages, was calculated as an estimated value regarding the ground truth.