# Tomato Maturity Estimation Using Deep Neural Network
## 3 Materials and Methods
### Model Training and Evaluation  
The DNN was trained for 300 epochs using the training and validation samples. Input samples were augmented every epoch, in order to minimize overfitting to the training samples, where the augmentation included vertical, oblique, and horizontal flips, as well as stretching. Limited data augmentation related to color features was conducted, because it may have affected the maturity estimation performance. Brightness was only changed for indirectly training the model to have light-invariant performance, and other features (e.g., contrast and saturation) were not augmented. In detail, the V (value) channel was scaled entirely with a random ratio for augmenting the brightness of samples by converting the RGB to HSV. The weights were updated using the Adam optimizer, where the learning rate and weight decay were set at 0.001 and 1 \(\times\) 10\({}^{-5}\), respectively. The model was trained with a batch size of 64 examples, and the training was terminated before the minimum validation loss, in order to enhance the generalization performance [24]. The model training was implemented in Python 3.7 using PyTorch 1.1, and the CPU and GPU used for image training were an i7-8700 K and NVIDIA Titan-V, respectively. The Titan-V, which is a GPU with 5120 CUDA cores and a 1455 MHz boost clock, was used to implement our approach in an optimal manner.  
The classification performance was evaluated through the use of general metrics, including accuracy, precision, recall, and F1 score, as shown in Equations (4)-(7), respectively.  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{4}\]  
\[Precision=\frac{TP}{TP+FP} \tag{5}\]  
\[Recall=\frac{TP}{TP+FN} \tag{6}\]  
\[F1=\frac{2\cdot Recall\cdot Precision}{Recall+Precision} \tag{7}\]  
where \(TP\) (true-positive) denotes the correct classification of a positive label, \(FP\) (false-positive) denotes the incorrect classification of a positive label, and \(FN\) (false-negative) denotes the incorrect classification of a negative label.  
When considering estimated tomato maturity, it is hard to validate the real values as they are difficult to measure, especially from captured scenes. We aimed to obtain continuous maturity in relation to the color distribution of tomato skin, which is difficult for a person to visually determine. Thus, the H (hue) channel of the HSV color model, which has a high correlation with tomato growth [20], was used to obtain a reference distribution of the test images for each class. Estimated tomato maturities were statistically compared with the averaged H values of the input images by class (i.e., maturity stage). The H values were also normalized to obtain a value between 0 and 1. We also tested our method in a Jetson board ([PERSON]; NVIDIA, Santa Clara, CA, USA), in order to evaluate its practical use in real time. The test was conducted using consecutive frames recorded from the greenhouse, and the total recording time was approximately 15 s with 30 fps. Our study aimed to estimate the maturity of detected tomatoes, which in each frame were already annotated with a bounding box. The maturities of the pre-determined tomato locations in each frame were estimated, and the inference time was calculated by frame. The inference time means the time taken only for maturity evaluation, excluding object detection.