## 2 Materials and Methods
### Segmentation of the Apples in the Video Frames  
The first step in the development of a complete computer vision system is image segmentation, that is, the separation of the object of interest from the background of the image. Different color spaces have been used for segmentation in agriculture; for example, the YCrCb space is used in [24] to segment rice seedlings with high accuracy; in this space, Y is the luma, and Cr and Cb are the blue and red-difference chroma components, respectively. In the present study, segmentation was done frame by frame using the L*u*v* color space. The process is based on a comprehensive segmentation algorithm presented in [25]. This segmentation algorithm is a combination of four main methods based on segmentation in the L*u*v* color space, texture segmentation using local standard deviation texture descriptor, luminosity segmentation using intensity transformation, and color segmentation in the RGB color space. Figure 3 shows an example of the different steps of apple segmentation in a typical video frame image. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Case Number** & **Light Intensity (lux)** & **Train and Test Frames** \\ \hline
1 & 289 & 653 \\
2 & 429 & 459 \\
3 & 598 & 936 \\
4 & 736 & 1236 \\
5 & 889 & 721 \\
6 & 991 & 398 \\
7 & 1206 & 893 \\
8 & 1329 & 567 \\
9 & 1638 & 692 \\
10 & 1837 & 1103 \\
11 & 1994 & 732 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Description of the images of the dataset. Each case corresponds to a video, from which different frames were extracted. The videos are ordered from lower to higher luminous intensity. Figure 2: Sample images of the four different apple ripening stages (_Malus Domestica_ L., var. Red Delicious) extracted from the captured frames under natural lighting conditions. (**a**) Unripe. (**b**) Half-ripe. (**c**) Ripe. (**d**) Overripe. The first step of the process is depicted in Figure 3b, where the input frame is converted to the L*u*v* color space, and a threshold is then applied to remove bluish colors. The threshold value was obtained through a preliminary trial and error method and is set to 90 (for u* and v* in the range of a byte). The second step (Figure 3c) is based on the local standard deviation texture descriptor. The original image is first converted to grayscale; then, the local standard deviation is computed for each pixel, that is, the standard deviation of the gray values for each neighborhood of 5 \(\times\) 5 pixels around the given pixel. A threshold is applied to remove pixels with large texture values, meaning that pixels with a hard texture are supposed to belong to the background. The third step of segmentation is shown in Figure 3d. It is based on intensity transformation using the R channel of RGB. It can be observed that most background pixels appear with darker colors, so they can be removed with this simple thresholding. In particular, a threshold level of 105 was applied to the R channel, meaning that all pixels with an R value lower than 105 are considered background. The final segmentation result is presented in Figure 3e. This final step is based on 25 color tests in the RGB color space, as proposed in [25]. These 25 tests are applied sequentially, one after another, and  
Figure 3: Example of the steps in apple segmentation. (**a**) Original video frame. (**b**) First step of segmentation based on color thresholding in the L*u*v* color space. (**c**) Second step, based on texture features. (**d**) Third step, based on intensity transformation. (**e**) Final step, based on thresholding in RGB color space. combined with an OR Boolean operator. That is, the pixel is considered as background if one of the tests is true. The tests have the following form:  
\[test_{1}(r,g,b)=(185\leq r\leq 195)\text{ and }(170\leq g\leq 185)\text{ and }(170\leq b\leq 185)\text{ and }(r-b<20) \tag{1}\]  
where \((r,g,b)\) are the RGB values of the pixel in bytes.