# Technologies for Forecasting Tree Fruit Load and Harvest Timing--From Ground, Sky and Time
## 3 Measurement of Fruit Size
### Machine Vision Methods  
In an early attempt to use RGB-based machine vision in the estimation of grape bunch weight, bunch pixel area and volume was estimated following object segmentation based on color within images acquired from side views of vine rows ([PERSON], 2019). Estimate errors were relatively large, at 16 and \(-17\%\) for area and volume-based estimates, respectively. However, no correction was made for camera to fruit distance in this study. Camera to fruit distance can be estimated given an object of known size is present in the fruit plane. A sticker on each fruit can provide an object of known size. While labeling  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Publication**} & \multirow{2}{*}{**Commodity**} & \multicolumn{2}{c}{**Allometry between Weight and**} & \multirow{2}{*}{**Prediction of Harvest Weight**} \\  & & & **Lineal Dimensions** & \\ \hline \multirow{3}{*}{[PERSON] et al. (2019)} & \multirow{3}{*}{Apple (Gala)} & \multirow{3}{*}{W = 17.03 \(-\) (1.48 \(\times\) d) + (0.046 \(\times\) d\({}^{2}\))} & \multirow{3}{*}{HW = \(-\)169.09 + (8.19 \(\times\) W\({}_{60\,\mathrm{DAB}}\))} \\  & & & R\({}^{2}\) = 0.972 & R\({}^{2}\) = 0.79 \\ \cline{1-1} \cline{3-4} \cline{5-5} \cline  
is labor intensive, it can be carried out for a number of sample fruits through the orchard, potentially identified by flowering event. In a parallel approach, [PERSON] et al. (2017) described use of mobile phone imaging of sampled mango fruit on-tree against a backing board with a scale. The app was reported to achieve lineal measurements of fruit size with an RMSE of 5 mm, for fruit of lineal dimensions between 50 and 130 mm. Several camera technologies exist for the estimation of camera to object distance, including stereo vision, structured light, and Time-of-Flight (ToF) cameras. Stereo vision, which relies on the matching of images from paired cameras, provides the least accurate assessment of distance of these methods. The structured light method is based on the distortion of a pattern of light projected onto the field of view. ToF technology is based on an estimation of the time taken for photons to traverse from an emitter in the camera to the object and back to a detector in the camera. Structured light and ToF cameras typically project a near infrared wavelength around 760 nm, and consequently, there can be operational issues in sunlight, which also contains radiation in this region. LiDAR employs a ToF technology to provide a point cloud presentation of a given scene. If the point cloud were dense enough, fruit could be recognized and lineal dimensions measured; however, this would require a long scanning time. Point cloud density is sparse in LiDAR images acquired using moving vehicles, but the LiDAR point cloud can be merged with RGB imagery, with the LiDAR distance obtained for objects detected as fruit in RGB images. [PERSON] et al. (2017) documented the use of a ToF camera (PMD CamCube 3.0) for the estimation of apple fruit sizes (n = 150 fruits on 25 random trees). The method tended to under-predict fruit size (bias = \(-\)9.6 mm, on actual diameter of 74.0 mm), with an average MAE of 15.2%. The largest source of error was poor segmentation of fruit in images, particularly for partially occluded fruits, leading to an estimate of dimensions not related to fruit. [PERSON] et al. (2017) utilized a Microsoft Kinect2 ToF camera for the estimation of mango fruit size from a moving farm vehicle-mounted platform, reporting an RMSE of 4.9 and 4.3 mm (RRMSE of 4.9 and 5.3%) for length and width, respectively, against a manual measurement with calipers. An ellipse was fitted to the detected objects (fruit), and objects were discarded if ellipse parameters did not match those expected for a fruit, thus rejecting partly occluded fruit. An allometric relation was established relating fruit weight to fruit length width (Table 5).