# Technologies for Forecasting Tree Fruit Load and Harvest Timing--From Ground, Sky and Time
## 2 Measurement of Fruit Number
### Machine Vision Methods
#### 2.4.3 Implementation on Ground Vehicles
An early attempt is demonstrated in the work of [PERSON] et al. [53] who used a back propagation neural network model to associate the actual tree fruit load to canopy characters derived from segmentation techniques. For the prediction of fruit load in a different season to that used for model training, the RMSE and R\({}^{2}\) between estimated and harvested apple yield were 2.6 kg/tree and 0.62 for early fruit growth (small, green fruit) and 2.5 kg/tree and 0.75 near harvest (red, large fruit), for trees with an average 18 kg of fruit. In a later attempt in the direct prediction of total tree fruit load, deep learning techniques were employed [12], and good predictions of current season tree fruit loads were achieved, but predictions of fruit load per tree for a subsequent season were poor. Further work should be undertaken to progress this concept. Another issue in orchard imaging is lighting. Daytime orchard imaging can be compromised by sun glare in image and strong shadowing of fruit (Figure 6). [PERSON] et al. [37] and [PERSON] et al. [38] addressed this issue by use of an intense strobe light system and a very short camera exposure time, such that the effect of sunlight was greatly diminished. [PERSON] et al. [54] placed cameras under an over-the-row cover to eliminate direct sunlight. Other researchers [55, 56, 57, 3, 40, 41] avoided the daylight issue by imaging at night, using artificial lighting. Figure 5: Manual tree fruit counts plotted against multi view machine vision estimates of mango tree fruit load for an orchard of smaller, open canopy trees (**top panel**) and larger, dense canopy trees (**bottom panel**). Units for RMSE and bias are counts of fruit per tree. Reports of the use of ground vehicle-based machine vision for tree or orchard fruit load estimation are summarized in Table 4 in terms of commodity, algorithm, imaging method, validation set used and result achieved. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Paper** & **Algorithm/Technique** & **Imaging Method** & **Validation Set** & **Result** \\ \hline \hline \multicolumn{5}{c}{Apple} \\ \hline [PERSON] et al. [54] & Color and shape thresholding & DV on ground vehicle with row cover & 20 trees & 20\% error on fruit in image, 18\% error on fruits per tree \\ [PERSON] and & CNN & MV on ground vehicle, day imaging & 15 rows & 10.8\% error on fruit counts per tree \\  & Color thresholding, & & & 10.7 and 8.9\% MAPE \\  & neural network & & Multi season cross & for estimations of fruit validations (2009-2011) & per tree after fruit drop \\ [PERSON] et al. [53] & utilizing ancillary variables (foliage area, fruit area) & & 180 trees total. & and during ripening \\ [PERSON] [57] & Speeded Up Robust Features & DV (2014) and MV (2016 set) ground vehicle, night imaging & 10 graphs of 20 random trees (2014 and 2016 & 2014: 4.8\% error and 2016: 5.4\% error (fruit/tree) \\ [PERSON] et al. [59] & YOLOv3 & Single view on stationary platform & 552 images & 93\% precision; 1\% error on fruit count per image \\ \hline \multicolumn{5}{c}{Citrus} \\ \hline [PERSON] et al. [60] & FRCNN & Fusion of thermal and RGB, day imaging & 1658 images & 95.5\% precision on fruit/image \\  & & & & 11.5, 4.3 and 5.8\% error against harvested kg for 2016, 2017, and 2018, respectively \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Example reported results of machine vision-based estimation of tree fruit load. Papers are ordered chronologically within commodities. Single view refers to collection of one image per tree and one row side only. Dual view (DV) refers to collection of one image per tree side from each interrow and multi view (MV) entails collection of several images in succession per tree side. Figure 6: Image of the same mango tree canopy at day and at night. Bounding boxes relate to tracking of fruit between frames. Imaging of every row of an orchard allows for the production of a yield map of the orchard, which can have farm management uses such as guiding early harvest or understanding the impact of an environmental variable on yield.