# Smart-Viticulture and Deep Learning: Challenges and Recent Developments on Yield Prediction

Lucas Mohimont, Lilian Hollard, Luiz Angelo Steffenel

► To cite this version:

Lucas Mohimont, Lilian Hollard, Luiz Angelo Steffenel. Smart-Viticulture and Deep Learning: Challenges and Recent Developments on Yield Prediction. Kornyshova, E., Deneckère, R., Brinkkemper, S. (eds). Smart Life and Smart Life Engineering, Springer Nature Switzerland, pp.187-207, 2025, 978-3-031-75886-7. 10.1007/978-3-031-75887-4_9. hal-04929528

HAL Id: hal-04929528

https://univ-reims.hal.science/hal-04929528v1

Submitted on 11 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

# Chapter 9
## Smart-Viticulture and Deep Learning: Challenges and Recent Developments on Yield Prediction

Lucas Mohimont, Lilian Hollard, and Luiz Angelo Steffenel

**Abstract** Smart agriculture is a trendy topic as it has a clear impact in both productivity, ecological impact, and improvement of working conditions. Smart viticulture is one of the domains that can benefit both from wireless sensor networks and mobile devices embarked in vineyard labor tools (e.g., on a straddler tractor). One important use case is related to the yield estimation, an invaluable information to drive the harvest organization, plant management, and business's economy. Traditional methods rely on destructive sampling and manual counting, resulting in error rates sometimes greater than 30%. In this chapter, we review existing techniques for the automation of yield estimation and, focusing on deep learning methods, propose some strategies and preliminary results obtained in a production environment.

**Keywords** Computer vision · Deep learning · Viticulture · Yield prediction

### 9.1 Introduction

The development of smart agriculture has raised interesting opportunities for improved automation, logistics, and working conditions. Numerous different technologies have been studied for practical applications in agriculture. For instance, wireless sensor networks allow the collection of data from different locations in a parcel. The acquired data, such as the temperature or the humidity, are used to predict the emergence of diseases (Pérez-Expósito et al. 2017; Steffenel et al. 2021). In addition, the sensors can include embedded cameras to detect the presence of symptoms of illness or deficiencies (Lloret et al. 2011) or simply to control the maturity state of the fruits or estimate the yield. Finally, other sensors such as LIDARs (laser imaging detection and ranging) can estimate the size of the

<footnote>L. Mohimont · L. Hollard · L. A. Steffenel (✉)</footnote>

<footnote>Université de Reims Champagne-Ardenne, LICIIS Laboratory, Reims, France</footnote>

<footnote>e-mail: lucas.mohimont@univ-reims.fr; lilian.hollard@univ-reims.fr;</footnote>

<footnote>luiz-angelo.steffenel@univ-reims.fr</footnote>
canopy or the number of missing plants (Grocholsky et al. 2011). The advance in image processing techniques and the emergence of affordable digital cameras and embarked processors also bring new opportunities for smart agriculture, with the promise to improve the work conditions and the accuracy of forecasts as it can better humans in repetitive, time-consuming, and tedious tasks.

Among these tasks, we can cite the automation of fruit counting, a central problem in smart agriculture as it has a direct impact on yield prediction and management. Several methods have been proposed in recent years, and this concept has been applied to the detection of oranges (Maldonado and Barbosa 2016), bell peppers (Song et al. 2014), and lemons (Dorj et al. 2017).

In the case of viticulture, winegrowers use yield estimation to get decisive information for the business's economy, plant management, and harvest organization. The interest in knowing the yields early enough involves several economic, administrative, and qualitative objectives (Liu et al. 2013). For example, winegrowers can manage the wine market by estimating and controlling the crop volume on a regional or national scale. Yield forecast also helps increase the quality of the wines, by eliminating part of the harvest by thinning out the bunches, as it has been proven that too high density may hinder the quality of the grapes (Xi et al. 2018). This is an essential element in some appellations such as Champagne (INAO), where annual production quotas in kg/ha are defined and any excessive production cannot be transformed in Champagne.

Traditionally, winemakers estimate their production by performing samplings on selected land plots, using a simple formula such as Eq. 9.1 for a given area.

$$ 
\text{Yield (kg/ha)} = \frac{\text{nb plants} \times \text{nb grapes} \times \text{avg grape weight}}{\text{surface}} \quad (9.1) $$

This procedure relies on both grape counting and weighting. Grapes are harvested among a random sample allowing an estimation of the number of grapes by vine, the number of berries per grape, and the weight of the berries. Furthermore, grape counting is mostly performed by a human operator, leading to uncertainties on the precision and dubious repeatability. Weighting of the grapes, on the other hand, requires them to be harvested prematurely, a destructive process that cannot be performed extensively. Some works use historical data to limit grape weighting, but significant variations from year to year can skew the predictions. As a result, a variation of 30% can be found between the estimations and the reality (Dami and Sabbatini 2011). Reducing this error is therefore essential for the organization of the harvest.

The moment yield forecasts are performed also impacts the solutions. Indeed, most winegrowers prefer to obtain estimations at least 1 month before harvest, which usually falls before the *veraison*, i.e., the start of the maturing period where grapes' colors change. This complicates the yield estimation using non-intrusive or non-destructive methods, as the color contrast between the leaves of the vine and the green grapes is reduced (Di Gennaro et al. 2019). In addition, the solutions shall
adapt to the winegrowers' working hours, being tolerant to different light intensities during the day and variable weather conditions (Zhang et al. 2022).

In this work, we focus on the analysis of works where deep learning models are used for grape and berry counting, the first step toward an accurate yield performance forecast. Recent works demonstrate that deep learning contributes to automatically count grapes in the vineyard (Santos et al. 2017; Heinrich et al. 2019) showing more robustness and accuracy than other computing vision methods based on signal processing or traditional machine learning (Diago et al. 2015; Dunn and Martin 2004; Liu et al. 2018).

We also present some results obtained by our team in an effort to develop a yield performance forecast model for the Champagne vineyards. Since 2019, we work on data collection and labeling, as well as the development and training deep learning models for smart-viticulture. Today, our focus is on the development of lightweight and robust models that can later be integrated in edge or embarked devices, allowing on-site real-time processing even in remote locations with no network coverage.

The remainder of the chapter is structured as follows: Sect. 9.2 introduced the literature on grape detection and counting, pointing out traditional and deep learning models as well as the remaining challenges and opportunities. Similarly, Sect. 9.3 focuses on berry counting and weighting. In Sect. 9.4, we introduce our works and some of our preliminary results. Finally, Sect. 9.5 concludes this work.

## 9.2 Grape Detection and Counting

The main component for automatic yield estimation is to detect and count grape clusters (also called *bunches of grapes* or simply *grapes*). In this case, images with grapes are sent to the detection algorithm, whose output includes the number of grapes present in the image and eventually an image containing the location of the grapes as well (see Fig. 9.1).

<figure><img src="image_3.png" /><figcaption>**Fig. 9.1** Example of grape cluster detection (Chardonnay, before veraison) (Mohimont et al. 2022)</figcaption></figure>
This is an important step because it enables several practical applications besides
yield estimation, for example, automatic harvesting using a robot (Reis et al. 2012;
Luo et al. 2016, 2018), the automatic spraying of growth hormones, Berenstein et al.
(2010), the characterization of phenotyping (Rose et al. 2016), or even the detection
of water stress to optimize irrigation (Kang et al. 2023).

Multiple obstacles shall be met during the creation of such an algorithm. First of all, the acquisition method is important: drone/UAV (unmanned aerial vehicle) images are good for general analysis on the canopy (water stress, diseases) but may lack the resolution to identify grape details (Ariza-Sentís et al. 2023). Indeed, precise grape counting and characterization often require ground-based image acquisition solutions, like a camera embedded in a vehicle. Furthermore, several variability factors (lighting, angle, distance, complex background) are present in natural images, and the algorithm must account for (ii) occlusions created by the foliage, and often there is (iii) color confusion between the grapes and the foliage.

A simple approach to detect grapes consists in using thresholding segmentation.
Thresholding segmentation is a simple approach that was first used by Dunn and
Martin (2004) to evaluate the potential of image processing for yield estimation.
Here, one or more thresholds (e.g., color or brightness filters) are applied to
the images to keep the areas that correspond to the fruits only. Thresholding
segmentation algorithms usually have short execution times and are easy to develop.
However, these algorithms have several drawbacks that limit their use in the field
without partial control of the environment. The main issue is that strict thresholds are
not robust to the color variations caused by natural lighting and the background. The
thresholds must be determined in each situation, and the grapes must have attained
the maturity to discriminate their color from the ground or the leaves (Di Gennaro
et al. 2019; Torres-Sánchez et al. 2021). For this reason, most works limit its usage
to nighttime with a lighting source, in order to erase the background elements (sky,
ground, and further rows). Indeed, threshold segmentation has been applied on the
field on red and white varieties but only during the night, with artificial lighting
(Reis et al. 2012).

A different approach relies on traditional machine learning (ML) to make the
detection of grapes more robust to lighting variations. It segments the image by
using small pixel neighborhoods, or blocks, as inputs for a classification model.
Examples of pixel neighborhoods are illustrated in Fig. 9.2.

The pixel classification model produces a binary output (grape or non-grape) that is applied to the central pixel or the entire input block. Classical ML techniques cannot be directly applied to the raw images, as features must be extracted first for each block. A simple feature is the mean values of the R, G, and B channels to produce a vector with only three components (Chamelat et al. 2006). Similarly, the mean value of the RGB (red, green, blue) channels was used in several studies (Casser 2016; Luo et al. 2016; Font et al. 2015). A genetic algorithm was suggested to select the best color channels among several possible color spaces: RGB, HSV (hue, saturation, value), and CIELAB (Behroozi-Khazaei and Maleki 2017). Nonetheless, pixel classification suffers from several limitations, including sensitivity to color (variety) and potentially long execution time, and it heavily
<figure><img src="image_4.png" /><figcaption>**Fig. 9.2** Examples of pixel neighborhoods extracted from a vine image (Pinot Noir, during veraison) (Mohimont et al. 2022)</figcaption></figure>

depends on the choice of extractors because the quality of the features will impact
the final results.

9.2.1 Using Deep Learning

Deep learning has recently been applied to the problem of detecting and count-
ing grapes. A naive approach uses the same pixel-wise classification discussed
previously, but with a neural network that combines both feature extraction and
classification (Lopes et al. 2017; Cecotti et al. 2020; Milella et al. 2019).

The use of convolutional neural networks (CNN) simplifies the detection step
because the model will learn the best features from the data. However, this method
is always limited by the small size of the blocks and the long calculations required
to segment one image. Several CNNs have been studied to predict the mass of
the grapes automatically (Silver et al. 2019). The error is relatively low, 11% in
controlled conditions with an artificial background, but the proposed method has
many drawbacks. The distance between the grapes and the camera varies, impacting
the results. This limit could be overcome using a depth sensor, but this approach
should also be evaluated on several rows and with different varieties to better
appreciate its potential.

Several popular object detection models, Faster R-CNN (Ren et al. 216), R-CNN (Girshick et al. 2014), R-FCN (Dai et al. 2016), and SSD (Liu et al. 2016), have been tested on the task of grape detection and counting using videos (Heinrich et al. 2019). SSD was applied to grapes detection at two stages (0.5 cm berries and 1.2
cm berries) in natural conditions and with real-time hardware acceleration (TPU) (Aguiar et al. 2021). The Mask R-CNN model (He et al. 2018), which detects objects and segments them, has also been applied to the detection of grapes (Santos et al. 2020). The counting of grapes from videos is corrected with a *structure-from-motion* method that estimates the 3D position along the camera path. The 3D position is used as an identifier to avoid counting one grape twice. Mask R-CNN and multiple Yolo models (Redmon et al. 2016) were compared on the dataset published by Deng et al. (2020), Santos et al. (2020). Yolo models were also compared on red grape detection from smartphone images (Li et al. 2021). The Faster R-CNN model was applied similarly with a tracking algorithm to process videos of Riesling and Pinot Noir vines taken at night with artificial lighting (Jaramillo et al. 2021). Mask R-CNN was studied in multiple recent works Barbole and Jadhav (2021), Ghiani et al. (2021), Yin et al. (2021). The authors of Barbole and Jadhav (2021) compared the performances of Mask R-CNN to other models like U-Net and Yolov3 (Redmon et al. 2018) (they also used the WGISD benchmark published by Santos et al. 2020). They found better precision with U-Net and better recall with Yolov3. The authors of Ghiani et al. (2021) applied Mask-RCNN to the GrapeCS-ML dataset published by Seng et al. (2018). It contains images of different varieties taken in natural conditions at different stages. One limitation of this dataset is that most images only contain one grape cluster, so the resulting model cannot process images with many clusters.

Mask R-CNN was also applied with stereo images to detect and reconstruct 3D models of the grapes for automated harvesting (Yin et al. 2021). The Yolov4 (Bochkovskiy et al. 2020) model was applied to low-resolution images of white grapes to measure the correlation between grape counting and fruit weight (Sozzi et al. 2021). A low correlation was found between the number of detected clusters and the actual number of clusters (0.24 R²). The correlation between the number of detected clusters and fruit weight was better (0.59 R²), indicating a potential application for yield estimation in future works. Similarly, YOLOv5s was successfully used by Shen et al. (2023) to perform real-time detection and counting in a defoliated dataset, reaching a 79% accuracy after applying filtering for motion-induced errors.

The authors of Chen et al. proposed a modified PSPNet (Zhao et al. 2016) model for grape segmentation (Chen et al. 2021). It was applied on white and red grapes after veraison. They reached good segmentation performances with an average of 87.42% IoU (Intersection over Union). Similar results were obtained on both red and white grapes. However, the main limitation of semantic segmentation models is their inability to separate overlapped clusters. This results in inaccurate grape counting. Similarly, the authors of Peng et al. (2021) applied a DeepLabV3+ model to multiple varieties of red, green, purple, and black colors, with or without spherical berries. They reached an IoU of 88.44% with an inference of 60 ms/image, which allows an automatic harvesting deployment. The authors are well aware of the difficulty of separating overlapping grapes and proposed the use of a depth sensor (Intel Realsense D435 stereo camera) to solve this problem (Peng et al. 2021).
Their results, 85.6% recall and 87.1% precision, show the viability of their solution. However, this method should be compared to object detection models.

Finally, a generative model was proposed to adapt images to different lighting conditions (Fei et al. 2021). A CycleGAN (Zhu et al. 2020) model was used to translate images taken in daylight to images taken at night with artificial lighting. This step can be useful to increase the size of existing datasets and make the models robust to the varying environmental conditions.

## 9.2.2 Remaining Challenges

Although effective, most of the methods presented in the previous section only have been applied to controlled environments and are seldom adapted to the processing of real-time video streams. For this reason, looking for more performing solutions offering advanced computing and AI capabilities is interesting, and several works now explore the interaction between different layers (cloud-fog-edge) (Vermesan et al. 2021), including Edge-AI capable devices.

Developing real-time Edge AI also helps improve the autonomy of the devices, as one of the main challenges in smart agriculture is the difficulty to deploy solutions capable of handling complex actions in environments subjected to high network constraints (absence of network coverage, high latency) and the limited performance of embarked devices.

Finally, another challenge comes from the complexity of the natural environments in which these devices are supposed to work. For example, Kumrai et al. (2020) demonstrate the difficulty of recognizing human activity according to the angle, capturing and extracting information from the captured images for a camera mounted on a robot. This is somehow similar to the problems faced when detecting and counting grapes on the vineyard, as the angle and other plant elements (leaves, branches) lead to image occlusions that affect the identification of the grapes or the variance of luminosity during the day. Hence, robust deep learning models shall withstand such environmental variance all while delivering accurate results.

## 9.3 Berry Counting and Weighting

If counting the grapes is the first step toward an automatic yield prediction, it is often insufficient to provide accurate results. Indeed, a complementary step is required to reduce the variability caused by different grape bunch sizes in the plot: counting and weighting the berries. Indeed, according to the work from Clingeleffer et al. (2001), the number of grapes per vine, the number of berries per grape, and the weight of the berries make up, respectively, 60%, 30%, and 10% of the yield variance in a model using these three factors.
<figure><img src="image_5.png" /><figcaption>**Fig. 9.3** Example of berry counting (Chardonnay, before veraison) (Mohimont et al. 2022)</figcaption></figure>

As this method is inherently destructive, the sample size is often reduced in traditional estimation methods, leading to important biases. As a consequence, several studies propose methods for automatic counting of berries through computer vision. These methods consist mainly in the detection of visible berries and the further estimation of the number of grape bunches. As stated before, these two factors account for up to 90% of the variance in the yield estimation. An example of automatic berry counting is illustrated in Fig. 9.3.

The problem with berries counting can therefore be formulated as a circle detection or local maxima detection task. Most signal processing solutions use the specular reflection caused by the light on the surface of the berries, producing a pattern that follows a Gaussian distribution. The berries appear as bright little spheres that make them more easily distinguishable from the background of the image. One of the first methods exploiting specular reflection used a Gaussian kernel to process images taken with a smartphone with flash (Grossetete et al. 2012), but it was limited to close-up images of one or several grapes to limit confusion caused by the background.

Several other image analysis algorithms have been used to detect berries: the h-maxima transform (Aquino et al. 2017, 2018), the fast radial symmetry transform (Nuske et al. 2014; Luo et al. 2021), and the Hough transform (Murillo-Bracamontes et al. 2012; Diago et al. 2015; Liu et al. 2015, 2020; Roscher et al. 2014; Rahman et al. 2014; Keresztes et al. 2018). The fast radial symmetry transform detects berry candidates rapidly, whereas the Hough transform potentially has higher memory and computing power requirements. An alternative to the h-maxima transform, named invariant maxima detector, is proposed by (Nuske et al. 2014) to detect berry candidates with artificial lighting. Indeed, these works show that detectors based on local maximums are too sensitive to variations in natural lighting, and the use of a flash or lamp creates a uniform specular reflection on the surface of the berries, making the detection easier.
This classical approach is therefore complex because it requires several algo-
rithms that need to be fine-tuned to (1) detect the berry candidates, (2) extract the
features of each candidate, and (3) filter the false positives with a classifier.

Of course, counting grapes and berries may not be enough to predict the
yield, as the final weight/volume also depends on other factors such as the
average grape/berry size or weight and the weather conditions prior to the harvest
(Anastasiou et al. 2023).

9.3.1 Deep Learning Strategies

Deep learning can simplify berry counting by processing raw images. CNNs can be used as a feature extractor and as a classifier at the same time to filter berry candidates. This technique has been implemented on a Raspberry Pi (Keresztes et al. 2018) for real-time detection. CNNs have also been shown to be more accurate than support vector machines (SVM) (Škrabánek 2018). Furthermore, CNNs have been adapted to object counting with density map prediction. The areas with strong density correspond to the berries’ location (Coviello et al. 2020). This method manages to count berries with an error rate of approximately 10%. CNNs have also been adapted to image segmentation. These CNNs have shown very promising results. A fully convolutional neural (FCN) model was proposed to count the berries with two classes of labeling, the first for the inside of the berries and the second for the edges (Zabawa et al. 2020). The model is faster and better than Mask R-CNN (He et al. 2018) and UNet (Ronneberger et al. 2015). This difference is explained by the structure of the models as Mask R-CNN uses a complex structure to detect and segment objects and by the small size of the database. Deng et al. (2020) used the Hough Transform for circle counting after grape detection with the Yolov4 model (it is similar to the method proposed by Rudolph et al. for flower counting Rudolph et al. 2019). One limitation of this algorithm is its sensibility to berries’ apparent size (the radius must be known).

Miao et al. proposed to use edge detection to solve the problem of counting overlapping berries (Miao et al. 2021). The proposed methodology is complex with a holistically nested edge detection model for berry edge segmentation, a Yolo model for berry detection, and a RANSAC algorithm for sphere fitting. It was evaluated on images of individual grape clusters in different conditions (laboratory or field conditions). A simpler object detection model based on RetinaNet (Lin et al. 2018), modified with a counting section, was evaluated on three plants for counting bananas-per-bunch, spikelets-per-wheats-spike, and berries-per-grape-cluster in natural conditions (Khoroshevsky et al. 2021).

Palacios et al. (2022a,b) proposed a three-step segmentation process, with SegNet models, for grapes, berries, and the canopy features extraction from images taken at night with lighting. The goal is to use segmentation to measure traits with predictive power for actual berry counting (visible + hidden) with regression models.
### 9.3.2 Current Challenges

A problem arises from deploying the berry detection algorithms in the field. They must be applied to image sequences to cover entire rows of vines. Doing this introduces the risk of counting the same grapes twice. Therefore, more or less complex solutions have been applied to avoid redundancy during the counting. A simple one reconstructs the full image of the row using a video and then keeps the highest counts in the overlapping areas (Nuske et al. 2014). It is also possible to follow the detected grapes from one image to the next based on the distance between the grapes in consecutive images (Heinrich et al. 2019). Another mechanical solution consists in taking an image based on the distance covered by a vehicle (one every two meters, for instance) (Aquino et al. 2018).

Another challenge is related to the estimation of the yield volume. The classic modeling approach estimated the total number of berries with a regression model, generally linear. The total estimation number is then converted into mass using the historical average weight of the berries or by extracting samples from the field. This method used by Aquino et al. (2018) with a linear model predicts the total weight based on the visible fruits. An average error rate of 12.83% was achieved on five varieties over 30 segments of three vines.

A recent method uses new techniques to count berries and estimate the total number of berries in a grape (Liu et al. 2020). A 3D reconstruction method that only needs a single 2D image is built by positioning the berries to fill the estimated profile, estimated from the edges, of a single grape bunch. Grape compactness is used as a parameter to simulate different varieties and growth stages. Therefore, this method directly estimates the number of berries in a grape bunch. Although it has only been applied to partially controlled conditions with images of individual grapes with an artificial background, a yield estimation was done by combining this method with another one proposed by the same authors Liu et al. (2017) based on the counting of the shoots. The error rates of predicted yield on three parcels were 3%, 6%, and 16%, which is comparable to or better than other methods.

Finally, the phenological stage also influences the results. For instance, berries in early stages are harder to count due to their reduced size, and their color is similar to the foliage before ripening (Fig. 9.4). Good results were observed in practice by Nuske et al. in 2014 up to 75 days before the harvest (Nuske et al. 2014). The counting can be more difficult once the grapes close (the berries touch each other). The work of Liu et al. shows similar yield estimation performances before and after the ripening (Liu et al. 2020).

Also, we must account for the grapes hidden by the foliage, one of the main problems currently limiting yield prediction performances. Contrarily to several works conducted in laboratory or controlled environments, a winemaker cannot trim his vines. A possible solution is to use a blower to remove the foliage temporarily, but it has not been quantitatively evaluated (Nellithimaru and Kantor 2019). An additional modeling step is hence necessary, and it could benefit from additional variables such as the porosity of the canopy (0.82 R² correlation between the canopy's porosity and the percentage of visible grapes) (Victorino et al. 2019).
<figure><img src="image_6.png" /><figcaption>Stage 27-29 - Young fruits begin to swell, berries small</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Stage 31 - Berries pea-sized</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Stage 33 - Begin of berry touch</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Stage 35 - begin of berry ripen; if it applies, loss of green color (veraison)</figcaption></figure>

**Fig. 9.4** Different stages of a vine

Recent studies have shown that the fruit area might be more robust to occlusion than berry counting (Victorino et al. 2020). This work was extended by comparing yield estimation from images to the classical manual sampling approach on six parcels (Victorino et al. 2022). They used artificial background and manual segmentation of the images to extract meaningful features (such as visible bunch area, canopy porosity, etc.) for yield estimation. In this manner, ideal performances of image analysis methods were evaluated against the current yield estimation process. An average error rate of 8% was found with image analysis compared to 31% for manual sampling. It is also noted that the error increases with canopy density. Although fruit area prediction potential is still highly impacted by foliage, moderate defoliation can help obtain a better correlation (Iñiguez et al. 2021). In practice, the conditions in the vineyard are more difficult: smaller green grapes, high canopy occlusion, natural background, lighting, etc.

Features related to occlusion and fruit area were used as yield predictors in the work of Palacios et al. (2022a,b). An error rate, including six varieties, of 29.77% NRMSE (Normalized root-mean-square error, in kg per vine, 0.83 R²) was reported with images taken 66 days before harvest. Error rates ranged from 16.47% to 39.17% depending on the variety. Zabawa et al. (2020), Zabawa et al. (2022) reported an MAE rate of 26% for 70 vines of Riesling. They only used berry detection and a simple equation (estimated number of berries multiplied by the berry weight).

A possible solution to solve the hidden-berries problem was proposed by Kierdorf et al. (2021). A CycleGAN generative model was used to predict the
location of hidden berries from pairs of vine images before and after defoliation. As a result, better counting performances were obtained, 0.88 $R^2$ compared to 0.72 $R^2$ on the images without defoliation. However, it has not yet been applied to more complex vine images with a natural background. More experiments are needed to determine the practicality of this new method for yield prediction.

## 9.4 Contributions

As illustrated above, several grape detection methods are using deep learning and image processing for grape detection as well as berry counting. These methods, although effective, do not focus on speed of execution or the use of real-time video streams. We are therefore faced with a problem confronting artificial intelligence models with high prediction performance against models capable of combining high prediction and execution time performance in the edge. Indeed, the ability to optimize and compress deep learning models leads toward the convergence of solutions between robust models in the cloud and stand-alone models with deep learning inference directly at the edge.

In order to achieve such objectives, we have conducted several experiences since 2019 for the European projects H2020 AI4DI (Mohimont et al. 2021) and KDT JU EdgeAI, using different cameras and methods, for example, with a GoPro fixed on a picket or embarked on a tractor (Fig. 9.5a). For instance, one of the models we developed was an alleged UNet encoder-decoder with a ResNet-34 backbone with 24M parameters and sufficiently optimized to be deployed in embedded computing boards like a Nvidia Jetson Nano 2GB. As the dataset is limited (approximately 600 photos were taken in the 2019 campaign, from which 386 photos were labeled), there is a risk of overfitting. To avoid this problem, we applied transfer-learning from a ResNet-34 backbone pre-trained on the ImageNet database. Transfer learning allow us to speed up the training process and, as the original model was trained in a larger variety of images, reduces the risk of overfitting. Another optimization we

<figure><img src="image_10.png" /><figcaption>Fig. 9.5 (a) Camera attached to the vehicle, (b) defoliated vine</figcaption></figure>
have made is to avoid processing the whole images but, instead, to proceed them by patches of 512×512 pixels. In this way, we greatly increase the size of the training base. A mini-batch size of 16 is then used, and about 30% of the database is used for model validation. To speed up the training phase, we have used the DGX-1 server from the ROMEO Supercomputing Center, a dedicated deep learning machine with eight Nvidia Tesla V100 GPUs.

After training, this model presents an IoU score of 0.69 and an F1 score of 0.8 (the IoU is limited due to the lack of precision in the labeling). Nonetheless, the model allows detecting nearly 100% of the grapes with a false-positive rate near 0%. In this work, we calibrated our deep learning algorithms for automated yield using a systematic tracking of several rows in the vineyard car (Rossignon 2020). This tracking, performed over four rows (200 vines) at different phenological stages, has included the counting of classic organs (vine, flowers/grapes) using traditional strategies (random counting or by the sampling of the parcels) as well as sampling the berries to estimate their volume and ripeness.

The counting of the grapes has also been done by unveiling hidden grapes by defoliation. Hence, the operator first counts the visible grapes in the plant and, after defoliation, takes a second picture (Fig. 9.5b). Around 30 images were taken in this way and labeled and used to help identify partially hidden grapes. The model can therefore be used to simply count grapes as seen in Fig. 9.6a or segment the image for further analysis (berry counting), like in Fig. 9.6b.

Thanks to the manual and automated counting data, a linear regression model has been generated for each row, followed by a cross-examination of each model using the three other rows (Fig. 9.7). Although the error rate varies from 0% to 31%, depending on the model and the row, we obtain an average error rate of 14%. This is better than current error rates with the traditional approach and can be further improved. Hence, the improvement of this analysis is based on a better distinction between grapes in the foreground and grapes in the background, as well as using non-linear regression models and other variables such as the porosity of the canopy (to account for occlusions).

<figure><img src="image_11.png" /><figcaption>**Fig. 9.6** Examples of image analysis with deep learning. (a) Grape counting. (b) Image segmentation for further berry counting</figcaption></figure>
Nombre de grappes totales selon le nombre de grappes visibles

<figure><img src="image_12.png" /><figcaption>**Fig. 9.7 Correlation between visible grapes and the total number of grapes**</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>**Fig. 9.8 (a) Prototype device installed in a tractor. (b) Biomass chart obtained with Physiocap**</figcaption></figure>

As this is an ongoing project, we regularly iterate with our partner Vranken-Pommery on how to effectively deploy the solution according to their needs. The optimization efforts made to execute the AI algorithm in edge devices allow us to run most of the steps in an embarked node, as illustrated in Fig. 9.8a. Processing most information onsite allows to reduce the amount of data to store and favor the deployment on areas with reduced or unavailable Internet connection.

We are now preparing the prototype devices to be tested in preparation to the Summer 2023 harvest, based on a Jetson Nano device equipped with a camera and a RTK GPS receiver, making it an affordable solution (less than 500 for the complete equipment). Grape count shall be performed from the middle of June 2023, and together with geolocation data, we aim at producing detailed grape density maps similar to the biomass maps from Fig. 9.8b produced by the Physiocap sensor,<fnref n="1" /> allowing a precise harvest planning (workers, material, logistics).

Berry counting is also being introduced in our workflow, as illustrated in Fig. 9.9. Additional work is required to extrapolate the total number of berries in the grape

<footnote n="1">¹ https://www.physiocap.eu/</footnote>
<figure><img src="image_14.png" /><figcaption>**Fig. 9.9** Examples of berry detection</figcaption></figure>

(3D counting) from a 2D image (visible berries) but also to allow real-time counting in an embedded device.

**9.5 Conclusion**

Deep learning has introduced several innovations on the domain of smart agriculture. Contrarily to traditional signal-processing and image analysis methods, deep learning seems to be more robust to natural variations, and pretraining allows easy transfer to similar problems. This is indeed the case of grapevine yield performance forecast, a smart-viticulture use case that can heavily benefit from automation. Generally speaking, counting fruits is a difficult task because the aspect of the plant can vary enormously depending on the point of view and development stages.

In this chapter, we focus on grape yield forecast strategies that use deep learning to count grapes or berries, providing a survey on existing works. We can observe that many works present good performances, but these results shall be put in perspective as most authors rely in controlled scenarios or specific lightning conditions that are not compatible with field deployment at scale. Nonetheless, they point toward some interesting techniques that can be further improved using more robust models and richer datasets.

We also presented some preliminary results from our team, which focus on grape detection and counting under natural conditions. The good performances we obtained show the interest of deep learning and encourage the pursuit of the works, even though additional modeling steps are still needed to account for the non-visible part of the fruits hidden by leaves or the number of berries per cluster and their average weight. Furthermore, a compromise between accuracy and inference speed is necessary for a practical implementation allowing a real-time and autonomous usage. We expect that future improvements in our models and more comprehensive data sources will allow the creation of better yield estimation models.
**Acknowledgments** This work has been started as part of the project AI4DI "Artificial Intelligence for Digitizing Industry" (ECSEL JU grant agreement No 826060) and continues as part of the EdgeAI project "Edge AI Technologies for Optimised Performance Embedded Systems" under grant agreement No 101097300. The EdgeAI project is co-funded by grants from Austria, Belgium, France, Greece, Italy, Latvia, Luxembourg, Netherlands, Norway, and the European Key Digital Technologies Joint Undertaking (KDT JU).

We want to thank Vranken-Pommery Monopole for allowing image collection in their vineyards. We also thank the ROMEO Computing Center<fnref n="2" /> of Université de Reims Champagne-Ardenne, whose Nvidia DGX-1 server allowed us to accelerate the training steps and compare several model approaches.

## References

Aguiar, A.S., Magalhães, S.A., dos Santos, F.N., Castro, L., Pinho, T., Valente, J., Martins, R., Boaventura-Cunha, J.: Grape bunch detection at different growth stages using deep learning quantized models. Agronomy **11**(9) (2021). https://www.mdpi.com/2073-4395/11/9/1890

Anastasiou, E., Templalexis, C., Lentzou, D., Biniari, K., Xanthopoulos, G., Fountas, S.: Do soil and climatic parameters affect yield and quality on table grapes? Smart Agric. Technol. **3**, 100088 (2023). https://doi.org/10.1016/j.atech.2022.100088

Aquino, A., Diago, M.P., Millán, B., Tardaguila, J.: A new methodology for estimating the grapevine-berry number per cluster using image analysis. Biosyst. Eng. **156**, 80-95 (2017). https://doi.org/10.1016/j.biosystemseng.2016.12.011

Aquino, A., Millan, B., Diago, M.P., Tardaguila, J.: Automated early yield prediction in vineyards from on-the-go image acquisition. Comput. Electron. Agric. **144**, 26-36 (2018). https://doi.org/10.1016/j.compag.2017.11.026

Ariza-Sentís, M., Vélez, S., Valente, J.: Dataset on UAV RGB videos acquired over a vineyard including bunch labels for object detection and tracking. Data Brief **46**, 108848 (2023). https://doi.org/10.1016/j.dib.2022.108848

Barbole, M.D., Jadhav, D.P.: Comparative analysis of deep learning architectures for grape cluster instance segmentation. Inform. Technol. Ind. **9**(11), 344-352 (2021). https://doi.org/10.17762/itii.v9i1.138

Behroozi-Khazaei, N., Maleki, M.R.: A robust algorithm based on color features for grape cluster segmentation. Comput. Electron. Agric. **142**, 41-49 (2017). https://doi.org/10.1016/j.compag.2017.08.025

Berenstein, R., Shahar, O.B., Shapiro, A., Edan, Y.: Grape clusters and foliage detection algorithms for autonomous selective vineyard sprayer. Intell. Service Robot. **3**(44), 233-243 (2010). https://doi.org/10.1007/s11370-010-0078-z

Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: Optimal speed and accuracy of object detection (2020)

Casser, V.: Using feedforward neural networks for color based grape detection in field images. In: CSCUBS 2016—Computer Science Conference for University of Bonn Students, pp. 23-33 (2016). https://doi.org/paper/Using-Feedforward-Neural-Networks-for-Color-Based-Casser/139ccba0b3a00565f61febcc62f98c6c44cca990

Cecotti, H., Rivera, A., Farhadloo, M., Pedroza, M.A.: Grape detection with convolutional neural networks. Expert Syst. Appl. **159**, 113588 (2020). https://doi.org/10.1016/j.eswa.2020.113588

<footnote n="2">² https://romeo.univ-reims.fr</footnote>
Chamelat, R., Rosso, E., Choksuriwong, A., Rosenberger, C., Laurent, H., Bro, P.: Grape detection by image processing. In: IECON 2006—32nd Annual Conference on IEEE Industrial Electronics, pp. 3697–3702 (2006). https://doi.org/10.1109/IECON.2006.347704

Chen, S., Song, Y., Su, J., Fang, Y., Shen, L., Mi, Z., Su, B.: Segmentation of field grape bunches via an improved pyramid scene parsing network. Int. J. Agric. Biol. Eng. **14**(6), 185–194 (2021)

Clingeleffer, P.R., Martin, S.R., Dunn, G.M., Krstic, M.P.: Crop development, crop estimation and crop control to secure quality and production of major wine grape varieties: a national approach. Adelaide, Grape and Wine Research and Development Corporation (2001)

Coviello, L., Cristoforetti, M., Jurman, G., Furlanello, C.: Gbcnet: In-field grape berries counting for yield estimation by dilated CNNs. Appl. Sci. **10**(1414), 4870 (2020). https://doi.org/10.3390/app10144870

Dai, J., Li, Y., He, K., Sun, J.: R-FCN: Object detection via region-based fully convolutional networks (2016)

Dami, I., Sabbatini, P.: Crop Estimation of Grapes. The Ohio State University Fact Sheet (2011)

Deng, G., Geng, T., He, C., Wang, X., He, B., Duan, L.: Tsgye: Two-stage grape yield estimation. In: Yang, H., Pasupa, K., Leung, A.C.S., Kwok, J.T., Chan, J.H., King, I. (eds.) Neural Information Processing, pp. 580–588. Springer, Cham (2020)

Di Gennaro, S.F., Toscano, P., Cinat, P., Berton, A., Matese, A.: A low-cost and unsupervised image recognition methodology for yield estimation in a vineyard. Front. Plant Sci. **10**, 559 (2019). https://doi.org/10.3389/fpls.2019.00559

Diago, M.P., Tardaguila, J., Aleixos, N., Millan, B., Prats-Montalban, J.M., Cubero, S., Blasco, J.: Assessment of cluster yield components by image analysis. J. Sci. Food Agric. **95**(66), 1274–1282 (2015). https://doi.org/10.1002/jsfa.6819

Dorj, U.O., Lee, M., seok Yun, S.: An yield estimation in citrus orchards via fruit detection and counting using image processing. Comput. Electron. Agric. **140**, 103–112 (2017). https://doi.org/10.1016/j.compag.2017.05.019

Dunn, G.M., Martin, S.R.: Yield prediction from digital image analysis: a technique with potential for vineyard assessments prior to harvest. Aust. J. Grape Wine Res. **10**(33), 196–198 (2004). https://doi.org/10.1111/j.1755-0238.2004.tb00022.x

Fei, Z., Olenskyj, A., Bailey, B.N., Earles, M.: Enlisting 3D crop models and GANs for more data efficient and generalizable fruit detection (2021)

Font, D., Tresanchez, M., Martínez, D., Moreno, J., Clotet, E., Palacín, J.: Vineyard yield estimation based on the analysis of high resolution images obtained with artificial illumination at night. Sensors **15**(4), 8284–8301 (2015). https://doi.org/10.3390/s150408284

Ghiani, L., Sassu, A., Palumbo, F., Mercenaro, L., Gambella, F.: In-field automatic detection of grape bunches under a totally uncontrolled environment. Sensors **21**(11) (2021). https://doi.org/10.3390/s21113908

Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation (2014)

Grocholsky, B., Nuske, S., Aasted, M., Achar, S., Bates, T.: A camera and laser system for automatic vine balance assessment. In: American Society of Agricultural and Biological Engineers Annual International Meeting 2011, ASABE 2011, vol. 7 (2011). https://doi.org/10.13031/2013.38146. Journal Abbreviation: American Society of Agricultural and Biological Engineers Annual International Meeting 2011, ASABE 2011

Grossetete, M., Berthoumieu, Y., Da Costa, J.P., Germain, C., Lavialle, O., Grenier, G.: Early estimation of vineyard yield: site specific counting of berries by using a smartphone. In: International Conference on Agriculture Engineering (AgEng) (2012)

He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN (2018). arXiv:1703.06870 [cs]

Heinrich, K., Roth, A., Breithaupt, L., Möller, B., Maresch, J.: Yield prognosis for the agrarian management of vineyards using deep learning for object counting. Wirtschaftsinformatik 2019 Proceedings p. 15 (2019)

INAO: AOC champagne—conditions de production
Íñiguez, R., Palacios, F., Barrio, I., Hernández, I., Gutiérrez, S., Tardaguila, J.: Impact of leaf occlusions on yield assessment by computer vision in commercial vineyards. Agronomy **11**(5) (2021). https://doi.org/10.3390/agronomy11051003

Jaramillo, J., Vanden Heuvel, J., Petersen, K.H.: Low-cost, computer vision-based, prebloom cluster count prediction in vineyards. Front. Agron. **3**, 8 (2021). https://doi.org/10.3389/fagro.2021.648080

Kang, C., Diverres, G., Achyut, P., Karkee, M., Zhang, Q., Keller, M.: Estimating soil and grapevine water status using ground based hyperspectral imaging under diffused lighting conditions: addressing the effect of lighting variability in vineyards. Comput. Electron. Agric. **212**, 108175 (2023). https://doi.org/10.1016/j.compag.2023.108175

Keresztes, B., Abdelghafour, F., Randriamanga, D., Da Costa, J.P., Germain, C.: Real-time fruit detection using deep neural networks. In: 14th International Conference on Precision Agriculture (2018)

Khoroshevsky, F., Khoroshevsky, S., Bar-Hillel, A.: Parts-per-object count in agricultural images: solving phenotyping problems via a single deep neural network. Remote Sensing **13**(13) (2021). https://doi.org/10.3390/rs13132496

Kierdorf, J., Weber, I., Kicherer, A., Zabawa, L., Drees, L., Roscher, R.: Behind the leaves—estimation of occluded grapevine berries with conditional generative adversarial networks (2021)

Kumrai, T., Korpela, J., Maekawa, T., Yu, Y., Kanai, R.: Human activity recognition with deep reinforcement learning using the camera of a mobile robot. In: 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom), pp. 1–10 (2020). https://doi.org/10.1109/PerCom45495.2020.9127376

Li, H., Li, C., Li, G., Chen, L.: A real-time table grape detection method based on improved yolov4-tiny network in complex background. Biosyst. Eng. **212**, 347–359 (2021). https://doi.org/10.1016/j.biosystemseng.2021.11.011

Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection (2018)

Liu, S., Cossell, S., Tang, J., Dunn, G., Whitty, M.: A computer vision system for early stage grape yield estimation based on shoot detection. Comput. Electron. Agric. **137**, 88–101 (2017). https://doi.org/10.1016/j.compag.2017.03.013

Liu, S., Li, X., Wu, H., Xin, B., Tang, J., Petrie, P.R., Whitty, M.: A robust automated flower estimation system for grape vines. Biosyst. Eng. **172**, 110–123 (2018). https://doi.org/10.1016/j.biosystemseng.2018.05.009

Liu, S., Marden, S., Whitty, M.: Towards automated yield estimation in viticulture. In: Proceedings of the Australasian Conference on Robotics and Automation, p. 9 (2013)

Liu, S., Whitty, M., Cossell, S.: A lightweight method for grape berry counting based on automated 3 d bunch reconstruction from a single image. In: ICRA, International Conference on Robotics and Automation (IEEE), Workshop on Robotics in Agriculture (2015)

Liu, S., Zeng, X., Whitty, M.: A vision-based robust grape berry counting algorithm for fast calibration-free bunch weight estimation in the field. Comput. Electron. Agric. **173**, 11 (2020). https://doi.org/10.1016/j.compag.2020.105360

Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., Berg, A.C.: Ssd: Single shot multibox detector. Lecture Notes in Computer Science pp. 21–37 (2016). https://doi.org/10.1007/978-3-319-46448-0_2

Lloret, J., Bosch, I., Sendra, S., Serrano, A.: A wireless sensor network for vineyard monitoring that uses image processing. Sensors **11**(6), 6165–6196 (2011). https://doi.org/10.3390/s110606165

Lopes, C., Torres, A., Guzman, R., Graça, J., Monteiro, A., Braga, R., Barriguinha, A., Victorino, G., Reys, M.: Using an unmanned ground vehicle to scout vineyards for non-intrusive estimation of canopy features and grape yield. In: 20th GiESCO International Meeting (2017)

Luo, L., Liu, W., Lu, Q., Wang, J., Wen, W., Yan, D., Tang, Y.: Grape berry detection and size measurement based on edge image processing and geometric morphology. Machines **9**(10), 233 (2021)
Luo, L., Tang, Y., Lu, Q., Chen, X., Zhang, P., Zou, X.: A vision methodology for harvesting robot to detect cutting points on peduncles of double overlapping grape clusters in a vineyard. Comput. Ind. **99**, 130–139 (2018). https://doi.org/10.1016/j.compind.2018.03.017

Luo, L., Tang, Y., Zou, X., Wang, C., Zhang, P., Feng, W.: Robust grape cluster detection in a vineyard by combining the AdaBoost framework and multiple color components. Sensors **16**(1212), 21 (2016). https://doi.org/10.3390/s16122098

Luo, L., Tang, Y., Zou, X., Ye, M., Feng, W., Li, G.: Vision-based extraction of spatial information in grape clusters for harvesting robots. Biosyst. Eng. **151**, 90–104 (2016). https://doi.org/10.1016/j.biosystemseng.2016.08.026

Maldonado, W., Barbosa, J.C.: Automatic green fruit counting in orange trees using digital images. Comput. Electron. Agric. **127**, 572–581 (2016). https://doi.org/10.1016/j.compag.2016.07.023

Miao, Y., Huang, L., Zhang, S.: A two-step phenotypic parameter measurement strategy for overlapped grapes under different light conditions. Sensors **21**(13) (2021). https://doi.org/10.3390/s21134532

Milella, A., Marani, R., Petitti, A., Reina, G.: In-field high throughput grapevine phenotyping with a consumer-grade depth camera. Comput. Electron. Agric. **156**, 293–306 (2019). https://doi.org/10.1016/j.compag.2018.11.026

Mohimont, L., Alin, F., Rondeau, M., Gaveau, N., Steffenel, L.A.: Computer vision and deep learning for precision viticulture. Agronomy **12**(10) (2022). https://doi.org/10.3390/agronomy12102463

Mohimont, L., Roesler, M., Rondeau, M., Gaveau, N., Alin, F., Steffenel, L.A.: Comparison of machine learning and deep learning methods for grape cluster segmentation. In: Boumerdassi, S., Ghogho, M., Renault, É. (eds.) Smart and Sustainable Agriculture, pp. 84–102. Springer, Cham (2021)

Murillo-Bracamontes, E.A., Martinez-Rosas, M.E., Miranda-Velasco, M.M., Martinez-Reyes, H.L., Martinez-Sandoval, J.R., Cervantes-de Avila, H.: Implementation of Hough transform for fruit image segmentation. Proc. Eng. **35**, 230–239 (2012). https://doi.org/10.1016/j.proeng.2012.04.185

Nellithimaru, A.K., Kantor, G.A.: Rols : Robust object-level slam for grape counting. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2648–2656 (2019). https://doi.org/10.1109/CVPRW.2019.00321

Nuske, S., Wilshusen, K., Achar, S., Yoder, L., Singh, S.: Automated visual yield estimation in vineyards. J. Field Robot. **31**(55), 837–860 (2014). https://doi.org/10.1002/rob.21541

Palacios, F., Diago, M.P., Melo-Pinto, P., Tardaguila, J.: Early yield prediction in different grapevine varieties using computer vision and machine learning. Precision Agriculture pp. 1–29 (2022)

Palacios, F., Melo-Pinto, P., Diago, M.P., Tardaguila, J.: Deep learning and computer vision for assessing the number of actual berries in commercial vineyards. Biosyst. Eng. **218**, 175–188 (2022). https://doi.org/10.1016/j.biosystemseng.2022.04.015

Peng, Y., Wang, A., Liu, J., Faheem, M.: A comparative study of semantic segmentation models for identification of grape with different varieties. Agriculture **11**(10) (2021). https://www.mdpi.com/2077-0472/11/10/997

Peng, Y., Zhao, S., Liu, J.: Segmentation of overlapping grape clusters based on the depth region growing method. Electronics **10**(22), 2813 (2021)

Pérez-Expósito, J.P., Fernández-Caramés, T.M., Fraga-Lamas, P., Castedo, L.: Vinesens: An eco-smart decision-support viticulture system. Sensors **17**(33), 26 (2017). https://doi.org/10.3390/s17030465

Rahman, A., Hellicar, A.: Identification of mature grape bunches using image processing and computational intelligence methods. In: 2014 IEEE Symposium on Computational Intelligence for Multimedia, Signal and Vision Processing (CIMSIVP), pp. 1–6 (2014). https://doi.org/10.1109/CIMSIVP.2014.7013272

Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection (2016)

Redmon, J., Farhadi, A.: Yolov3: An incremental improvement (2018)
Reis, M.J.C.S., Morais, R., Peres, E., Pereira, C., Contente, O., Soares, S., Valente, A., Baptista, J., Ferreira, P.J.S.G., Bulas Cruz, J.: Automatic detection of bunches of grapes in natural environment from color images. J. Appl. Logic **10**(44), 285–290 (2012). https://doi.org/10.1016/j.jal.2012.07.004

Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards real-time object detection with region proposal networks (2016)

Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Lecture Notes in Computer Science, pp. 234–241. Springer, Berlin (2015). https://doi.org/10.1007/978-3-319-24574-4_28

Roscher, R., Herzog, K., Kunkel, A., Kicherer, A., Töpfer, R., Förstner, W.: Automated image analysis framework for high-throughput determination of grapevine berry sizes using conditional random fields. Comput. Electron. Agric. **100**, 148–158 (2014). https://doi.org/10.1016/j.compag.2013.11.008

Rose, J.C., Kicherer, A., Wieland, M., Klingbeil, L., Töpfer, R., Kuhlmann, H.: Towards automated large-scale 3d phenotyping of vineyards under field conditions. Sensors **16**(1212), 2136 (2016). https://doi.org/10.3390/s16122136

Rossignon, L.: Vers une méthode optimale d'estimation du rendement de la vigne basée sur l'intelligence artificielle. Master's thesis, AgroParisTech (2020)

Rudolph, R., Herzog, K., Töpfer, R., Steinhage, V.: Efficient identification, localization and quantification of grapevine inflorescences in unprepared field images using fully convolutional networks. J. Grapevine Res. **58**(3) (2019). https://doi.org/10.5073/vitis.2019.58.95-104

Santos, T., Bassoi, L., Oldoni¹, H., Martins, R.: Automatic grape bunch detection in vineyards based on affordable 3D phenotyping using a consumer webcam. XI Congresso Brasileiro de Agroinformática (SBIAgro 2017) (2017)

Santos, T.T., de Souza, L.L., dos Santos, A.A., Avila, S.: Grape detection, segmentation, and tracking using deep neural networks and three-dimensional association. Comput. Electron. Agric. **170**, 105247 (2020). https://doi.org/10.1016/j.compag.2020.105247

Seng, K.P., Ang, L., Schmidtke, L.M., Rogiers, S.Y.: Computer vision and machine learning for viticulture technology. IEEE Access **6**, 67494–67510 (2018). https://doi.org/10.1109/ACCESS.2018.2875862

Shen, L., Su, J., He, R., Song, L., Huang, R., Fang, Y., Song, Y., Su, B.: Real-time tracking and counting of grape clusters in the field based on channel pruning with yolov5s. Comput. Electron. Agric. **206**, 107662 (2023). https://doi.org/10.1016/j.compag.2023.107662

Silver, D.L., Monga, T.: In vino veritas: Estimating vineyard grape yield from images using deep learning. In: Meurs, M.J., Rudzicz, F. (eds.) Advances in Artificial Intelligence, Lecture Notes in Computer Science, pp. 212–224. Springer, Berlin (2019). https://doi.org/10.1007/978-3-030-18305-9_17

Škrabánek, P.: Deepgrapes: Precise detection of grapes in low-resolution images. IFAC-PapersOnLine **51**(66), 185–189 (2018). https://doi.org/10.1016/j.ifacol.2018.07.151

Song, Y., Glasbey, C., Horgan, G., Polder, G., Dieleman, J., van der Heijden, G.: Automatic fruit recognition and counting from multiple images. Biosyst. Eng. **118**, 203–215 (2014). https://doi.org/10.1016/j.biosystemseng.2013.12.008

Sozzi, M., Cantalamessa, s., Cogato, A., Kayad, A., Marinello, F.: Grape yield spatial variability assessment using yolov4 object detection algorithm. pp. 193–198 (2021). https://doi.org/10.3920/978-90-8686-916-9_22

Steffenel, L.A., Langlet, A., Hollard, L., Mohimont, L., Gaveau, N., Copola, M., Pierlot, C., Rondeau, M.: Ai-driven strategies to implement a grapevine downy mildew warning system. In: Industrial Artificial Intelligence Technologies and Applications, pp. 177–187. Rivers Publishers (2022)

Torres-Sánchez, J., Mesas-Carrascosa, F.J., Santesteban, L.G., Jiménez-Brenes, F.M., Oneka, O., Villa-Llop, A., Loidi, M., López-Granados, F.: Grape cluster detection using UAV photogrammetric point clouds as a low-cost tool for yield forecasting in vineyards. Sensors **21**(9) (2021). https://doi.org/10.3390/s21093083
Vermesan, O., Martinsen, J.E., Kristoffersen, A., Bahr, R., Bellmann, R.O., Hjertaker, T., Breiland, J., Andersen, K., Sand, H.E., Rahmanpour, P., Lindberg, D.: Optimisation of soybean manufacturing process using real-time artificial intelligence of things technology. In: Vermesan, O., John, R., Luca, C.D., Coppola, M. (eds.) Artificial Intelligence for Digitising Industry Applications, pp. 301–325. Rivers Publisher (2021)

Victorino, G., Braga, R.P., Santos-Victor, J., Lopes, C.M.: Comparing a new non-invasive vineyard yield estimation approach based on image analysis with manual sample-based methods. Agronomy **12**(6) (2022). https://doi.org/10.3390/agronomy12061464

Victorino, G., Maia, G., Queiroz, J., Braga, R., Marques, J., Lopes, C.: Grapevine yield prediction using image analysis – improving the estimation of non-visible bunches. European Federation for Information Technology in Agriculture, Food and the Environment (EFITA), p. 6 (2019)

Victorino, G.F., Braga, R., Santos-Victor, J., Lopes, C.M.: Yield components detection and image-based indicators for non-invasive grapevine yield prediction at different phenological phases. OENO One **54**(4), 833–848 (2020). https://doi.org/10.20870/oeno-one.2020.54.4.3616

Xi, X., Zha, Q., Jiang, A., Tian, Y.: Stimulatory effect of bunch thinning on sugar accumulation and anthocyanin biosynthesis in Shenhua grape berry (Vitis vinifera × v. labrusca). Aust. J. Grape Wine Res. **24**(2), 158–165 (2018). https://doi.org/10.1111/ajgw.12323

Yin, W., Wen, H., Ning, Z., Ye, J., Dong, Z., Luo, L.: Fruit detection and pose estimation for grape cluster-harvesting robot using binocular imagery based on deep neural networks. Front. Robot. AI **8**, 163 (2021). https://doi.org/10.3389/frobt.2021.626989

Zabawa, L., Kicherer, A., Klingbeil, L., Töpfer, R., Kuhlmann, H., Roscher, R.: Counting of grapevine berries in images via semantic segmentation using convolutional neural networks. ISPRS J. Photogramm. Remote Sensing **164**, 73–83 (2020). https://doi.org/10.1016/j.isprsjprs.2020.04.002

Zabawa, L., Kicherer, A., Klingbeil, L., Töpfer, R., Roscher, R., Kuhlmann, H.: Image-based analysis of yield parameters in viticulture. Biosyst. Eng. **218**, 94–109 (2022). https://doi.org/10.1016/j.biosystemseng.2022.04.009

Zhang, C., Ding, H., Shi, Q., Wang, Y.: Grape cluster real-time detection in complex natural scenes based on yolov5s deep learning network. Agriculture **12**(8) (2022). https://doi.org/10.3390/agriculture12081242

Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network (2016). https://doi.org/10.48550/ARXIV.1612.01105

Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks (2020)

**Open Access** This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

<figure><img src="image_15.png" /></figure>