# GBCNet: In-Field Grape Berries Counting for Yield Estimation by Dilated CNNs
## Materials and Methods  
### From Crowd to Berries Counting  
GBCNet stems from the family of Dilated CNNs [44] and integrates geometry-adaptive kernels [45] to solve the problem of grape cluttering in the images. We demonstrate the potentiality of GBCNet on two original datasets CR1, with 7 different varieties, and CR2, with only one variety; good performances are achieved in terms of Mean Absolute and Squared Error (MAE/MSE), with variability induced by the different grapevine varieties. Overall, Mean Average Error (MAE) varies from 0.85% for Pinot Gris to 11.73% for Marzemino on CR1 and reaches 7.24% on the Teroldego CR2 dataset, supporting the claim that GBCNet achieves a good compromise between minimal sensor cost, in-field efficiency and yield estimate reliability. The core of GBCNet for yield estimation is the ability of accurate automatic counting of berries from pictures taken in the grape fields (Figure 1). We will show that, for these tasks, techniques developed in the context of automatic crowd counting can successfully be adapted [39; 46]: in the case of congested scene recognition presented in [44], the input picture is processed by the Deep Neural Network CSRNet returning a density map, in which the integral is the estimated amount of subjects to count, in our case the number of berries in the image, as shown in Figure 2.  
The CSRNet architecture employs the first ten convolutional layers of VGG16 [47] pretrained on ImageNet [48] as feature extractor and a dilated CNN [49; 50] for density map generation. Training from scratch the full network requires an enormous amount of annotated data, and annotation is an expensive operation, in particular with grape images where labeling is required at the level of single  
Figure 1: GBCNet architecture: the model takes in-field smartphone images as the input and estimates a density map in which the integral represents berries count. The second block uses a dilation factor of 2. Every convolutional layer is followed by a ReLU operation, except for the last one. Figure 2: Example of application of CSRNet on a CR2 image (**a**), its associated ground truth (**b**) and model output (**c**). berry. To reduce the number of annotations required for the training we adopt for GBCNet a transfer learning approach where a pre-trained VGG16 model is used as a generalized feature extractor for the training of the last part of the network. The use of dilated convolutions, i.e., convolutions with non contiguous kernels with a larger receptive field, aggregates multi-scale contextual information while maintaining the same spatial resolution. The training phase is based on the generation of density maps as ground truth. This requires the annotation of the images at single berry level: given an input image, a berry at the position \(x_{i}\) is represented as a Dirac delta function \(\delta(x-x_{i})\), which represents a binary mask with only the point \(x_{i}\) set to 1. After the annotation the image is represented as:  
\[H(x)=\sum_{i=1}^{N}\delta(x-x_{i})\, \tag{3}\]  
where \(N\) is the number of labeled points. To obtain a continuous density function \(F(x)\) from the discrete representation \(H(x)\)[45], GBCNet employs a convolution with a Gaussian kernel \(G_{\sigma}\) using \(F(x)=H(x)*G_{\sigma}(x)\) as introduced in [51], where the \(\sigma\) fixes the level of smoothing in the mask. Additionally, to tackle the presence of dense scenes in the images, GBCNet is endowed by geometry-adaptive kernels [45] evaluating the distribution of the neighbors of a labeled point. Geometry-adaptive kernels are available in the Python module Scikit-Image [52,53], and they are defined as follows:  
\[F(x)=\sum_{i=1}^{N}\delta(x-x_{i})*G_{\sigma_{i}}(x),\text{ with }\sigma_{i}= \beta\overline{d_{i}}\, \tag{4}\]  
where \(\overline{d_{i}}\) is the average distance of the \(k\) nearest neighviors of \(x_{i}\) and \(\beta\) is a regularization parameter. In all the experiments we use the same configuration as by [PERSON] and colleagues in [44], setting \(k=3\) and \(\beta=0.3\).