# Deep Regression Neural Networks for Proportion Judgment
## 4 Summary and Discussion  
The application of the proposed models on the olive flowering phenophases dataset shows that CNNs typically perform better than human experts, especially considering that, in the case of estimating thousands of images, this can be a mentally very demanding process. The results should be also analyzed in the context of the reliability of determining the ground truth [47]. We used manual and automated methods to illustrate the importance of ground truth data design and use. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset (Samples)** & **Model** & _MAE_ & _RMSE_ & \(\mathbf{R^{2}}\) \\ \hline \multirow{8}{*}{TOYds (10,000)} & Xception + SVR & 0.41 & 0.58 & 0.998 \\  & Xception + & 0.45 & 0.69 & 0.998 \\  & RandomForestRegressor & 0.49 & 0.69 & 0.998 \\  & Xception * 3 & 0.49 & 0.69 & 0.998 \\  & InceptionResNetV2 + SVR & 0.56 & 0.75 & 0.997 \\  & InceptionResNetV2 + & 0.44 & 0.65 & 0.998 \\  & RandomForestRegressor & 0.44 & 0.65 & 0.998 \\  & InceptionResNetV2 * 3 & **0.39** & **0.55** & **0.998** \\ \hline \multirow{8}{*}{TOY*ds (25,000)} & Xception + SVR & 0.33 & 0.45 & 0.999 \\  & Xception + & 0.24 & 0.43 & 0.999 \\  & RandomForestRegressor & & & \\  & Xception * 3 & **0.21** & **0.33** & **0.999** \\  & InceptionResNetV2 + SVR & 0.32 & 0.47 & 0.999 \\  & InceptionResNetV2 + & 0.27 & 0.46 & 0.999 \\  & RandomForestRegressor & & & \\  & InceptionResNetV2 * 3 & 0.29 & 0.47 & 0.999 \\ \hline \multirow{8}{*}{OFPds (1314)} & Xception + SVR & 5.99 & 8.88 & 0.881 \\  & Xception + & 5.61 & 8.75 & 0.883 \\  & RandomForestRegressor & 5.42 & 8.68 & 0.888 \\  & InceptionResNetV2 + & 5.87 & 8.75 & 0.881 \\  & InceptionResNetV2 + & 5.60 & 8.69 & 0.882 \\  & RandomForestRegressor & & & \\  & InceptionResNetV2 * 3 & **5.35** & **8.41** & **0.902** \\ \hline \multirow{8}{*}{OFPds augmented (8509)} & Xception + SVR & 3.52 & 5.21 & 0.959 \\  & Xception + & & & \\  & RandomForestRegressor & 2.88 & 4.35 & 0.971 \\  & Xception * 3 & 3.06 & 4.97 & 0.962 \\  & InceptionResNetV2 + SVR & 3.55 & 5.34 & 0.944 \\  & InceptionResNetV2 + & 2.87 & 4.28 & 0.975 \\  & InceptionResNetV2 * 3 & **2.75** & **3.87** & **0.979** \\ \hline \multirow{8}{*}{AILds (18,000)} & Xception + SVR & 1.93 & 3.71 & 0.947 \\  & Xception + & & & \\  & RandomForestRegressor & 1.78 & 3.30 & 0.958 \\ \cline{1-1}  & Xception * 3 & **1.62** & **3.12** & **0.982** \\ \cline{1-1}  & InceptionResNetV2 + SVR & 1.96 & 3.85 & 0.940 \\ \cline{1-1}  & InceptionResNetV2 + & 1.79 & 3.45 & 0.954 \\ \cline{1-1}  & InceptionResNetV2 * 3 & 1.73 & 3.32 & 0.961 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The prediction performance of the hybrid/ensemble models on the test datasets. For the toy dataset, to guarantee the accuracy of the ground truth data, the exact share of triangles (ground truth) was computed during the image generation process. This fact, combined with 25,000 examples in the dataset, results in _MAE_ values between 0.2 and 0.3 for the best models. However, we mentioned that the number of epochs (for all datasets) was limited during the experiment to a value of 100, in order to make the results comparable. With this in mind, after the experiment was finished, we tested some of the best-performing models further. We found that, if the maximum number of epochs is increased to 500 and early stopping patience to 30, an additional reduction of _MAE_ to 0.14 can be achieved. As already stated, for the olive flowering phenophases dataset, the ground truth is provided by human expert annotators. Generally speaking, flowers are defined as \"open\" when the reproductive parts are visible between or within unfolded or open flower parts. The application of this definition is not simple in practice, as can be seen in Figure 9.  
To give an illustration of how this can be problematic in practice, we could say that there are a total of 20 buds and flowers in an image, but experts cannot agree on the classification of one particular flower.