# Analysing the Characteristics of Neural Networks for the Recognition of Sugar Beets

Luca Brodo, Stefan Henkler, Kristian Rother

► To cite this version:

Luca Brodo, Stefan Henkler, Kristian Rother. Analysing the Characteristics of Neural Networks for the Recognition of Sugar Beets. 7th International Embedded Systems Symposium (IESS), Nov 2022, Lippstadt, Germany. pp.115-126, 10.1007/978-3-031-34214-1_10. hal-04645289

HAL Id: hal-04645289

https://inria.hal.science/hal-04645289v1

Submitted on 11 Jul 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Analysing the Characteristics of Neural Networks for the Recognition of Sugar Beets*

Luca Brodo¹, Stefan Henkler¹, and Kristian Rother¹

Hamm-Lippstadt University of Applied Sciences, 59557 Lippstadt, Germany
luca.brodo001@stud.fh-dortmund.de {stefan.henkler, kristian.rother}@hshl.de

**Abstract.** Neural Networks (NNs) are used in various application areas to identify objects. Reliable behavior of NNs is an important aspect, especially for embedded systems. In this paper, we focus on the analysis of NNs to find correlations between their characteristics in order to be reliable and predictable in time, with the aim of making sugar beet recognition more transparent. Although obtaining promising results, as we are only going to focus on analysing models and finding correlations between some of their characteristics, this paper is only the first milestone towards using this correlation to optimize smart farming applications to improve the production and sustainability of the plantations around the world.

**Keywords:** Neural Networks · Embedded Systems · Smart Farming

## 1 Introduction

In the last years there has been an increased focus on integrating cutting-edge technology with precision farming to improve quality and quantity of agricultural production, while at the same time lowering the inputs - like manual labour - significantly [14]. This system is also known as “smart farming” and it is based on the adoption of autonomous robots, which could be both wheeled robots and Unmanned Aerial Vehicles (UAVs), and it has been enabled by the astonishing advancements in the field of the Internet of Things (IoT).

Smart Farming allows more cost-efficient and timely production and management practices, as shown by Glaroudis et al. in [9], and at the same time, the reduction of the inherent climate impact by enabling real-time reactions to infestations, such as weed, pest or diseases, and by enabling a more adequate use of resources such as water, pesticides or agro-chemicals [14].

One of the cultivations which will benefit immensely by the adoption of smart farming is the sugar beet. The *Beta vulgaris* L, commonly referred to as sugar

<footnote>* This research is supported by a grant from the Ministry of Economic Affairs, Industry, Climate Action and Energy of the State of North Rhine-Westphalia (MWIDE) as part of the 5G-Landwirtschaft-ML project in the context of the program 5G.NRW (01.05.2022 – 31.12.2024, grant number 005-2108-0039).</footnote>
beet, is ranked as the second most cultivated sugar crop all over the world next to sugarcane [2]. As showed by May in [21], being a slow-growing crop early in the season, this plant seems to be a very poor competitor against weed, a claim also backed up by Schweizer in [29], which reports that sugar beet root yields can be reduced by 26–100% by uncontrolled weed growth.

As herbicides are still the most used method to control weed infestations ([4]), in order to limit their use and therefore the impact they have on the environment, the new frontier for weed management is to automatize the more traditional mechanical techniques ([25], [6], [19]). Mechanical techniques, and even techniques which use selective spraying of herbicides, however, require the sugar beet to be localized before they can be applied to avoid ruining the plantation.

Although examples of different techniques employed for the recognition tasks are not uncommon ([16], [33]), these type of tasks are usually carried on by Convolutional Neural Networks (CNNs)([7], [30], [26], [22], [23]).

CNNs, however, impose great challenges to the developers due to their unpredictability. For example, in real time applications, classification time is of the highest importance since it affects the ability to respect deadlines. Furthermore, the performance of CNNs is influenced by a multitude of factors and, being able to predict their performances before their deployment, will allow engineers to precisely model the application to tailor them for the network of their choice.

In challenges such as the ImageNet classification challenge ([28]) the ultimate goal is to achieve the highest accuracy possible, neglecting other performance metrics like inference time [3].

Although accuracy is of high importance, in practical applications other metrics are to be considered as well, depending on the different requirements. As pointed out by Canziani *et al*. in [3], metrics like inference time, parameters and operations count are hard constraints for the deployment of Neural Networks in practical applications. Furthermore, training time is often a time consuming process which highly depends on factors like the complexity of the task, size of the network and training set ([24]).

Finding relations between these characteristics and other factors as well, like influence of a given input feature to the prediction of the model ([11]), will allow us to optimise applications, saving time and resources in the process. In this paper, we are going to investigate whether it is possible to make those predictions by empirically studying the metrics which can influence CNNs' performances and find correlations amongst them in order to be able to be predictable, especially time-wise. In other words, the approach presented attempt to compare and evaluate the performance of CNNs under different conditions and workloads. In comparison, we look at the characteristics of CNNs in order to develop techniques to measure performance and from this to determine - i.e. predict - the behavior of CNNs.

As a result of our investigation, we aim to develop a toolbox for the employment of CNNs in sugar beet recognition. In the next section we give an overview of the related work. In section 3, we introduce the benchmarking tool utilized. The analysis of the NNs characteristics including the benchmarking results are
presented in Section 4. We conclude our paper with a summary and outlook in
Section 5.

## 2 Related Work

Regarding the study of neural networks’ performances, both academic and in-
dustrial organizations have already developed numerous benchmark solutions to
evaluate the behaviour of neural networks under different workloads and on dif-
ferent devices. For example, authors of [18] and [13] have proposed benchmarks
of neural networks specifically for mobile devices. Hendrycks et al. in [10] devel-
oped a benchmark to assess the robustness of image classifiers under condition
of perturbations.

Reddi et al. in [27] propose a benchmark, namely MLPerf, to evaluate inference of Machine Learning system under various workloads. They divided workloads under high level tasks, such as image classification, and they provide a reference model for it.

Zou et al. in [34], among other contributions, proposed a benchmark suite to analyse neural networks’ training time of eight different state-of-the-art models under six different application domains. The metrics collected during the profiling are throughput, i.e. the amount of input samples processed by the networks, GPU utilization, FP32 utilization, i.e. the percentage of floating operations done during training, since typically training CPU utilization and Memory Consumption.

A similar goal described in the previous section is also the one shared by Unterthiner et al. in [31], which demonstrates that the prediction of accuracy of neural networks using the weights is viable. More generally, in recent years, other authors have shown possible predict the performance of neural networks by parameters ([5], [20]). In this paper, however, rather than on the parameters, we will rely upon methods to find those characteristics empirically, i.e. through benchmarking.

## 3 Analysing the Models

As mentioned previously, we are going to predict neural network characteristics based on an empirical analysis carried on through benchmarking. Benchmarking is a widely used method to evaluate the performance of a system, either software or hardware, whose main goal is to produce consistent and precise measurements of said systems ([1]). High precision and reliability in benchmarks could bring knowledge about the timing behaviour of tasks, more specifically their worst-case execution times (WCETs), which is of the highest importance for building reliable and dependable real-time systems ([32])

According to von Kistowski et al. in [15], benchmarks should follow five key characteristics in order to be reliable, namely relevance, reproducibility, fairness, verifiability and usability. Relevance measures how close the behavior of the benchmark relates to the behaviors it is trying to test. Reproducibility is the
ability to consistently obtain similar, if not equivalent, results if the benchmark is
tested on the same environment ([15]). Furthermore, a benchmark has to be fair,
namely it has to allow the results obtained in different systems with completely
different hardware and software to be comparable. A good benchmark should be
also able to provide trustworthy results and also results whose trustworthiness
can be verified. To ensure the verifiability of these results, good benchmarks
usually run self-verification tasks during run-time, for e.g. if hyper-threading is
still active when it should not. Finally, usability is the characteristics of a bench-
mark which aims to remove roadblocks for users to run the benchmark in their
environments.

Some of these characteristics are tightly correlated to one another, for e.g. repro-
ducibility is highly related to the ability of describing the environment in perfect
detail, so that it could be performed on the same environment and expect sim-
ilar results, which is also an important factor for usability. In general, however,
it is important to have a benchmark tool which respects those characteristics.
Such tool will allow us to study the behaviour of neural networks in order to
find correlations amongst the characteristics of neural networks, therefore the
main requirement it needs to adhere to is flexibility. Applications in the field
of sugar beet recognition may be subject to various requirements and different
conditions, hence the tool must be able to reflect that. In other words, it should
be flexible enough so that we are able to configure it in multiple ways to reflect
the various condition, but at the same time it must also be able to produce re-
sults that are specific for our scope, i.e. finding the correlation between different
metrics. Furthermore, we need the results to be verifiable and trustworthy. Be-
ing the foundation for further steps, the results need to be trustworthy and they
need to be precise and accurate enough, so that they can be reasoned about.
For the porpuse of this paper, the choice of the architecture is limited amongst
Resnet, Alexnet and VGG. Both the indication regarding the dataset and the
choice of the models will be indicated in an external configuration file by the
user. The tool will read this file before initializing the test, in order to collect
the necessary information needed to run properly. This configuration file helps
to improve both flexibility and usability, in addition to automatically creating a
documentation of the run for reproducibility.

The actual implementation of the tool is done using Python and revolves around
the *fastai* library. This library is built on top of Pytorch, a very common library
for machine learning, and offers powerful functions and high flexibility without
a significant drop in performance. [12]

*Fastai* perfectly suits the goals of the benchmarking tool as it is "approachable
and rapidly productive, while also being deeply hackable and configurable". [12]

# 4 Analysis of the characteristics

Thanks to the knowledge and the tool described in the previous section in our
hands, we can now move on to discuss a real-world use case as a proof of concept.
For the purpose of this paper, the characteristics we are mainly focusing on are inference time, accuracy, training time and loss. The experiment has been run on the same machine running Ubuntu 20.04.3 LTS (Focal Fossa). For the specific of the machine, please refer to table 1.

<table><caption>Table 1: Specifics of the machine which run the experiments</caption><tr><td>CPU</td><td>2x AMD EPYC 7452 32-Core Processor</td></tr><tr><td>CPU MHz</td><td>1499.324</td></tr><tr><td>CPU max MHz</td><td>2350,0000</td></tr><tr><td>CPU min MHz</td><td>1500,0000</td></tr><tr><td>Total memory</td><td>1 TB (16x64 GB)</td></tr><tr><td>GPU</td><td>Nvidia A100-PCIE-40GB</td></tr><tr><td>Number of GPUs</td><td>7</td></tr></table>

The models have been trained on the GPUs, using the default configuration offered by *Fastai*. Furthermore, the models have not been pre-trained, hence no transferred learning is applied, and the models have been trained using full precision. Although a relatively powerful machine, and rather different from the mobile-robots used in the solutions we discussed, this experiment gives us important insights to how such analysis can be carried on.

We are going to use the dataset proposed by *Giselsson et al.* in [8], which we are going to refer to as the ‘plant_seedlings_v2’ dataset. This dataset contains ~1000 RGB images with a pixel density of 10 pixels per mm divided in 12 different plant species. Even though the dataset is mainly focused on seedlings and it contains pictures of other plants as well, this will give us proper insights of the models’ behaviours in a farming setting.

The first metrics we are going to analyse are training time, number of epochs and accuracy. We will study those metrics in order to recognize some patterns and use those to be able to find correlations between the three, with the final aim being able to predict one of them, whilst knowing the others. We are going to start our analysis by studying the results obtained when the models are trained for 100 epochs, which are shown in Fig. 1a and 1b.

Alexnet is the model that achieved the lowest accuracy, overall reaching ~86% after 99 epochs. The highest accuracy has been achieved by Resnet101, peaking at ~97% after 66 epochs. Moreover, VGG16 and VGG19 achieved the same peak accuracy at ~95%, however VGG16 required 13 epochs less (43 compared to the 56 needed for VGG19). Similarly, Resnet18 and Resnet34 achieved comparable top accuracies at 94.39% and 94.48% respectively, however Resnet34 required considerably less time to reach this number peaking after 62 epochs, while Resnet18 achieved this number when the training cycle was almost done, namely after 96 epochs.

Fig. 1a shows the response of all models during the training based on the epoch used for training. The response of the model shows that, within the first ten epochs, the accuracy increases quickly and tends to stabilise afterwards, in-
<figure><img src="image_3.png" /><figcaption>(a) Accuracy achieved by the models after (b) Accuracy achieved when training for being trained for 100 epochs. The x axis is 100 epochs in relation with training time. the number of epochs, while the y axis is The x axis is the training time in seconds, the accuracy achieved while the y axis is the accuracy achieved</figcaption></figure>

<caption>Fig. 1: Training results of all models calculated over 100 epochs. The accuracy is calculated over the training set.</caption>

creasing slowly over time. However, we can observe a compact graph, meaning all the models (besides Alexnet) achieved accuracies not too far off from each other. As a matter of fact, the difference in accuracy between the highest performing model (Resnet101) and the lowest performing model (Resnet18) was only 3%.

Fig. 1b, on the other hand, depicts the accuracy graphed over training time. Alexnet took the least amount of time to finish the training cycle (747 seconds), however Resnet18 took only ~40 seconds more and reached a far better accuracy(94% compared to 85%). Resnet152 took 47 Minutes and 30 Seconds (2851 seconds) to complete the test and is the model that required the most time.

<figure><img src="image_4.png" /><figcaption>Fig. 2: Training loss and validation loss of all models calculated over 100 epochs</figcaption></figure>
In Fig. 2, we can analyse the loss calculated for the models during the training. Fig. 2a refers to the loss calculated over the training set, while Fig. 2b depicts the loss calculated over the validation set, which comprises of pictures not included in the training set. As shown in Fig. 2a, all the models (besides Alexnet) reached closed to zero training loss within the first 30 epochs. Usually, when the training loss is close to zero, we are expecting a case of over-fitting, which would be proven by a high validation loss. We can then study the validation loss of the models in Fig. 2b. From their response, we can observe that the validation loss does not increase drastically. On the contrary, the difference between validation loss and training loss is rarely more than 0.4. Furthermore, the curve seems to be quite stable, especially for shallower networks, and the validation loss tends to increase slowly.

As we are mostly focused on sugar beet recognition, it is safe to assume use cases where field robots would scan the field to recognize the vegetation, similarly to the one proposed by Lottes et al. in [17]. In such setting, the time taken to classify the image, ultimately, results in a soft deadline, as the time taken to scan the field is greatly influenced by it, therefore being able to estimate the needed inference time could help optimise this part of the application.
In order to test for inference time, we are going to use a dataset composed of ~120 pictures taken from the original dataset. These pictures were taken before starting the process, therefore they have not been used for training. The results are shown in Fig. 3.

Fig. 3a shows the inference time based on the accuracy, while figure 3b shows the inference time based on the epoch used to train.

From Fig. 3a we can start to observe some rather interesting properties. For each model, as they achieve accuracy less than 88%, the inference time is rarely measured to be more than 60 ms, with only 2 exceptions (~100ms and ~390ms). Furthermore, the only model that achieved an accuracy smaller than ~82% is Alexnet.

<figure><img src="image_5.png" /><figcaption>Fig. 3: Inference time measured for each model using the dataset discussed previously. The inference time is in milliseconds</figcaption></figure>
For accuracies over 88%, although the behaviour of the models seems to be less compact, the inference time rarely measures more than 100 milliseconds, with only two outliers (~135ms and ~180s).

From Fig. 3b we can see that, with the exception of the outliers we identified before, the number of epochs that measured the highest value for inference is 30. As a matter of fact, when trained for different numbers of epochs, the models never require more than 100 milliseconds to process the images.

<figure><img src="image_6.png" /><figcaption>Fig. 4: Inference time measured for Resnet101 and Resnet152. The inference time is in milliseconds.</figcaption></figure>

A closer investigation of the individual performance of each model shows further interesting aspects. For the purpose of this experiment, we are going to analyse only Resnet101 and Resnet152, whose behaviour is shown in Fig. 4a and Fig. 4b respectively. For both models, we can see that the highest accuracy is achieved when trained for only one (~91% for both) and two epochs (slightly lower of 91% for Resnet18 and ~90% for Resnet152). For the other epoch, the accuracy achieved was consistently lower than 90%. In the case of Resnet101 it resulted in an overall faster behavior when trained for ten and five epochs, with an inference time of less than 45 milliseconds, with the exception of only two occasions. Although the fastest, the model trained with these epochs also achieved the lowest accuracy. The slowest inference time has been achieved when the model has been trained for two epochs, with an inference as high as 135 milliseconds. However, the model achieved the worst overall performance when trained for 30 epochs. As a matter of fact, even though it measured the lowest inference time when trained for two epochs, this occurred in only one case, while, when trained for 30 epochs, for a considerable amount of pictures, the inference time was in a range between 57 and 95 milliseconds. Resnet152, on the other hand, showed more consistent results, with an inference time being in the range of 40 to 90 milliseconds for all epochs with the exception of one case where it reached 180 milliseconds. The fastest inference time for this model is measured when the model has been trained for 20 epochs.

From this investigation we can conclude that the performances of these two mod-
els are quite comparable, when it comes to inference. Both in terms of inference
time and accuracy achieved, the two models show similar results, albeit for dif-
ferent levels of training. For example, in the case of Resnet152, with a training
time of 580 seconds (20 epochs), we obtain a model that can make predictions
in a range between 45 and 85 milliseconds with an accuracy of ~89%. In the case
of Resnet101, we can obtain similar results (prediction time between 70 and 30
milliseconds with an accuracy of ~89%) with a training time of 210 seconds (10
epochs).

We can also run the tool to discover which pictures took the most amount of
time to be processed. Fig. 5 shows the ten slowest images for each model graphed
based on their sizes.

<figure><img src="image_7.png" /><figcaption>Fig. 5: This graph shows the size in kb (y axis) of the ten slowest images over the time taken to be processed (x axis)</figcaption></figure>

The graph shows a rather compact and stable behaviour, with the slowest
time measured for images having a size bigger than 1500 kb.

For pictures of more than 100 kb and less than 300 kb, the inference time is
between 0 and 100 ms, with each model performing rather similarly.

For this experiment, both for training and for testing inference, we used a dataset
with images having rather similar size and resolution, therefore this analysis is
not as valuable as expected.

5 Summary and Outlook

Sugar beet plantations, due to their poor performances against other competi-
tors like weed, will benefit from the advancements brought by the introduction
of smart farming techniques. Most of the solutions in this field proposed method-
ologies for increasing production and sustainability based on the recognition and
localization of sugar beets using Neural Networks. Neural Networks, however,
impose obstacles which, despite their great flexibility, make tailoring applica-
tions around them challenging.
In this paper, we investigated how the analysis of the characteristics of Neural Networks brings to find correlations between them that can be used to predict the performances of the networks in a time-wise reliable way. The output of our investigation was a software tool which we used to study a use case to demonstrate how we can effectively find these correlations and these characteristics, mentioning how they could be used in future applications.

Even though we achieved promising and insightful results, this paper leaves many interrogatives open which need to be explored in future works. First of all, we only investigated some of the characteristics which neural networks possess and can be correlated with each other. For example, we did not focus on parameters which influence the performances of the networks, like for e.g. learning rate, batch size or throughput. It is left for further studies to identify other characteristics that can be worth studying to find more correlations.

## References

1. Beyer, D., Löwe, S., Wendler, P.: Reliable benchmarking: requirements and solutions. International Journal on Software Tools for Technology Transfer **21**, 1–29 (2017)

2. Bhadra, T., Paul, S.: Weed management in sugar beet: A review. Fundamental and Applied Agriculture **5**, 1 (2020)

3. Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. CoRR abs/1605.07678 (2016)

4. Cioni, F., Maines, G.: Weed Control in Sugarbeet. Sugar Tech **12**(3-4), 243–255 (Dec 2010)

5. DeChant, C., Han, S., Lipson, H.: Predicting the accuracy of neural networks from final and intermediate layer outputs. In: ICML 2019 Workshop on Identifying and Understanding Deep Learning Phenomena (2019)

6. Frasconi, C., Martelloni, L., Fontanelli, M., Raffaelli, M., Emmi, L., Pirchio, M., Peruzzi, A.: Design and full realization of physical weed control (PWC) automated machine within the RHEA project. Second International Conference on Robotics and associated High-technologies and Equipment for Agriculture and forestry (RHEA-2014)- New trends in mobile robotics, perception and actuation for agriculture and forestry, May 21-23, 2014, Madrid, Spain **New trends in mobile robotics, perception and actuation for agriculture and forestry**, 3–11 (2014)

7. Gao, J., French, A.P., Pound, M.P., He, Y., Pridmore, T.P., Pieters, J.G.: Deep convolutional neural networks for image-based Convolvulus sepium detection in sugar beet fields. Plant Methods **16**(1), 29 (Dec 2020)

8. Giselsson, T., Jørgensen, R., Jensen, P., Dyrmann, M., Midtiby, H.: A public image database for benchmark of plant seedling classification algorithms. arXiv: Computer Vision and Pattern Recognition (11 2017)

9. Glaroudis, D., Iossifides, A., Chatzimisios, P.: Survey, comparison and research challenges of IoT application protocols for smart farming. Computer Networks **168**, 107037 (Feb 2020)

10. Hendrycks, D., Dietterich, T.: Benchmarking neural network robustness to common corruptions and perturbations. In: International Conference on Learning Representations (2019)
11. Hooker, S., Erhan, D., Kindermans, P.J., Kim, B.: A benchmark for interpretability methods in deep neural networks (2019)

12. Howard, J., Gugger, S.: Fastai: A layered api for deep learning. Information **11**(2) (2020)

13. Ignatov, A., Timofte, R., Kulik, A., Yang, S., Wang, K., Baum, F., Wu, M., Xu, L., Gool, L.V.: Ai benchmark: All about deep learning on smartphones in 2019. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (IC-CVW). pp. 3617–3635 (2019)

14. Islam, N., Rashid, M.M., Pasandideh, F., Ray, B., Moore, S., Kadel, R.: A Review of Applications and Communication Technologies for Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) Based Sustainable Smart Farming. Sustainability **13**, 1821 (Feb 2021)

15. v. Kistowski, J., Arnold, J.A., Huppler, K., Lange, K.D., Henning, J.L., Cao, P.: How to build a benchmark. In: Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering. p. 333–336. ICPE ’15, Association for Computing Machinery, New York, NY, USA (2015)

16. Lottes, P., Hoeferlin, M., Sander, S., Muter, M., Schulze, P., Stachniss, L.C.: An effective classification system for separating sugar beets and weeds for precision farming applications. In: 2016 IEEE International Conference on Robotics and Automation (ICRA). pp. 5157–5163. IEEE, Stockholm, Sweden (May 2016)

17. Lottes, P., Hoeferlin, M., Sander, S., Müter, M., Schulze, P., Stachniss, L.C.: An effective classification system for separating sugar beets and weeds for precision farming applications. In: 2016 IEEE International Conference on Robotics and Automation (ICRA). pp. 5157–5163 (2016)

18. Luo, C., He, X., Zhan, J., Wang, L., Gao, W., Dai, J.: Comparison and benchmarking of ai models and frameworks on mobile devices (2020)

19. Machleb, J., Peteinatos, G.G., Sökefeld, M., Gerhards, R.: Sensor-Based Intrarow Mechanical Weed Control in Sugar Beets with Motorized Finger Weeders. *Agronomy* **11**(8), 1517 (Jul 2021)

20. Martin, C.H., Mahoney, M.W.: Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning. CoRR abs/1810.01075 (2018)

21. May, M.J.: Economic consequences for UK farmers of growing GM herbicide tolerant sugar beet. Annals of Applied Biology **142**, 41–48 (Feb 2003)

22. Milioto, A., Lottes, P., Stachniss, C.: Real-time blob-wise sugar beets vs weeds classification for monitoring fields using convolutional neural networks. ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences **4**, 41–48 (2017)

23. Nasirahmadi, A., Wilczek, U., Hensel, O.: Sugar beet damage detection during harvesting using different convolutional neural network models. Agriculture **11**(11) (2021)

24. Perugini, Engeler: Neural network learning time: effects of network and training set size. In: International 1989 Joint Conference on Neural Networks. vol. 2, pp. 395–401 (1989)

25. Raja, R., Nguyen, T.T., Slaughter, D.C., Fennimore, S.A.: Real-time robotic weed knife control system for tomato and lettuce based on geometric appearance of plant labels. Biosystems Engineering **194**, 152–164 (Jun 2020)

26. Ramirez, W., Achanccaray, P., Mendoza, L.F., Pacheco, M.A.C.: DEEP CONVOLUTIONAL NEURAL NETWORKS FOR WEED DETECTION IN AGRICULTURAL CROPS USING OPTICAL AERIAL IMAGES. The International
Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences **XLII-3/W12-2020**, 551–555 (Nov 2020)

27. Reddi, V.J., Cheng, C., Kanter, D., Mattson, P., Schmuelling, G., Wu, C.J., Anderson, B., Breughe, M., Charlebois, M., Chou, W., Chukka, R., Coleman, C., Davis, S., Deng, P., Diamos, G., Duke, J., Fick, D., Gardner, J.S., Hubara, I., Idgunji, S., Jablin, T.B., Jiao, J., John, T.S., Kanwar, P., Lee, D., Liao, J., Lokhmotov, A., Massa, F., Meng, P., Micikevicius, P., Osborne, C., Pekhimenko, G., Rajan, A.T.R., Sequeira, D., Sirasao, A., Sun, F., Tang, H., Thomson, M., Wei, F., Wu, E., Xu, L., Yamada, K., Yu, B., Yuan, G., Zhong, A., Zhang, P., Zhou, Y.: Mlperf inference benchmark. In: 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA). pp. 446–459 (2020)

28. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) **115**(3), 211–252 (2015)

29. Schweizer, E., Dexter, A.: "Weed control in sugarbeets (Beta vulgaris) in North America". Reviews of Weed Science **3**, 113–133 (1987)

30. Suh, H.K., IJsselmuiden, J., Hofstee, J.W., van Henten, E.J.: Transfer learning for the classification of sugar beet and volunteer potato under field conditions. *Biosystems Engineering* **174**, 50–65 (Oct 2018)

31. Unterthiner, T., Keysers, D., Gelly, S., Bousquet, O., Tolstikhin, I.O.: Predicting neural network accuracy from weights. ArXiv abs/2002.11448 (2020)

32. Wägemann, P., Distler, T., Eichler, C., Schröder-Preikschat, W.: Benchmark generation for timing analysis. In: 2017 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS). pp. 319–330 (2017)

33. Yang, R., Tian, H., Kan, J.: Classification of sugar beets based on hyperspectral and extreme learning machine methods. Applied Engineering in Agriculture **34**, 891–897 (01 2018)

34. Zhu, H., Akrout, M., Zheng, B., Pelegris, A., Jayarajan, A., Phanishayee, A., Schroeder, B., Pekhimenko, G.: Benchmarking and analyzing deep neural network training. In: 2018 IEEE International Symposium on Workload Characterization (IISWC). pp. 88–100 (2018)