# An annotated image dataset of vegetable crops at an early stage of growth for proximal sensing applications

Louis Lac, Barna Keresztes, Marine Louargant, Marc Donias, Jean-Pierre da Costa

<figure><img src="image_1.png" /></figure>

## To cite this version:

Louis Lac, Barna Keresztes, Marine Louargant, Marc Donias, Jean-Pierre da Costa. An annotated image dataset of vegetable crops at an early stage of growth for proximal sensing applications. Data in Brief, 2022, 42, pp.108035. 10.1016/j.dib.2022.108035. hal-03728222

HAL Id: hal-03728222

https://hal.science/hal-03728222v1

Submitted on 20 Jul 2022

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_3.png" /></figure>

Data Article

# An annotated image dataset of vegetable crops at an early stage of growth for proximal sensing applications

<figure><img src="image_4.png" /></figure>

Louis Lac<sup>a,*</sup>, Barna Keresztes<sup>a</sup>, Marine Louargant<sup>b</sup>, Marc Donias<sup>a,c</sup>, Jean-Pierre Da Costa<sup>a,d</sup>

ARTICLE INFO

Article history:
Received 16 February 2022
Accepted 4 March 2022
Available online 9 March 2022

Dataset link: Vegetable Crops Dataset for Proximal Sensing (VCD) (Original data)

Keywords:
Vegetable crops
Crop structures
Precision hoeing
Phenotyping
Leaf counting
Precision agriculture
Crop detection
Deep learning

ABSTRACT

This article introduces a dataset of 2 801 images of vegetable crops. Maize (Zea mays), bean (Phaseolus vulgaris) and leek (Allium ampeloprasum) crops at an early stage of development (between 2 and 5 weeks from seeding of transplanting) are supported. Two kinds of annotations are provided: (i) bounding boxes enclosing the crops of interest or their stems, weeds being left apart, and (ii) crop structures in the form of star graphs whose vertices are the plant organs (stems and leaves) and whose edges represent the connections between them. The images have been captured in various production and experimentation plots in France using an acquisition module which controls light conditions. They present a wide variety of soil conditions, weed infestation and growth stages. This dataset can benefit precision hoeing and in-field crop monitoring applications that are based on proximal imaging.

DOI of original article: 10.1016/j.compag.2021.106606

* Corresponding author.
E-mail address: louis.lac@ims-bordeaux.fr (L. Lac).

Article history:
Received 16 February 2022
Accepted 4 March 2022
Available online 9 March 2022

<footnote><sup>a</sup> IMS UMR 5218, CNRS, Talence F-33405, France</footnote>

<footnote><sup>b</sup> CTIFL, 28 Route des Nebouts, Prigonrieux, France</footnote>

<footnote><sup>c</sup> ENSEIRB-Matmeca, Bordeaux INP, Talence F-33402, France</footnote>

<footnote><sup>d</sup> Bordeaux Sciences Agro, Gradignan F-33175, France</footnote>
<table><caption>**Specifications Table**</caption><tr><td>Subject</td><td>Agricultural Sciences: Agriculture Engineering</td></tr><tr><td>Specific subject area</td><td>Vegetable crops detection and phenotyping by proximal imagery</td></tr><tr><td>Type of data</td><td>RGB images<br/>Bounding box annotations<br/>Crop structure annotations</td></tr><tr><td>How the data were acquired</td><td>Various RGB camera such as the industrial-grade Basler acA2500-gc equipped with a C125-0418-5M F1.8 f 4 mm Basler lens. Light conditions are artificially controlled.<br/>Bounding boxes are annotated with https://github.com/tzutalin/labellmg (XML files).<br/>Crop structures are annotated with https://github.com/laclouis5/StructureAnnotator (JSON files).</td></tr><tr><td>Data format</td><td>Raw, Analyzed, Filtered, Annotated</td></tr><tr><td>Description of data collection</td><td>The images were collected in vegetable fields (maize (Zea mays), bean (Phaseolus vulgaris) and leek (Allium ampeloprasum)) at an early stage of development (2 to 5 weeks from seeding). The camera is orthogonal to the ground plane (i.e. facing the ground) and at an elevation between 35 cm and 45 cm. Light conditions are artificially controlled.</td></tr><tr><td>Data source location</td><td>INRAE, Montoldre, France (46°20'20.9"N 3°25'54.7"E)<br/>CTIFL, Prigonrieux, France (44°50'45.9"N 0°24'52.7"E)<br/>Fermes Larrre, Liposthey, France (44°25'40.8"N 0°51'46.1"W)<br/>Bordeaux Sciences Agro, Bordeaux, France (44°47'30.8"N 0°36'26.6"W)</td></tr><tr><td>Data accessibility</td><td>Repository Name: Mendeley Data<br/>Data identification number: 10.17632/d7kbzjr83k.1<br/>Direct URL to data: https://data.mendeley.com/datasets/d7kbzjr83k/1</td></tr><tr><td>Related research article</td><td>L. Lac, J.-P. Da Costa, M. Donias, B. Keresztes, A. Bardet, Crop stem detection and tracking for precision hoeing using deep learning, Computers and Electronics in Agriculture. 192 (2022). https://doi.org/10.1016/j.compag.2021.106606.</td></tr></table>

**Value of the Data**

* This dataset is a collection of images of vegetable crops at an early stage of development. The images are annotated with crop bounding boxes and crop structures, i.e. the precise location of the organs (stem and leaves) and their relations. This data is highly valuable to develop automated solutions for precision hoeing and in-field crop monitoring applications.

* These data can benefit research institutes working on the automatic detection of crops as well as commercial applications exploiting Deep Learning models for real-time precision hoeing and crop monitoring.

* These data can be used to reproduce our results [2,3]. They can also benefit any research in computer vision and artificial intelligence that aims at developing new crop detection and location algorithms, the performance presented in the related research articles being the bottom line for experimental comparison.

**1. Data Description**

The dataset is composed of 2801 JPEG images. Each image is uniquely identified by its file name encoded in the format `<date>_<location>_<label>_<serie>_<id>.jpg` where:
<figure><img src="image_5.png" /><figcaption>**Fig. 1.** Illustration of eight images drawn at random from the dataset. They show different soil conditions, crop growth stages and weed infestation levels. Some parts of the acquisition module can be visible at the periphery of the images (the black plastic curtain, weeding tools in green or gray). The crops of interest are usually laid out on a horizontal line at half the image height.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Fig. 2.** Illustration of three images of the dataset with bounding boxes annotations overlaid. Whole crop annotations are in red, green and orange, and stem annotations are in pink, blue and yellow.</figcaption></figure>

* `<date>` is the date of image capture in ISO 8601 standard format YYYY-MM-DD,

* `<location>` is the location of the image capture,

* `<label>` is the crop specie,

* `<serie>` is an optional number identifying the culture row, and

* `<id>` is the index of the culture in that row.

A collection of images that are not dated, located or that do not have a label is also provided. For such images, the text `<date>_<location>` or `<label>` is replaced by the text *misc*. Finally, a collection of empty images (i.e. without visible crops) is provided. Such images are identified with the file name *no-obj_<id>*.jpg. Fig. 1 exhibits some images drawn at random in the dataset. They present different soil conditions, stages of growth and weed infestations.

## 2. Bounding Boxes Annotations

2610 images are annotated with rectangular bounding boxes. Each crop is annotated with two bounding boxes: one for the whole crop and another one centered on the crop stem entry point in the soil. Fig. 2 illustrates the bounding box annotations on three images of the dataset. Whole crop annotations are depicted in red, green and orange, and stem annotations are depicted in pink, blue and yellow, respectively for maize, bean and leek crops.
<caption>**Table 1**</caption>

Number of bounding boxes annotations in the dataset by crop type. "Crops" refers to the number of whole crop bounding box annotations while "Stems" refers to the number of stem bounding box annotations. The label "no-obj" refers to the images without any crop.

<table><thead><tr><td>Crop Type</td><td>Images</td><td>Crops</td><td>Stems</td></tr></thead><tbody><tr><td>Maize</td><td>1 065</td><td>2 264</td><td>2 274</td></tr><tr><td>Bean</td><td>779</td><td>2 913</td><td>2 918</td></tr><tr><td>Leek</td><td>601</td><td>3 070</td><td>3 192</td></tr><tr><td>no-obj</td><td>135</td><td>0</td><td>0</td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>**Fig. 3.** Illustration of two images from the dataset with crop structure annotations overlaid. The crop bounding box is depicted with a blue rectangle and a blue dot at the bounding box center connected to the corresponding crop stem. Its stem keypoint is depicted with a green dot and its leaves are depicted by red dots connected to the stem with a red line. The crop type is overlaid near the stem.</figcaption></figure>

The bounding boxes annotations are provided in the standard PASCAL VOC format [1] (XML file). Each bounding box has a label depicting the crop specie (maize, bean or leek) and if the bounding box represents a stem of not, e.g. the label of a maize crop is `maize` and the label of a bean stem is `stem_bean`.

Each annotation file has the same name as the corresponding image excepted the `.xml` file extension. Table 1 summarizes the number of whole crop and stem bounding boxes annotations of the dataset by crop type.

## 3. Crop Structures Annotations

1 135 images are annotated with crop structures. A crop structure is composed of the whole plant bounding box and a star graph where the plant organs (stem and leaves) are the vertices and the connections between them are the edges. Fig. 3 illustrates the crop structure annotations on two images of the dataset. For each crop, the bounding box is depicted with a blue rectangle, the blue dot being the bounding box center. The stem keypoint is depicted by a green dot and the leaf keypoints are depicted by red dots connected to the stem by a red line.

There is no standard serialization format for star graph annotations so crop structures are provided in a custom JSON format. Each annotation has the same name as the corresponding image excepted the `.json` file extension. Listing 1 presents an example of a JSON annotation file for one image containing one bean crop with two leaves:

* The top level field `image_name` identifies the image corresponding to this annotation.

* The other top level field `objects` lists the crop annotations of the image. Each crop annotation have:
<caption>**Table 2**</caption>

Number of crop structures annotations in the dataset by crop types. "Images" refers to the numbers of images. "Crops/stems" refers to the number of whole crop bounding box annotations, which is equal to the number of stem keypoint annotations. "Leaves" refers to the number of leaf tip keypoint annotations. The label "no-obj" refers to the images without any crop.

<table><thead><tr><td>Crop Type</td><td>Images</td><td>Crops/stems</td><td>Leaves</td></tr></thead><tbody><tr><td>Maize</td><td>505</td><td>1 298</td><td>3 250</td></tr><tr><td>Bean</td><td>347</td><td>1 815</td><td>3 855</td></tr><tr><td>no-obj</td><td>135</td><td>0</td><td>0</td></tr></tbody></table>

- a label field depicting the crop specie ("maize" or "bean"),

- a box field storing the bounding box coordinates of the whole crop, and

- a parts field storing a list of keypoint annotations. Each keypoint annotation has a kind field (either "stem" or "leaf") depicting the keypoint type and a location field indicating the keypoint location in the image.

For each crop, there is exactly one stem keypoint and one bounding box as well as arbitrary number of leaf keypoints, zero included. The bounding boxes and keypoints coordinates are expressed in pixels in the usual image coordinates system whose origin is the top-left corner. Table 2 summarizes the number of crop structures annotations in the dataset and details the number of annotated stem and leaf keypoints.

```json
{
  "image_name": "2019-05-23_montoldre_bean_1_000000.jpg",
  "objects": [
    {
      "label": "bean",
      "box": {"x_min": 9, "y_min": 526, "x_max": 185, "y_max": 762},
      "parts": [
        {
          "kind": "stem",
          "location": {"x": 91, "y": 707}
        },
        {
          "kind": "leaf",
          "location": {"x": 62, "y": 532}
        },
        {
          "kind": "leaf",
          "location": {"x": 155, "y": 748}
        }
      ]
    }
  ]
}
```

**Listing 1.** Example of a JSON annotation file for one image containing one bean crop with two leaves.

Additionally, the dataset contains 4 text files listing the images used for the training and validation of the deep neural networks presented in our published articles. The two files `sdnet-train.txt` and `sdnet-valid.txt` list the images used for the training and then validation of SDNet [3]. The two files `yolo-train.txt` and `yolo-valid.txt` list the images used for the training and the validation of the neural networks presented in [2].
<figure><img src="image_8.png" /><figcaption>**Fig. 4.** Illustration of the acquisition module. The camera and the cultures are protected from direct sunlight and outdoor conditions by a hull and two led panels provide a controlled illumination of the scene. Modified from Lac et al. [3].</figcaption></figure>

# 4. Experimental Design, Materials and Methods

The images were acquired at four different locations in France: INRAE Montoldre, CTIFL Prigonrieux, Fermes Larrmere at Liposthey and Bordeaux Sciences Agro in Bordeaux. Three species of vegetable crops are covered by this dataset: maize (*Zea mays*), bean (*Phaseolus vulgaris*) and leek (*Allium ampeloprasum*). The crops are at an early stage of development (2 to 5 weeks from the seeding). The crop rows have not been hoed or treated with phytosanitary products. Natural weeds (mostly purslane (*Portulaca oleracea*), black nighshade (*Solanum nigrum*) and couch grass (*Elymus repens*)) or manually seeded ones (mustard (*Sinapis alba*), raygrass (*Lolium spp.*), matricaria (*Matricaria chamomilla*) and lamb's quarter (*Chenopodium album*)) may be present.

The acquisition of images is performed with the acquisition module depicted in Fig. 4. The camera (Basler acA2500-gc equipped with a C125-0418-5M F1.8 f 4 mm Basler lens for the current acquisition module) is facing the soil at an elevation *h* between 35 cm and 45 cm from the ground. The light conditions are artificially controlled: a hull isolates the camera from the exterior and two 20 W led panels provide a constant and homogeneous scene illumination.

# Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

# Data Availability

Vegetable Crops Dataset for Proximal Sensing (VCD) (Original data) (Mendeley Data)

# CRediT Author Statement

**Louis Lac:** Methodology, Software, Data curation, Writing – original draft; **Barna Keresztes:** Methodology, Investigation, Software; **Marine Louargant:** Data curation, Investigation; **Marc Do-**
nias: Conceptualization, Supervision, Validation, Writing – review & editing; **Jean-Pierre Da**

Costa: Supervision, Validation, Writing – review & editing.

## Acknowledgments

This work was supported by the French Research Agency (ANR) [grant ANR-17-ROSE-0001 - BIPBIP]. We thank the technical teams of CTIFL, Fermes Larrre, Bordeaux Sciences Agro and INRAE for the preparation and provision of the plots. We also thank the company ELATEC and the researchers of the LaBRI laboratory for the conception of the acquisition module.

## References

[1] M. Everingham, L.V. Gool, C.K.I. Williams, J. Winn, The pascal visual object classes (VOC) challenge, Int. J. Comput. Vis. 88 (2010) 303–338, doi:10.1007/s11263-009-0275-4.

[2] L. Lac, J.-P.D. Costa, M. Donias, B. Keresztes, A. Bardet, Crop stem detection and tracking for precision hoeing using deep learning, Comput. Electron. Agricult. 192 (2022), doi:10.1016/j.compag.2021.106606.

[3] Lac, L., Costa, J.-P. D., Donias, M., Keresztes, B., & Marine, L. (2021). SDNet: Unconstrained object structure detector network for in-field real-time crop part location and phenotyping. In Proceedings of the British Machine Vision Conference. https://www.bmvc2021-virtualconference.com/conference/papers/paper_1278.html.