# Towards the characterization of crop and weeds at leaf scale: a large comparison of shape, spatial and textural features

Jehan-Antoine Vayssade, Gawain Jones, Jean-Noël Paoli

► To cite this version:

Jehan-Antoine Vayssade, Gawain Jones, Jean-Noël Paoli. Towards the characterization of crop and weeds at leaf scale: a large comparison of shape, spatial and textural features. Smart Agricultural Technology, 2023, 5, pp.art. 100245. 10.1016/j.atech.2023.100245. hal-04092538

HAL Id: hal-04092538

https://institut-agro-dijon.hal.science/hal-04092538

Submitted on 24 May 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

Check for
updates

# Towards the characterization of crop and weeds at leaf scale: A large comparison of shape, spatial and textural features

Jehan-Antoine Vayssade, Gawain Jones, Jean-Noël Paoli *

Agroécologie, Institut Agro Dijon, INRAE, Univ. Bourgogne-Franche-Comté, F-21000 Dijon, France

## ARTICLE INFO

Editor: Stephen Symons

**Keywords:**

Leaf classification

Crop and weed discrimination

Remote sensing

Multispectral

## ABSTRACT

In the context of crop and weeds discrimination, different methods are used to detect and classify plants from an acquisition system. Various estimators and descriptors are commonly used to characterize plants within an image. However, the available studies are based on disparate criteria, plants, and acquisition materials which does not allow an accurate estimation of the potential of criteria combinations applied to a new study. Thus, the objective of this study is to: (1) experimentally evaluate the discrimination potential of each criterion at the leaf scale, using images taken in field condition; (2) optimize the parameters of these criteria; and (3) determine the best combination of criteria to use.

A literature review is conducted to determine the set of criteria that could be used. A set of 3545 criteria is studied with an algorithm defined to select the best subsets of features (evaluated on a ground truth dataset). Finally, a classification of the vegetation cover is proposed, using the best performing subset. Results show the importance of selecting a smaller set of properties (at most 20 features among the 3545 available) and associating different feature types (for instance spatial with textural and morphological features).

## 1. Introduction

Crop and weed discrimination is a long-standing concern in modern agriculture, as weeds are generally considered a nuisance since they have a possible negative impact on crop yield or harvest quality. To have a reliable knowledge of the risks of yield loss over time, locally acquired experiences on 110 weed control trials under comparable conditions were conducted in France between 1993 and 2015 on three major annual crops (soft wheat, rapeseed and sunflower). A reduction of 25 q/ha, 4 q/ha and 3.5 q/ha was respectively observed [15]. Weeds are thus competing with their neighbours through competition for nutrients (nitrogen, potassium, phosphorus, carbon), light and water.

To face this problem the main weed control approach is based on the application of herbicide with sprayers. The methods have evolved from uniform chemical weeding of the plot to localized weeding in precision agriculture and the use of new technologies and digital tools. In this impetus, today's precision agriculture is seeking to discriminate even finer elements like the plant in real time. It allows to weed out undesirable individuals, to treat diseased crops or to supply nutrients for crops to reduce the use of synthetic products and the ecological impact of these products.

Major reviews of weed mapping techniques have been carried out by Monteiro et al. [36], Gao et al. [19], Mostajer Kheirkhah and Asghari [37] and Wang et al. [52] and show multiple approaches to weed mapping and feature extraction. Current research on crop/weed discrimination are either based on criteria extraction and exploitation or deep learning methods and provide good discrimination results, up to 77-97% [2] depending on the criteria, acquisition conditions and observed plants.

The extraction and exploitation of criteria are usually included in a computer vision pipeline that follows the key steps of image analysis including preprocessing, segmentation [48], feature extraction and classification [39]. The feature or criteria can be classified according to the nature of the information. It is possible to distinguish (i) shape criteria which are based on different representations of the contours, [1,40], (2) spatial criteria based on the distance of the individual to another element of the image, such as the distance to the nearest crop row [33], (3) texture features based on the analysis of the spatial distribution of the pixels (may include the soil) [43] and, finally (4) spectral indices at specific points on the surface can also be used [31], which could also be considered as a subset of texture features. Fig. 1 illustrates this general workflow.

<footnote>* Corresponding author.</footnote>

<footnote>E-mail addresses: jehan-antoine.vayssade@inrae.fr (J.-A. Vayssade), gawain.jones@agrosupdijon.fr (G. Jones), jean-noel.paoli@agrosupdijon.fr (J.-N. Paoli).</footnote>
<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Key steps of image analysis for crop/weed discrimination with (a) acquisition enhancement (b) vegetation indices (c) instance segmentation (d) feature extraction (e) classification.</figcaption></figure>

Deep learning approaches provide good results for the dataset they have been trained on but their performances are usually reduced on new datasets. Feature extraction could be a more generic approach. However, extracted features may lose effectiveness in real conditions because they are limited by the previous stages of the pipeline. In particular by the plant instance segmentation step, which is still a scientific challenge.

In this context, Vayssade et al. [48] have proposed a method based on deep learning in a previous study, which gives promising results in terms of instance segmentation and allows to work in dense foliage. This method detects not the whole plant but the leaves as they offers more stable criteria: there is less intra-individual variability than with a plant. For example, depending on the growth stage, the plant will contain a variable number of leaves with different orientations and sizes, these leaves may also have a partial overlap between them leading to an infinite number of cases. Thus the extracted criteria are viable for a given stage of development, for given crops and specific acquisition conditions. Some of these issues do not occur at the leaf level.

Moreover, the extracted criteria are more understandable at leaf scale: shape criterion, such as length or perimeter is less subject to interpretation at the leaf level than at the plant level. Texture criterion, such as gradients, gives the veins of the leaf, but there is no obvious correspondence at the plant scale. Thus the study of these criteria at the leaf scale could be promising to discriminate leaves between crops and weeds. In addition, such leaf-scale criteria have been used successfully for tree species discrimination, Cerutti et al. [14,13] showing an accuracy up to 90% with only shape criteria at leaf scale for species classification.

However, each type of criterion has different effectiveness and limitations. Spectral and texture features are more affected by illumination variations, while shape features are affected by segmentation quality. Moreover, because of these limitations, studies usually combine these types of features to take advantage of all the benefits. For instance Lin et al. [30] uses 4 vegetation indices, 3 textures, and 4 shape features. Zamani and Baleghi [56] uses 15 morphological, 12 spectral, 10 textural, 11 thermal features. However, this number of features remains low.

Therefore, in this study, 3545 features have been extracted for an exhaustive description at leaf level. Moreover, each feature has been optimized to obtain the best discrimination between crops and weeds in field condition, and the best feature sets have been established among all optimized features. In addition to this study, a Python framework allowing the extraction of these criteria is also available on github.com.

## 2. Material and data

### 2.1. Experimental plot

Data is acquired at the site of INRAE in Montoldre (Allier, France, at 46°20′30.3″N 3°26′03.6″E) within the framework of the ANR Challenge RoSE in 2019. The objective of the Challenge is to objectively compare the solutions proposed by participants [4,41]. Within this context, the challenge provides an evaluation plan to contestants and a set of experimental plots of bean and corn plants. In addition various natural weeds (yarrows, amaranth, geranium, plantago,...) and sowed ones

(mustards, goosefoots, mayweed and ryegrass) is managed to compare performances.

### 2.2. Multispectral camera

The images were acquired with the Airphen (Hyphen, Avignon, France) six-band multispectral camera. This is a multispectral scientific camera developed by agronomists for agricultural applications. The camera has been configured using the 450/570/675/710/730/850 nm bands with a 10 nm FWHM. The focal length of each lens is 8 mm. The raw resolutions for each spectral band are 1280x960 px with 12 bit precision. Due to the conception of the camera, spectral images are not aligned in space. A registration method based on previous work for this camera, with a registration accuracy down to sub-pixel is used [49]. After the registration, all spectral bands are rescaled to 1200 × 800 px and concatenated to channel-wise where each dimension refers to a spectral band.

### 2.3. Image acquisition and annotation

From the presented experimental plots a set of images were acquired. The camera is attached in front of an hybrid autonomous tractor called "TREKTOR" launched by SITIA (Bouguenais, France) in 2019. The camera is setup to have a top-down view of crop rows, thus it is placed at the end of a 2 m pole in front of the platform allowing to remove visible part of the robot and at 1.8 m from the ground to see all three crop rows. Crops and weeds were between phenological state 3 and 4 which means they had between 2 and 6 leaves. The ground truth is manually defined on images by experts with polygons around each leaf boundaries. In addition, polygons contain a crop/weed classification label. These annotations are performed using the VIA annotation software [16] and a total of 300 images of bean were annotated, 170 in June and 130 in October. This dataset is now available online<fnref n="1" /> and may help other studies working on leaves classification. In addition the crop rows for each image are also manually set. The total number of leaves is 90 538, with 31 524 for crops and 59 019 for weeds.

## 3. Methodology

The objective of this paper is to extract a wide variety of features and evaluate their performances for classifying leaves into crops and weeds, in addition to defining the best subset of feature space. The methodology followed in this study is as follows:

1. In order to evaluate criteria that could be used at the leaf level to discriminate between crop and weed leaves in field condition, a literature review was conducted. These criteria are organized into shape, spatial and textural features and presented in the section 3.1.

2. Most of the features have parameters that impact the extraction and classification performances of these features. To solve this problem,

<footnote n="1">¹ https://data.inrae.fr/dataset.xhtml?persistentId=doi:10.15454/JMKP9S&version=1.0.</footnote>
this article proposes to optimize these parameters by using an opti-
misation algorithm to efficiently sample the parameter space, it is
described in section 3.2.

3. The results of the literature review indicate that there is no con-
sensus on which of these criteria are the best for discriminating
between crop and weed (at plant or leaf scale). In order to de-
termine the best combination of criteria to use in discriminating
between crop and weed leaves in field condition, an algorithm to
select the best subsets of features is used and described in sec-
tion 3.3.

4. The performance of each method is evaluated on the ground truth dataset, using Area Under the Curve (AUC) as the performance metric. The best performing feature subset is presented to classify leaves into crops and weed. These subsets are extracted for each type of feature (locally) and among all feature (globally), this is available in the results section 4.

5. Finally a classification of the vegetation cover is proposed using the best performing features subsets and the results are discussed in section 4.3.

**3.1. Literature review**

In computer vision, shape, spatial and textural features are com-
monly used for describing and classifying objects. This section presents
a review of these features. To facilitate the extraction of features and
their merging, they have been programmed using the OOP<fnref n="2" /> paradigm
in Python. Each feature is thus named according to its type (shape /
spatial / spectral) and internal properties. Thus the following subsec-
tions present each of them according to its type. An overview of all the
criteria is available as supplementary material (Table 8).

3.1.1. *Shape criteria*

Shape properties are based on a structural analysis of the contours of related regions, it mostly consists of detecting morphological and anatomical traits. The shape can be coded as a set of positions (pixels) that go around the boundary. From this description, various information can be used. This type of analysis is found particularly in granulometry [21], geomatics [34], leaf classification [13] or even animal posture detection [47,11]. These criteria are the most intuitive, easy to implement, and unaffected by lighting. Each of the following paragraphs explain an extraction method as well as the resulting properties used for discrimination.

**Shape ellipse** Watcharabutsarakham et al. [53] have mentioned that the major and minor axis can be used to classify leaves. These properties can be obtained through an elliptical Hough algorithm as described by the article. From the retrieved ellipse, few more information can be computed such as the distance between the ellipse center and the detected shape center. Three factors of eccentricity, the directrix, the angle and the focal distance. Some of these properties were also used by Lottes et al. [32] for the classification of plant covers.

**Shape particle** Based on the work of Hentschel and Page [21], these shape metrics have been used to characterise and classify particles. Vayssade et al. [47] used it to classify goats activities within pasture, Bonneau et al. [11] to estimate the sow posture with an accuracy near deep learning methods (94 % vs 96 %). This set of 18 characteristics is based on a variety of ratios of the Feret diameter, area and perimeter. Some of them are known as: Equivalent Diameter, Eccentricity, Aspect Ratio, Form Factor, Roundness or Convexity. Others are unnamed features. The Feret diameter is defined as (1) the longest distance between two points on the contour, and (2) the shortest distance across the shape. Some of these properties are also used by Lottes et al. [32] and Saha et al. [44].

**Shape solidity** The solidity factor is widely used for weed plant clas-
sification [1]. The solidity factor is defined as a ratio between the
segmentation mask area and convex hull area. Thus, it is bound from 0
to 1. This factor indirectly computes the holes area of the regions. When
applied to weed discrimination, this factor allows to discriminate some
specific species like carrot from others. When carrots are segmented,
the segmentation mask is sparse, filled by a large amount of holes, due
to leaves structures. This does not occur for dicotyledon plants, and it is
less impacted by other monocotyledons weeds. At leaf scale, such crite-
ria provide information about the contour (leaf teeth) as the inner leaf
mask should be filled. This approach has been used for species classifi-
cation [45] and weed discrimination [32].

**Shape metrics** This set of 8 properties comes from the documentation of the shape metrics tools of the University of Connecticut,<fnref n="3" /> these are used for shape classification of lands [3] and building [38]. Few properties, such as Cohesion, Traversal, Range and Viable Interior was not implemented due to time complexity. The others 8 implemented metrics are named: Proximity, Exchange, Spin, Perimeter Index, Depth, Girth, Dispersion and Detour.

**Shape fragstat** The book by McGarigal [34] shows some of the features used at the landscape, class, or parcel scale for classification or shape similarity measures. The fragstats documentation shows a small set of criteria<fnref n="4" /> that can be exploited at the leaf scale such as: Perimeter-Area Ratio, Shape Index, Fractal Dimension Index, Linearity Index, Related Circumscribing Circle, Contiguity Index and the Perimeter Area Fractal Dimension. These features have been selected as shape criterion in our study because they reflect shape complexity for a given spatial scale and correspond to: Perimeter-area-ratio, Shape Index, Fractal Dimension Index, Linearity Index, Related Circumscribing Circle, Contiguity Index and Perimeter Area Fractal Dimension.

**Shape skeletonize** The skeletonization is commonly used in shape recognition techniques. It transforms the input mask (i.e. the shape) to a new structural representation of the shape [17]. This representation captures part of internal hierarchy. As instance, the Lottes et al. [32] study uses the skeleton length for weed discrimination. In this study more features are extracted from that structural representation. These properties are defined as: the number of *end_point*, the number of internal *branches*, the number of *intersections* and the total length of the skeleton (*pixel_count*).

**Shape angle** The distribution of angles between points around the contours can be exploited to measure the similarities between shapes [9]. This method was used to classify symbols (digits and letters). The procedure is defined as follows: for a point of the contour, the relative angles of all the other points that are in a given radius are computed. The radii and angles are stored in a “shape matrix” of dimensions: *radial_bins* × *angle_bins*. Each element (i, j) of the matrix contains a counter which corresponds, for a given point, to the number of points which are in this bin of radius *i* and in the bin of angle *j*. Finally, the procedure is repeated for each point of the contour to feed the “shape matrix”. This one is used as a discriminating variable.

**Shape Hu Moments** The Hu Moments have been proposed long time ago by Hu [23] and are part of structural analysis. The contours of the related regions are transformed into “moments”, which are then transformed into Hu Moments. The importance of this extraction comes from its translation, rotation, and scaling invariance. A similar element, e.g. two different bean leaves at various positions, rotation or scale will then

<footnote n="2">² Object-Oriented Programming.</footnote>

<footnote n="3">³ http://clear.uconn.edu/tools/Shape_Metrics/method.htm.</footnote>

<footnote n="4">⁴ http://www.umass.edu/landeco/research/fragstats/documents/Metrics/Shape%20Metrics/SHAPE%20METRICS.htm.</footnote>
be represented identically. This assertion remains true if intra-species and intra-individual variability are not taken into account. Hu Moment have been used for leaf classification challenge [42].

### 3.1.2. Spatial criteria

The spatial properties are based on the extraction of global information related to the agricultural plot. These extraction methods are based on a distance which is calculated between the extracted global information and the centroid of the related region. The distribution of the values is strongly correlated to the vegetation density, allowing to discriminate the weeds, mostly between rows.

**Spatial Row** The distance between the plant and the crop line is one of the most used spatial descriptors when such prior knowledge is observable [33,27,5]. The number of techniques to extract such rows within an agronomic image is large, most of them are based on Hough Transform applied to line detection, as [33]. Recent studies in deep learning show that CNN can be used to detect crop row in agricultural acquisition [6]. As the objective of this section is to evaluate the impact of the distance between plants and crop row (not the row detection performance), this study uses the crop rows from the ground truth.

**Spatial Blob** In most cases, plants can be classified into two types: monocotyledons and dicotyledons. At an early stage, when the plant emerges from the ground, its appearance is totally different according to this initial type. Monocotyledons usually look like a filament, while dicotyledons look like an oval shape when seen from above. With this information, a specific shape detection algorithm can be implemented. Such as Blob detection, which aims to detect groups of connected pixels in an image that share a common property (shape, color, ...). The objective of Blob detection is to identify and mark these regions. These marks are used to calculate the Euclidean distance between the centroid of leaf and the nearest blob. Thus, the distribution of values is strongly correlated to the density of vegetation.

**Spatial Corner** The detection of points of interest can be used to discriminate weeds. This is what is proposed in the article of Xu et al. [55], using an algorithm of detection of corners in the image. From these points of interest, the distance and the angle between the center of the plant and the closest points of interest are used as discriminating properties. In reality, this detection finds the leaves apexes. Here only the distance between the centroid and the nearest points of interest is computed since the angle is already present in Shape Ellipse properties.

### 3.1.3. Textural, color and spectral criteria

Texture properties are vast and include many things, like image transformations (Fourier, Wavelets, color spaces, ...), color, spectral properties or histograms to characterise observed surfaces. The article Mekhalfa and Yacef [35], have tested texture and color properties on RGB images of soybeans. The proposed color characteristics, are the means and standard deviations of the image in two color spaces (RGB and HSV). In terms of texture properties, i.e. an analysis of the spatial distributions of the colors, the paper relies on GLCM (Gray-Level Cooccurrence Matrix), Haralick and LBP (Local Binary Pattern) features. According to the study, these properties provide classification rates between 89.43% and 96.17%. Here, this work has been improved by adding their location to the extracted properties. Thus, all the extractions defined hereafter are automatically done on each individual bands, on NDVI and on the standard deviation (std) between spectral bands (which is also an image).

**Spectral signature** The spectral signature is defined as the value (at the center of the shape and at the center of its bounding box) of all spectral bands, the NDVI value and the standard deviation between spectral bands.

<table><caption>**Table 1**
Synthesis of colors moments.</caption><thead><tr><th>min</th><th>max</th><th>range</th></tr></thead><tbody><tr><td>mean</td><td>std</td><td>median</td></tr><tr><td>skewness</td><td>kurtosis</td><td>entropy</td></tr></tbody></table>

**Spectral Stats** As shown by Lottes et al. [32], statistical properties of the underlying bounding box (that contains the shape) can be computed and used for weed discrimination. According to this article, a set of nine properties for each spectral bands is extracted separately as well as a vegetation index (NDVI) and the standard deviation (std). These properties are based on color moments (Table 1).

**Spectral HoG** The HoG descriptors for Histogram of Oriented Gradient was used by Saha et al. [44] for weed discrimination. It is often used in computer vision, both for object detection and texture discrimination because of its properties of geometric invariance. The HoG descriptors are based on gradient intensity and edge directions. From this information, a histogram is computed and used as descriptor. Applied to a leaf, these properties allow to characterise leaf veins [46]. These features are generally robust to illumination changes and color differences due to the leaf maturity.

**Spectral DoG** Difference of Gaussian (DoG) is a texture feature extraction method that is used to identify and highlight edges and boundaries within an image. It is a type of band-pass filter that enhances image details and removes noise. The DoG algorithm works by taking the difference of two Gaussian filters with different standard deviations. The larger standard deviation produces a smoother image, while the smaller standard deviation enhances the edges and details. The difference between these two Gaussian filtered images creates the DoG image, which emphasizes the high-frequency components of the image, such as edges and textures.

**Spectral Hu Moment** Moments and Hu Moments are generally applied to segmentation masks or contours. It retrieves shape properties, such as perimeter, centroid, etc. Computing these features on spectral bands and NDVI could include more discriminating information.

**Spectral Gabor** Gabor filters have been widely used for texture analysis in mono-bands images. By extension, Jain and Healey [25] have proposed to use properties based on the results of Gabor filters applied to multispectral texture classification. They use a set of symmetric and circular Gabor filters, with three octave scales and four orientations to compute properties, such as averages and energy. These filters have been used in the field of weed discrimination by Ishak et al. [24]. In the current study, a symmetric and circular Gabor filters are applied with a single (optimized) octave scale and four orientations ([0, 40, 90, 158]) as defined by Jain and Healey [25]. However, extracted properties from such filters are based on color moments (Table 1) on each Gabor orientation filter.

**Spectral LBP** The LBP descriptor, for Local Binary Pattern is a feature that encodes texture information. The general principle of LBP is to compare the level of a pixel luminance with the levels of its neighbours (0 if inferior, and 1 else). Depending on the neighbour position, a weight is applied to the positive value ([1, 2, 4, 8, 16, 32, 64, 128]), the LBP value is then the sum of weighted positives values. The last feature, is the histogram of the leaf texture for each spectral bands and NDVI. This gives an account of information relating to regular patterns in the image, in other words texture.

**Spectral CSLBP** The CS-LBP operator for Center-Symmetric Local Binary Pattern is an extension of the previously defined LBP operator [8]. For each pixel, the absolute and symmetric difference of the neighbourhood is computed, if the value is higher than a fixed threshold $T = 0.1$
the element takes the value 1 (0 otherwise). The set of results is coded as LBP through a weighted sum depending on the neighbour location. These features are considered to be more robust against rotation than standard LBP.

**Spectral OCLBP** The OC-LBP (Opposite Color LBP) operator is an extension of the LBP operation. Banerji et al. [8] propose an LBP extension to the colorimetric domain by considering the inter channel relationship instead of local and spatial relationship of a pixel. For each color pair $(u, v), u > v$ the inter-channel properties are considered by $C = I(x, y, u) - I(x, y, v)$. With $I$ the multispectral image, $x, y$ the pixel position and $u, v$ the color channels to compare. The obtained image is defined in the same way as LBP: a weighted sum of positives values. These properties have been used by Waghmare et al. [51] for grape plant diseases detection.

**Spectral GLCM** This is a square matrix of dimension $N_g$, where $N_g$ is the number of gray levels in the image. The $[i, j]$ element of the matrix is generated by counting the number of times a pixel of value $i$ is adjacent to a pixel of value $j$, and then dividing the entire matrix by the total number of comparisons made. Each entry is thus considered as the probability that a pixel of value $i$ is adjacent to a pixel of value $j$ denoted $p(i, j)$, defining for us the co-occurrence matrix $G$. This has been used by Bakhshipour et al. [7] for weed discrimination.

**Spectral Haralick** The basis of these features is the GLCM (grayscale co-occurrence matrix). From that, Haralick has defined 14 texture descriptors that can be computed. This estimator was tested for canopy discrimination by Wong et al. [54].

**Spectral Polyfit** A new type of feature based on polynomial fitting is produced. Multispectral images may lack information due to the selection of the spectral domain (The Airphen camera uses 6 spectral bands). To recover the original spectrum distribution across the surface, a simple solution that can be used is curve fitting [12], by using the value of observed pixel and their distances to the center of the shape. The result of the fitting is the polynomial parameter, which can be used as a discriminant variable.

**Spectral Shi Tomasi** The Shi-Tomasi feather is a kind of key point detector using the eigenvalue of the second moment of the color matrix [20]. On these values, a threshold is applied to determine if a pixel can be a corner or not. This algorithm has an interesting step, it transforms the input color image space into a gray scale color space, through this eigenvalue decomposition. Here, instead of finding the local maxima of this transformation, which are the key points. A histogram is used to describe the observed surface, so the values of corresponding to a corner accumulate in a specific bins of the histogram, while flat surfaces accumulate on other bins, etc. This transformation is applied on the first 3 bands of our data (450 nm, 570 nm, 650 nm) which are nearest to a RGB color space.

## 3.2. Optimization

Some features have parameters, like the number of bins in case of histogram analysis, the inner and outer radius of “shape angles” and so on. Some parameters are discrete numbers, but others are in a continuous space, like four parameters of the “texture gabor” which requires to set them before using it (sigma, $lm$, $gm$, $ps$). Setting these parameters has a critical impact on classification performances, they can be set empirically. But to optimize the performances of extracted features, these parameters must be optimized properly and automatically. This is known as an parameter optimization problem.

There are many ways to sample the parameter space. One approach is grid search, where the parameter space is divided into a number of discrete regions (or grids) and a search is performed in each point of

<figure><img src="image_5.png" /><figcaption>Fig. 2. Feature parameter optimization procedure.</figcaption></figure>

<caption>Table 2</caption>

Five epochs of the optimization procedure through ASHA algorithm. The objective column shows the model performance.

<table><thead><tr><th>epoch</th><th>sigma</th><th>lm</th><th>gm</th><th>ps</th><th>objective</th></tr></thead><tbody><tr><td>1616</td><td>1.865</td><td>5.720</td><td>0.409</td><td>0.403</td><td>0.621</td></tr><tr><td>1617</td><td>1.838</td><td>5.732</td><td>0.363</td><td>0.421</td><td>0.604</td></tr><tr><td>1618</td><td>1.894</td><td>4.960</td><td>0.431</td><td>0.335</td><td>0.556</td></tr><tr><td>1619</td><td>3.363</td><td>8.253</td><td>0.083</td><td>0.690</td><td>0.465</td></tr><tr><td>1620</td><td>7.705</td><td>4.655</td><td>0.986</td><td>0.090</td><td>0.517</td></tr></tbody></table>

the grid [29]. Another approach is random search, where a random point is chosen in the parameter space and the performance if evaluated [10]. However, they can also lead to suboptimal solutions, because both methods are brute force approaches that scale poorly with the features parameters number. Therefore, it is important to choose an appropriate sampling method for the problem at hand. To achieve this optimization, more complex algorithms must be used to sample the parameter space efficiently.

An asynchronous early stop method based on the continuous halving algorithm is used to solve this challenge, this is the ASHA algorithm (Asynchronous Successive Halving). The ASHA algorithm is a randomized, asynchronous and incremental algorithm for sampling the parameter space. It is based on the continuous halving algorithm, which is a greedy algorithm that divides the parameter space into two halves and selects the best half at each step. The ASHA algorithm uses this approach to iteratively divide the parameter space into smaller subsets. At each step, it selects the best subset based on a performance metric and stops when the subset is small enough that the performance is no longer improving. This approach is more efficient than grid search or random search, because it can find good solutions without exploring the entire parameter space. The asynchronous part of this algorithm allows it to run on multiple processors, which can speed up the search [22].

In addition, to sample the parameter space correctly, each parameter has been manually bounded to limit the search space. The ASHA algorithm has been implemented in the Sherpa software package [22] in Python. Finally, the optimization is done through 10 000 epochs. At each epoch a 2-fold cross-validation is used for training and validating a decision tree classifier. This should enforce the learning of more stable properties and avoid overfitting, because the first fold contains the majority of data obtained in June, while the second fold contains the data obtained in October. The decision tree classifier is chosen to use simple decision criteria, rather than complex model, to extract most relevant features. The performance of the extracted features is therefore the performance of the underlying learned classifier (average of the cross-validation procedure). The Fig. 2 synthesizes the optimization procedure.

As an example, Table 2, shows 5 epochs for the optimization of spectral Gabor features (parameters: $sigma$, $lm$, $gm$ and $ps$), the objective value is the value to be maximized, it corresponds to the performances of classification model evaluated by the Area Under the Curve metric [18]. After the optimization process, all features are extracted.
<figure><img src="image_6.png" /><figcaption>Fig. 3. Sequential feature selection algorithm.</figcaption></figure>

### 3.3. Feature selection

Now that all the features are optimized and extracted, the main question of this article can be explored. What are the best features to classify leaves into crops and weeds. Since the number of features is huge (3545), as shown in Table 8, a manual estimation of each feature performances cannot be established properly. To answer this question, researchers in this field typically use a metric to estimate how well a feature fits the problem (correlation coefficient or root mean squared error), and then the best features are used. However, a better combination might exist, e.g., merging two high rank criteria might be less accurate than merging a high rank with a low rank. As an example, row distance and a texture characteristic, the efficiency may be greater than that of two texture criteria, because row distance is likely complementary to texture property.

Instead of evaluating each feature through a metric, a data mining technique is proposed, such as “Sequential Forward Selection” (SFS) [26]. It is an algorithm that analyzes each feature to check whether it is going to be useful for the classification task. To put it in another way the motivation behind this algorithm is to automatically select the subset of features most relevant to the problem. Thus, it reduces the initial *d*-dimensional feature space to a *k*-dimensional feature subspace, where *k* < *d*. The Fig. 3 shows how the algorithm works.

## 4. Results and discussion

### 4.1. Feature optimization

Among the 25 types of properties, only 12 of them have parameters that need to be optimized. For each of them the parameters and the underlying classifier performances have been extracted in the below Table 3 (for spatial and shape feature) and Table 4 (for spectral features).

Table 3 and Table 4, show a noticeable improvement of the feature performances through this optimization procedure. Few features are strongly enhanced, from 50% to 62.81% for “Shape Angles”, or from 46.75% to 72.39% for “Spectral DoG”, showing the necessity of feature parameter optimisation. Irrelevant features start to appear here, for features that have parameters, this is the case for “Spatial Corner”, “Spatial Blob” or “Spectral HOG” that does not exceed 60% of AUC with a decision tree classifier.

The high accuracy of feature classification induced by feature parameter optimization can be explained by different reasons. First, these optimization techniques allow for an efficient exploration of the parameter search space, enabling the identification of the optimal values that maximize their individual performances. Second, the improvement is

<caption>**Table 3**</caption>

Feature parameter optimization through ASHA algorithm, the objective column shows the performance of the underlying classifier.

<table><caption>**Table 4**</caption><thead><tr><th>Type</th><th>Parameter</th><th>Value</th><th>Objective</th><th>Worst</th></tr></thead><tbody><tr><td rowspan="4">Shape Angles</td><td>bins radius</td><td>2</td><td rowspan="4">62.81</td><td rowspan="4">50.00</td></tr><tr><td>bins_theta</td><td>22</td></tr><tr><td>radius_inner</td><td>0.9827</td></tr><tr><td>radius_outer</td><td>9.6870</td></tr><tr><td rowspan="4">Spatial Corner</td><td>blockSize</td><td>2</td><td rowspan="4">59.52</td><td rowspan="4">51.49</td></tr><tr><td>k</td><td>0.0995</td></tr><tr><td>ksize</td><td>1</td></tr><tr><td>threshold</td><td>0.09926</td></tr><tr><td rowspan="4">Spatial Blob</td><td>area</td><td>0</td><td rowspan="4">52.00</td><td rowspan="4">51.23</td></tr><tr><td>circularity</td><td>0</td></tr><tr><td>inertia</td><td>0</td></tr><tr><td>convexity</td><td>1</td></tr></tbody></table>

Feature optimization through ASHA algorithm. To show the benefit of the feature optimization process, the column **Worst**, shows is the least discriminant parameters. The column **Objective** shows the best performance of the model.

<table><thead><tr><th>Type</th><th>Parameters</th><th>Values</th><th>Objective</th><th>Worst</th></tr></thead><tbody><tr><td rowspan="2">Spectral DoG</td><td>sigma_a</td><td>17.7569</td><td rowspan="2">72.39</td><td rowspan="2">46.75</td></tr><tr><td>sigma_b</td><td>0.0475</td></tr><tr><td rowspan="4">Spectral Gabor</td><td>gm</td><td>37.51</td><td rowspan="4">71.68</td><td rowspan="4">49.95</td></tr><tr><td>lm</td><td>3.8094</td></tr><tr><td>ps</td><td>0.6573</td></tr><tr><td>sigma</td><td>8.7758</td></tr><tr><td>Spectral GLCM</td><td>normed<br>symmetric</td><td>false<br>true</td><td>68.96</td><td>61.55</td></tr><tr><td rowspan="2">Spectral Zernike</td><td>m</td><td>1</td><td rowspan="2">60.14</td><td rowspan="2">55.12</td></tr><tr><td>n</td><td>1</td></tr><tr><td>Spectral Haralick</td><td>size</td><td>128</td><td>69.66</td><td>52.99</td></tr><tr><td>Spectral HOG</td><td>size</td><td>256</td><td>54.96</td><td>51.08</td></tr><tr><td>Spectral LBP</td><td>nbins</td><td>256</td><td>69.17</td><td>53.16</td></tr><tr><td>Spectral Shi-Tomasi</td><td>nbins</td><td>14</td><td>67.23</td><td>51.84</td></tr><tr><td>Spectral CSLBP</td><td>nbins</td><td>22</td><td>69.53</td><td>55.33</td></tr></tbody></table>

measured between best and worst performances (instead of best and expert / usual performances), resulting in an important gain. An expert choice of parameters (or the use of usual values) may lead to better performances than the worst case and reduce the gain induced by parameter optimization.

Once the feature parameter optimization is performed, all features (with or without parameters) can be extracted. Thus, the next step is to use the feature selection algorithm to select the best criteria from all extracted features. The next section will show the results of the feature selection algorithm.

### 4.2. Feature selection by type

For this study, the SFS algorithm is applied independently on each feature type (locally), and on all extracted features (globally). This allows us to check which features are the most relevant for their categories as well as globally. For the SFS algorithm, the number of features included in the subspace is considered a stopping criterion. Here, the stopping criterion has been set to *k* = 10 for each feature type and *k* = 20 for all features. The result of the procedure is available on the next Table 5 for spatial feature, in Table 6 for shape feature and in Table 7 for spectral features. Finally Fig. 4 shows best global criteria. On all of these Tables, the best subspace size is highlighted in bold for each feature type, and the green cells indicate the best feature type to use for a specific feature space size.

Since these 3 feature types extract only one feature each, Table 5 shows only one performance for each feature type. But it also shows
<table><caption>**Table 5**
Best features for spatial features. Best results for a specific feature count are highlighted.</caption><thead><tr><th>feature count</th><th>1</th></tr></thead><tbody><tr><td>Spatial Blob</td><td>51.80</td></tr><tr><td>Spatial Corner</td><td>56.59</td></tr><tr><td>Spatial Row</td><td>66.90</td></tr></tbody></table>

that among the 3 spatial feature “Spatial Row”, which is the distance of the leaf to the crop row, has the most relevant performance. This is expected, as it allows to discriminate inter-row weeds. However, the weed coverage in October were high and overlapped the crop row, reducing this feature performance. The worst is “Spatial Blob”, which may be due to a lack in the optimization procedure for this feature, or more likely to the lack of discriminating information in this analysis, as for “Spatial Corner” that offers moderate performances. The next Table 6 is for shape criteria.

As shown in Table 6, “Shape Solidity” that have been found relevant at plant scale [1] is less relevant at leaf scale. The first feature selected for “Shape Skeletonize” is the pixel-count. It refers to the length of the skeleton, which is the most discriminating property among the others of this feature type. Among all shape feature, the “Shape Particle” is the most important one, the four best features are respectively “inv-area”, “m1”, “m2”, and “form-factor” with a score of 76.08 %. While the second best shape property is “Shape Fragstats” that is used to classify land plot, its 3 best features are respectively the ‘perimeter-area-ratio’, the ‘related-circumscribing-circle’, and the ‘perimeter-area-fractal-dimension’ with a score of 74.83 %. The four first features selected for the “Shape Ellipse” seem quite good, such as “center-distance”, “minor-axis”, “angle”, and “eccentricity 2” with an accuracy score of 69.76 %. The other ones seem to been less relevant for crop and weed discrimination at leaf scale. Finally, Table 9 in

the supplementary material, shows the name of the ten top best shapes features.

As shown in Table 7, the best features are the ones related to the texture and color of leaves. If a single feature should be used then, the best feature is the “Spectral Shi-Tomasi”, which is related to the roughness of the texture, since it is the first step of corner detection, however the remaining features are less discriminant than the others feature types. Among these spectral properties, the LBP, OCLBP and CSLBP were expected to produce better performances (they max out at around 70 %), this means that the histogram construction is not producing relevant features. The same remarks can be established for the Zernike features even if their performances are slightly better (around 76.60 %).

“Spectral Signature” is commonly used for vegetation classification, it uses the spectral value of the center of the shape (or window) as a criterion. The performances of this feature are better than the features previously discussed but max out at around 80 %. The next feature, which is the “Spectral Hu Moment”, is a transformation of the spectral value of the center of the shape through Hu Moment, this transformation shows a significant increase in the performances of such features, from 79.53 % for “Spectral Signature” to 83.19 % for “Spectral Hu-Moment”, in addition this transformation seems more interesting on spectral properties than shape properties as seen in Table 6.

Features “Spectral Polyfit” and “Spectral Stats” are relatively close by definition but show a significant difference on their performances. “Spectral Stats” extracts statistical properties (Table 1) of the underlying sub-texture which contains the leaf and the soil around it, while “Spectral Polyfit” uses the spectral information of the sub-texture to reconstruct the spectral distribution of the leaf surface, then the polynomial values are used as discriminating information. The initial assumption was that “Spectral Polyfit” (which is an attempt to reconstruct the initial spectrum distribution from the discrete values) should have been more relevant than “Spectral Stats”.

“Spectral HOG” for Histogram of Oriented Gradient, is generally used to retrieve gradient information. In case of leaf analysis, this trans-

<caption>**Table 6**</caption>

Best classification features for each feature type through ASHA algorithm. Each line presents the results for a specific feature type. The feature count is the number of feature(s) combined for a specific feature type. The presented results are the optimal combinations for each feature counts. Column “full” is the result obtained with all the features for a specific feature type. Best results for a type of features are in bold, best results for a specific feature count are highlighted.

<table><caption>**Table 7**</caption><thead><tr><th>feature count</th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>10</th><th>full</th></tr></thead><tbody><tr><td>Shape Angles</td><td><strong>65.73</strong></td><td>65.72</td><td>65.73</td><td>65.72</td><td>65.72</td><td>65.73</td><td>65.72</td><td>65.73</td><td>65.73</td><td>65.72</td><td>68.38</td></tr><tr><td>Shape Ellipse</td><td>60.84</td><td>67.47</td><td>69.46</td><td><strong>69.76</strong></td><td>69.56</td><td>69.48</td><td>69.28</td><td>69.10</td><td>68.80</td><td>68.54</td><td>67.58</td></tr><tr><td>Shape Fragstats</td><td>67.85</td><td><strong>70.10</strong></td><td><strong>74.83</strong></td><td>73.62</td><td>73.10</td><td>72.55</td><td>-</td><td>-</td><td>-</td><td>-</td><td>72.55</td></tr><tr><td>Shape Hu-moment</td><td>56.87</td><td>59.79</td><td>63.30</td><td>64.87</td><td>65.52</td><td><strong>65.78</strong></td><td>65.74</td><td>-</td><td>-</td><td>-</td><td>65.74</td></tr><tr><td>Shape Metrics</td><td>57.01</td><td>62.74</td><td>66.88</td><td>67.81</td><td>68.52</td><td>68.76</td><td><strong>68.98</strong></td><td>68.89</td><td>-</td><td>-</td><td>68.89</td></tr><tr><td>Shape Particle</td><td>67.88</td><td>70.15</td><td>74.50</td><td><strong>76.08</strong></td><td>76.04</td><td>75.99</td><td>76.03</td><td>75.95</td><td>75.83</td><td>75.73</td><td>73.13</td></tr><tr><td>Shape Skeletonize</td><td><strong>68.81</strong></td><td>68.81</td><td>68.65</td><td>68.52</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>68.52</td></tr><tr><td>Shape Solidity</td><td><strong>59.04</strong></td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>59.04</td></tr></tbody></table>

Best features individually for each feature type for classification through ASHA algorithm. Each line presents the results for a specific feature type. The feature count is the number of feature(s) combined for a specific feature type. The presented results are the optimal combinations for each feature counts. Column “full” is the result obtained with all the features for a specific feature type. Best results for a type of features are in bold, best results for a specific feature count are highlighted.

<table><thead><tr><th>feature count</th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>10</th><th>full</th></tr></thead><tbody><tr><td>Spectral LBP</td><td>69.81</td><td>70.71</td><td><strong>70.72</strong></td><td>70.71</td><td>70.71</td><td>70.71</td><td>70.71</td><td>70.71</td><td>70.71</td><td>70.71</td><td>72.61</td></tr><tr><td>Spectral OCLBP</td><td>61.69</td><td>65.09</td><td>68.26</td><td>69.34</td><td>69.94</td><td>70.34</td><td>70.47</td><td>70.66</td><td>70.56</td><td><strong>70.60</strong></td><td>73.03</td></tr><tr><td>Spectral CSLBP</td><td><strong>70.36</strong></td><td>70.36</td><td>70.36</td><td>70.36</td><td>70.36</td><td>70.36</td><td>70.36</td><td>70.36</td><td>70.36</td><td>70.36</td><td>68.12</td></tr><tr><td>Spectral Zernike</td><td>57.26</td><td>68.67</td><td>72.58</td><td>73.42</td><td>74.44</td><td>75.42</td><td>75.80</td><td>76.19</td><td>76.37</td><td><strong>76.60</strong></td><td>76.91</td></tr><tr><td>Spectral Shi-Tomasi</td><td><strong>71.11</strong></td><td>67.74</td><td>70.02</td><td>70.29</td><td>71.25</td><td>72.01</td><td>72.59</td><td>72.84</td><td>72.94</td><td><strong>73.03</strong></td><td>72.00</td></tr><tr><td>Spectral Signature</td><td>66.76</td><td>67.37</td><td>74.40</td><td>75.79</td><td>77.25</td><td>78.18</td><td>79.23</td><td>79.35</td><td><strong>79.58</strong></td><td>79.53</td><td>63.65</td></tr><tr><td>Spectral Hu Moment</td><td>68.29</td><td>68.71</td><td>70.48</td><td>74.21</td><td>81.70</td><td>83.09</td><td>83.15</td><td>83.05</td><td><strong>83.22</strong></td><td>83.19</td><td>71.29</td></tr><tr><td>Spectral Stats</td><td>68.97</td><td>72.58</td><td>79.09</td><td>81.61</td><td>83.36</td><td>84.37</td><td>84.98</td><td>-</td><td>85.65</td><td><strong>86.29</strong></td><td>74.58</td></tr><tr><td>Spectral Polyfit</td><td>59.57</td><td>62.80</td><td>66.01</td><td>66.94</td><td>67.70</td><td>68.40</td><td>68.97</td><td>69.54</td><td>70.48</td><td><strong>70.63</strong></td><td>70.20</td></tr><tr><td>Spectral HOG</td><td>55.99</td><td>64.73</td><td>69.83</td><td>72.01</td><td>73.93</td><td>74.83</td><td>75.44</td><td>75.95</td><td>76.37</td><td><strong>76.67</strong></td><td>74.40</td></tr><tr><td>Spectral DoG</td><td>70.54</td><td>74.04</td><td>78.36</td><td>80.76</td><td>82.54</td><td>83.53</td><td>84.64</td><td>85.42</td><td>86.03</td><td><strong>86.33</strong></td><td>75.76</td></tr><tr><td>Spectral Gabor</td><td>69.50</td><td>74.48</td><td>-</td><td>80.07</td><td>82.23</td><td>83.53</td><td>84.52</td><td>85.14</td><td>85.60</td><td><strong>86.40</strong></td><td>74.92</td></tr><tr><td>Spectral GLCM</td><td>59.57</td><td>66.22</td><td>73.84</td><td>76.87</td><td>78.93</td><td>80.12</td><td>80.97</td><td>81.61</td><td>82.13</td><td><strong>82.56</strong></td><td>79.52</td></tr><tr><td>Spectral Haralick</td><td>63.80</td><td>-</td><td>75.85</td><td>78.89</td><td>81.16</td><td>82.56</td><td>83.69</td><td>84.62</td><td>85.01</td><td>85.33</td><td><strong>85.51</strong></td><td>81.30</td></tr></tbody></table>
<figure><img src="image_7.png" /><figcaption>Fig. 4. Best features among all extracted feature for classification through ASHA algorithm. Best overall result is highlighted.</figcaption></figure>

formation detects leaf veins [28], which is a criterion to discriminate
leaf species. Since this analysis doesn't produce good enough perfor-
mances, it is possible that the spatial resolution is too low for this kind
of methodology.

The remaining “DoG”, “Gabor”, “GLCM” and “Haralick” features of-
fer good performances as expected. “Spectral Gabor” is the best of the
four when ten features are used, while “DoG” and “GLCM” show slightly
less accurate results. Finally, Table 10 in the supplementary material,
shows the name of those ten top best spectral features.

4.3. *Best of all features*

As shown previously (Table 7), the best starting feature is the second
bin of “Spectral Shi-Tomasi” histogram, thus it is selected firstly and its
performance level is found in the first point of Fig. 4 (lower-left). The
second selected feature is “Spatial Row”, it is expected that this feature
should be on the top as it is a complementary with the color/texture fea-
tures. At this stage, it is interesting to note that this combination is less
accurate 74.98 % than two texture Haralick features 75.85 %. The next
three features are “Spectral Stats 2_min”, “Spectral Stats 5_max” and

"Spectral Gabor 3_0 mean". Which is the minimum spectral value of the
underlying sub-texture of the 650 nm spectral band for the first feature.
In the same way, it is the maximum for the 850 nm spectral band. The
Gabor filter is applied on all spectral bands, the discriminating criterion
is one from the 450 nm. This leads to a performance of 85.95 %. The next
feature is "Shape Metrics Detour", with a performance of 87.41 % which
beats any previous classification model.

To conclude on these results, the best classification model includes 1
spatial feature, 6 shape features and 13 spectral features. In the previous
Fig. 4, only 20 features are shown, if more features are extracted, the
performances begin to decrease (21 features = 91.16 %), this means that
the performance reaches a vertical asymptote. To show this a model
has been learnt with all features (3545), the cross-validation score is
79.97 %.

4.4. Visual results

Fig. 5 shows two rows with field beans plants. Weeds are located between these two rows and on each of them. Most of plant and weed leaves are correctly classified (plant leaves are colored green, and weed leaves are colored blue). However, this image suggests a problem remains for the smallest leaves that are in the center of the plant. These crop leaves, colored in red, are classified as weed leaves. This classification is induced by shape features, which are less relevant for this category of leaves than for the others.

Fig. 6 focuses on specific parts of plant rows. Weeds on Fig. 6.a are mostly monocots, whereas those visible on Fig. 6.b are mostly dicots. These two figures illustrate the quality of classification, which is equivalent for the two weed groups. Fig. 6.c presents a dense foliage, with highly developed wild mustard. On this figure, the major part of mustard leaves is correctly classified. The classification of field bean leaves depends on their visibility. Several partially occluded leaves are classified as weed leaves (in red). Conversely, Fig. 6.d presents sparse foliage, where many bean plants have not emerged. In that case, the small bean leaves are also classified as weed leaves.

All of these images illustrate the difficulty in classifying leaves
which, being in the row of crops, have particular shapes, because they
are partially obscured or because they are not sufficiently developed.

5. Conclusion

The features evaluated were identified from a thorough literature
review. These features can be grouped as spatial properties, based on

<figure><img src="image_8.png" /><figcaption>Fig. 5. This figure shows the classification of leaves, from an image taken in June 2019. The blue color corresponds to well classified weeds. The green color corresponds to well classified crops. While the purple and red color correspond respectively to weeds and crops poorly classified.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>**Fig. 6.** This figure shows the classification of leaves, from parts of four images taken in June 2019. The blue color corresponds to well classified weeds. The green color corresponds to well classified crops. While the purple and red color correspond respectively to weeds and crops poorly classified. These pictures show that leaves inside crop rows are correctly classified, as are monocots. On the other hand, the emerging leaves of crops and some weed type leaves of dicotyledons are incorrect.</figcaption></figure>

notions of distances at the image scale, shape properties allowing to characterize the morphological features of the individuals. And, finally, properties of textures, colors and spectra were proposed to extract information related to the composition and internal structures of leaves.

From this survey, it came out that some methods have parameters that influence the discrimination potential of these properties. Therefore, an algorithm was proposed to optimize these feature parameters before the extraction and evaluation of these features. It has been shown that the optimization of these feature parameters plays a crucial role in the performance of these criteria. For example, “Shape Angles” properties went from 50% to 62.81% of discrimination potential after optimization, as well as the “Spectral DoG” properties from 46.75% to 72.39%, etc.

After the optimization of these properties, the set of properties was extracted from the dataset for evaluation. In this evaluation, it was shown the importance of selecting a smaller set of properties, the overall score is better with 20 features than all the features (3545). It is also important to notice the contribution of each feature decreases as the number of features increases, resulting in a less interesting “performance over computing time” ratio. For each property type, the top 10 properties were defined. Then the top 20 among all properties combined were defined. These results show a crop/weed discrimination performance up to 91 percent with 20 properties, while a performance of 79.97% is observed using the 3545 properties. Indeed, the usual classification techniques do not seem to be able to handle large dimensions of properties. But here, the performances of classifications proposed by smaller sets are superior to the use of all the criteria. Thus, the definition of this minimal set also allows to decrease the computation time.

## 6. Further research

In this study, feature selection is only based on classification performances of features. Computation speed is neither estimated nor taken into account. However, depending on the application, a compromise between computation speed and classification performance could be

used. The selection of the best features for real-time weed detection and elimination is of particular interest. Fig. 4 shows that the major classification improvement comes from the 10 first features (from 71 to 90%), the next 10 features improve the classification result of a bit more than 1%. This observation shows that a study focused on performance over computing time should be relevant. This is also a key step in developing on-board weed control systems in autonomous vehicles where computing power can be restricted and time matters the most.

An hybrid approach could also be explored to improve performances for specific applications or decrease computing time. Deep learning could be used to improve feature extraction. Following the approach that has been undertaken in a previous article [50], an alternative based on function approximators is possible. The best shape properties could then be defined from a fixed representation of the contour. The same is true for texture and spectral properties. Moreover, the computing time needed to extract these features does not allow a real time use, which reinforces the need to study deep learning.

Considering precision farming applications, this approach could be replicated to select the best features for a specific set of crop(s) and weeds. The objective could change from an application to another and should be linked to the classification accuracy requirements and the time available. A real-time application may focus on computing time with less consideration for classification performance. A UAV approach may allow more computing time as the result is already delayed from the acquisition. In both case, the image resolution may be an important setting: too high and the computing time may be too important, too low and the classification performance may drop too much. This resolution may also impact the feature contribution to the overall classification with spectral or shape feature requiring highest resolution than spatial features. Consequently, a study focusing on the impact of image resolution on various features could provide insightful knowledge. It could also be interesting to work on several datasets with different crop and weeds association to evaluate the genericity of the proposed approach.
# Funds

Funding was received from the European Union's Horizon 2020 research and innovation program under grant agreement No 727321 (project acronym, IWMPRAISE). It was also funded by the French National Research Agency Challenge RoSE [grant agreement ID: ANR-17-ROSE-0002] (project acronym: ROSEAU). In addition, a collaboration with SITIA company is existing.

# Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

# Data availability

Data will be made available on request.

# Appendix A. Supplementary material

Supplementary material related to this article can be found online at https://doi.org/10.1016/j.atech.2023.100245.

# References

[1] F. Ahmed, H.A. Al-Mamun, A.H. Bari, E. Hossain, P. Kwan, Classification of crops and weeds from digital images: a support vector machine approach, Crop Prot. 40 (2012) 98-104, https://doi.org/10.1016/j.cropro.2012.04.024.

[2] N. Ahmed, M.U. Khan, S. Asif, An automatic leaf based plant identification system, Sci. Int. 28 (2016) 427-430.

[3] S. Angel, J. Parent, D.L. Civco, Ten compactness properties of circles: measuring shape in geography, Can. Geogr. 54 (2010) 441-461.

[4] G. Avrin, D. Boffety, S. Lardy-Fontan, R. Régnier, R. Rescoussié, V. Barbosa, Design and validation of testing facilities for weeding robots as part of rose challenge, in: Evaluating Progress in IA (EPAI), 2020.

[5] M.D. Bah, A. Hafiane, R. Canals, Deep learning with unsupervised data labeling for weeds detection on UAV images, CoRR, arXiv:1805.12395, 2018.

[6] M.D. Bah, A. Hafiane, R. Canals, Crownet: deep network for crop row detection in uav images, IEEE Access 8 (2019) 5189-5200.

[7] A. Bakhshipour, A. Jafari, S.M. Nassiri, D. Zare, Weed segmentation using texture features extracted from wavelet sub-images, Biosyst. Eng. 157 (2017) 1-12.

[8] S. Banerji, A. Verma, C. Liu, Lbp and color descriptors for image classification, in: Cross Disciplinary Biometric Systems, Springer, 2012, pp. 205-225.

[9] S. Belongie, J. Malik, J. Puzicha, Shape matching and object recognition using shape contexts, IEEE Trans. Pattern Anal. Mach. Intell. 24 (2002) 509-522.

[10] J. Bergstra, Y. Bengio, Random search for hyper-parameter optimization, J. Mach. Learn. Res. 13 (2012).

[11] M. Bonneau, B. Benet, Y. Labrune, J. Bailly, E. Ricard, L. Canario, Predicting sow postures from video images: comparison of convolutional neural networks and segmentation combined with support vector machines under various training and testing setups, Biosyst. Eng. 212 (2021) 19-29, https://doi.org/10.1016/j.biosystemseng.2021.09.014.

[12] A. Brown, Spectral curve fitting for automatic hyperspectral data analysis, IEEE Trans. Geosci. Remote Sens. 44 (2006) 1601-1608, https://doi.org/10.1109/TGRS.2006.870435.

[13] G. Cerutti, L. Tougne, D. Coquin, A. Vacavant, Leaf margins as sequences: a structural approach to leaf identification, Pattern Recognit. Lett. 49 (2014) 177-184, https://doi.org/10.1016/j.patrec.2014.07.016.

[14] G. Cerutti, L. Tougne, J. Mille, A. Vacavant, D. Coquin, Understanding leaves in natural images - a model-based approach for tree species identification, Comput. Vis. Image Underst. 117 (2013), https://doi.org/10.1016/j.cviu.2013.07.003.

[15] S. Cordeau, F. Dessaint, C. Denieul, L. Bonin, F. Vuillemin, M. Delattre, A. Rodriguez, J. Guillemin, B. Chauvel, La nuisibilité direct des adventices en grandes cultures: quelles réponses nous apportent les essais désherbage, in: 23e Conférence du COLUMA-journées internationales sur la lutte contre les mauvaises herbes, DIJON, 2016.

[16] A. Dutta, A. Zisserman, The VIA annotation software for images, audio and video, in: Proceedings of the 27th ACM International Conference on Multimedia, ACM, New York, NY, USA, 2019.

[17] A. Erdem, S. Tari, A similarity-based approach for shape classification using Aslan skeletons, in: Meta-Heuristic Intelligence Based Image Processing, Pattern Recognit. Lett. 31 (2010) 2024-2032, https://doi.org/10.1016/j.patrec.2010.06.003.

[18] T. Fawcett, An introduction to roc analysis, in: ROC Analysis in Pattern Recognition, Pattern Recognit. Lett. 27 (2006) 861-874, https://doi.org/10.1016/j.patrec.2005.10.010.

[19] J. Gao, W. Liao, D. Nuyttens, P. Lootens, J. Vangeyte, A. Pižurica, Y. He, J.G. Pieters, Fusion of pixel and object-based features for weed mapping using unmanned aerial vehicle imagery, Int. J. Appl. Earth Obs. Geoinf. 67 (2018) 43-53, https://doi.org/10.1016/j.jag.2017.12.012.

[20] C. Harris, M. Stephens, et al., A combined corner and edge detector, in: Alvey Vision Conference, Citeseer, 1988, pp. 10-5244.

[21] M.L. Hentschel, N.W. Page, Selection of descriptors for particle shape characterization, Part. Part. Syst. Charact., Meas. Descr. Part. Prop. Behav. Powders Other Disperse Syst. 20 (2003) 25-38.

[22] L. Hertel, J. Collado, P. Sadowski, J. Ott, P. Baldi, Sherpa: robust hyperparameter optimization for machine learning, SoftwareX 12 (2020) 100591.

[23] M.K. Hu, Visual pattern recognition by moment invariants, IRE Trans. Inf. Theory 8 (1962) 179-187.

[24] A. Ishak, A. Hussain, M. Mustafa, Weed image classification using Gabor wavelet and gradient field distribution, Comput. Electron. Agric. 66 (2009) 53-61, https://doi.org/10.1016/j.compag.2008.12.003.

[25] A. Jain, G. Healey, A multiscale representation including opponent color features for texture recognition, IEEE Trans. Image Process. 7 (1998) 124-128, https://doi.org/10.1109/83.650858.

[26] A. Jain, D. Zongker, Feature selection: evaluation, application, and small sample performance, IEEE Trans. Pattern Anal. Mach. Intell. 19 (1997) 153-158, https://doi.org/10.1109/34.574797.

[27] B.T. Koot, Weed detection with Unmanned Aerial Vehicles in agricultural systems, Ph.D. thesis, Wageningen University and Research Centre, 2014. Registration number: 891024465110.

[28] M.G. Larese, R. Namías, R.M. Craviotto, M.R. Arango, C. Gallo, P.M. Granitto, Automatic classification of legumes using leaf vein image features, Pattern Recognit. 47 (2014) 158-168.

[29] E.L. Lawler, D.E. Wood, Branch-and-bound methods: a survey, Oper. Res. 14 (1966) 699-719.

[30] F. Lin, D. Zhang, Y. Huang, X. Wang, X. Chen, Detection of corn and weed species by the combination of spectral, shape and textural features, Sustainability 9 (2017), https://doi.org/10.3390/su9081335.

[31] B. Liu, R. Li, H. Li, G. You, S. Yan, Q. Tong, Crop/weed discrimination using a field imaging spectrometer system, Sensors 19 (2019) 5154, https://doi.org/10.3390/s19235154.

[32] P. Lottes, M. Hoeferlin, S. Sander, M. Müter, P. Schulze, L.C. Stachniss, An effective classification system for separating sugar beets and weeds for precision farming applications, in: 2016 IEEE International Conference on Robotics and Automation (ICRA), 2016, pp. 5157-5163.

[33] M. Louargant, G. Jones, R. Faroux, J.N. Paoli, T. Maillot, C. Gée, S. Villette, Unsupervised classification algorithm for early weed detection in row-crops by combining spatial and spectral information, Remote Sens. 10 (2018), https://doi.org/10.3390/rs10050761.

[34] K. McGarigal, FRAGSTATS: Spatial Pattern Analysis Program for Quantifying Landscape Structure, vol. 351, US Department of Agriculture, Forest Service, Pacific Northwest Research Station, 1995.

[35] F. Mekhalfa, F. Yacef, Supervised learning for crop/weed classification based on color and texture features, arXiv preprint, arXiv:2106.10581, 2021.

[36] A. Monteiro, A. Von Wangenheim, P. Júnior, Weed Mapping on Aerial Images - a Systematic Literature Review, Technical Report INCoD.LAPIX.01.2019.E, Federal University of Santa Catarina, 2019.

[37] F. Mostajer Kheirkhah, H. Asghari, Plant leaf classification using gist texture features, IET Comput. Vis. 13 (2019) 369-375.

[38] V. Pászto, A. Brychtová, L. Marek, On shape metrics in cartographic generalization: a case study of the building footprint geometry, in: Modern Trends in Cartography, Springer, 2015, pp. 397-407.

[39] F. Perez-Sanz, P.J. Navarro, M. Egea-Cortines, Plant phenomics: an overview of image acquisition technologies and image data analysis algorithms, GigaScience 6 (2017), https://doi.org/10.1093/gigascience/gix092.

[40] A. Pérez, F. López, J. Benlloch, S. Christensen, Colour and shape analysis techniques for weed detection in cereal fields, Comput. Electron. Agric. 25 (2000) 197-212, https://doi.org/10.1016/S0168-1699(99)00068-X.

[41] R. Regnier, G. Avrin, V. Barbosa, D. Boffety, A. Kalouguine, S. Lardy-Fontan, Validation de méthodologies d'évaluation de solutions de désherbage autonomes, dans le cadre des projets Challenge ROSE et METRICS, Rev. Ouverte Intell. Artif. 2 (2021) 11-32, https://doi.org/10.5802/roia.8.

[42] M.B.H. Rhouma, J. Žunić, M.C. Younis, Moment invariants for multi-component shapes with applications to leaf classification, Comput. Electron. Agric. 142 (2017) 326-337.

[43] C. Rojas, L. Guzmán, N. Velasco Toledo, Weed recognition by svm texture feature classification in outdoor vegetable crop images, Ing. Investig. 37 (2017) 68, https://doi.org/10.15446/ing.investig.v37n1.54703.

[44] D. Saha, G. Hamer, J.Y. Lee, Development of inter-leaves weed and plant regions identification algorithm using histogram of oriented gradient and k-means clustering, in: Proceedings of the International Conference on Research in Adaptive and Convergent Systems, 2017, pp. 136-139.

[45] K. Thyagharajan, I.K. Raji, A review of visual descriptors and classification techniques used in leaf species identification, Arch. Comput. Methods Eng. 26 (2019) 933-960.
[46] D.G. Tsolakidis, D.I. Kosmopoulos, G. Papadourakis, Plant leaf recognition using Zernike moments and histogram of oriented gradients, in: Hellenic Conference on Artificial Intelligence, Springer, 2014, pp. 406–417.

[47] J.A. Vayssade, R. Arquet, M. Bonneau, Automatic activity tracking of goats using drone camera, Comput. Electron. Agric. 162 (2019) 767–772.

[48] J.A. Vayssade, G. Jones, C. Gée, J.N. Paoli, Pixelwise instance segmentation of leaves in dense foliage, Comput. Electron. Agric. 195 (2022) 106797, https://doi.org/10.1016/j.compag.2022.106797.

[49] J.A. Vayssade, G. Jones, J.N. Paoli, C. Gée, Two-step multi-spectral registration via key-point detector and gradient similarity. Application to agronomic scenes for proxy-sensing, in: Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, La Valette, Malta, 2020, https://hal-agrosup-dijon.archives-ouvertes.fr/hal-02499730.

[50] J.A. Vayssade, J.N. Paoli, C. Gée, G. Jones, Deepindices: remote sensing indices based on approximation of functions through deep-learning, application to uncalibrated vegetation images, Remote Sens. 13 (2021), https://doi.org/10.3390/rs13122261.

[51] H. Waghmare, R. Kokare, Y. Dandawate, Detection and classification of diseases of grape plant using opposite colour local binary pattern feature and machine learning for automated decision support system, in: 2016 3rd International Conference on Signal Processing and Integrated Networks (SPIN), 2016, pp. 513–518.

[52] A. Wang, W. Zhang, X. Wei, A review on weed detection using ground-based machine vision and image processing techniques, Comput. Electron. Agric. 158 (2019) 226–240, https://doi.org/10.1016/j.compag.2019.02.005.

[53] S. Watcharabutsarakham, W. Sinthupinyo, K. Kiratiratanapruk, Leaf classification using structure features and support vector machines, in: 2012 6th International Conference on New Trends in Information Science, Service Science and Data Mining (ISSDM2012), IEEE, 2012, pp. 697–700.

[54] W. Wong, C. Ali, K. Brandon, C.C. Wee, M. Marriapan, Co-occurrence matrix with neural network classifier for weed species classification: a comparison between direct application of co-occurrence matrix (glcm) and Haralick features as inputs, Int. J. Enhanc. Res. Sci. Eng. 2 (2013) 1–6.

[55] Y. Xu, R. He, Z. Gao, C. Li, Y. Zhai, Y. Jiao, Weed density detection method based on absolute feature corner points in field, Agronomy 10 (2020), https://doi.org/10.3390/agronomy10010113.

[56] S.A. Zamani, Y. Baleghi, Early/late fusion structures with optimized feature selection for weed detection using visible and thermal images of paddy fields, Precis. Agric. 24 (2023) 482–510, https://doi.org/10.1007/s11119-022-09954-8.