# Foundations and state of the art

Nathalie Mitton, Ludovic Brossard, Tassadit Bouadi, Frédérick Garcia, Romain Gautron, Nadine Hilgert, Dino Ienco, Christine Largouët, Evelyne Lutton, Véronique Masson, et al.

► To cite this version:

Nathalie Mitton, Ludovic Brossard, Tassadit Bouadi, Frédérick Garcia, Romain Gautron, et al. Foundations and state of the art. Agriculture and Digital Technology: Getting the most out of digital technology to contribute to the transition to sustainable agriculture and food systems, 6, INRIA, pp.30-75, 2022, White book Inrira. hal-03609470v2

HAL Id: hal-03609470

https://hal.inrae.fr/hal-03609470v2

Submitted on 13 May 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
3

Foundations
and state of play

**Authors** – Nathalie Miton, Ludovic Brossard, Tassadit Bouadi, Frédérick Garcia, Romain Gautron, Nadine Hilgert, Dino Ienco, Christine Largouët, Evelyne Lutton, Véronique Masson, Roger Martin-Clouaire, Marie-Laure Mugnier, Pascal Neveu, Philippe Preux, Hélène Raynal, Catherine Rousset, Alexandre Termier, Véronique Bellon-Maurel.

**Acknowledgements (contribution, proofreading, editing)** – Isabelle Piot-Lepetit.

<figure><img src="image_2.png" /></figure>
Having described the different challenges facing agriculture in the previous chapter, in particular those of agroecology and sustainable food systems, which will be the “target” of our reflections, in this chapter we will address the foundational aspects of digital technology, their use in agriculture and current research. In the Introduction, we reviewed the pillars of digital agriculture, which can be summarised as data, processing capacities, connectivity to allow data and information exchange and, finally, automation. The challenges facing agriculture concern all levels of the data cycle, from capture to use via collection, traceability, processing, storage, interpretation, provision and application in automatic and robotic systems.

<figure><img src="image_3.png" /></figure>

# 3.1 Data

The use of digital technology in agriculture produces large volumes of highly heterogeneous data that can be qualified as “big data” (Bellon-Maurel et al., 2018). It is uniquely complex because it includes observations of complex objects and environments of different natures and operating at very different spatio-temporal levels (for example from the gene to the field), with strong intra- and inter-level interactions and involving numerous actors. This complexity leads to questions about what data to collect (nature, frequency, objective, etc.) to guide the deployment of a technical solution at all levels (hardware, software, interface, etc.).

## Data capture (what, why, where and how)

The challenges of data capture are both hardware- and software-related. Knowing what the data is intended for helps to determine the choice of measurement equipment.

First, the nature of the measurement (temperature, air or soil humidity, condition of a plant's leaves, weight of an animal, etc.) and the accuracy required must be specified. These requirements, which depend on the needs defined, vary greatly from one use to another. The second issue is how to capture the data. The nature, size, weight, bulk and robustness of the sensor will also depend on the nature of the measurement, the object to which it is applied and the environment in which it will be placed: a sensor worn by an animal will be chosen according to the weight and bulk of the equipment and the size of the animal. Similarly, a sensor for field measurements on soil or plants will require protection to make it resistant to the surrounding environment (humidity, temperature variations, shock resistance, etc.). Finally, how the data will be used will define the sampling method, in particular the collection location and spatial and temporal resolution (Brun-Laguna et al., 2018). For example, should the sensors be positioned per m²
or per km²? If the aim is to monitor animals, should all or just a few of the animals
in the herd be equipped (Jabbar et al., 2017)? What time frequency is required
and should it be constant? Some applications require high spatial and temporal
frequency, offered by satellite remote sensing. Others need less frequent mea-
surements, such as those obtained from participatory data (Minet et al., 2017).

The decision of which technology and equipment to use and which
methodologies to implement for the deployment of sensors has been the subject
of numerous studies in recent decades, with applications in both cropping and
livestock production: identification and geolocation using RFID (Ruiz-Garcia and
Lunadei, 2011) or GPS; imagery (2D, 3D, infrared, hyperspectral), accelerometery,
acoustics, biochemical measurements on fluids (including biomarkers),
measurement robots such as weighing scales, water or milk meters, feed dispensers,
etc. (Chastant-Maillard and Saint-Dizier, 2016; Halachmi et al., 2019). In most
cases, trade-offs must be made between cost, resolution, precision and practicality
(Foubert and Mitton, 2019). Research aims to limit these concessions, either by
developing sensors that are increasingly precise, energy-efficient, smaller, less
intrusive and less costly, or by designing massive data acquisition devices (using
satellite images, drones, etc.). The deployment of new satellite constellations
(Sentinel-2),<fnref n="28" /> which produce high-resolution images (both spatial and temporal)
made available free of charge, offers new monitoring opportunities.

In conclusion, the work to develop acquisition systems is inherently multidisci-
plinary and requires collaboration between agronomists, biologists, zootechnicians,
geneticists, computer scientists, electronic engineers and end users to ensure
that the requirements of users (who are sometimes researchers themselves in
another field) are met and to combine knowledge of the objects of study, their
specificities and their constraints with knowledge of digital technology.

<footnote n="28">28. https://sentinel.esa.int/web/sentinel/missions/sentinel-2</footnote>
<figure><img src="image_4.png" /></figure>

For example...

Several INRAE units are developing such acquisition devices for phenotyping or monitoring animals or crops. Examples include: a high-speed 3D image acquisition device and its associated processing methods to measure the physical condition and morphology of dairy cows, developed by the **PEGASE** Mixed Research Unit (UMR) in collaboration with the Agricultural Technical Institute **IDELE** and the company **3D Ouest**. An automatic feeder has been developed by the **PEAT** Experimental Unit (UE) and the **BOA** UMR for studying the feed intake and individual feeding behaviour of poultry reared in groups. The electronic mounting detector "ALPHA" (the company Wallace), based on an automatic RFID reader worn by a ram, was designed by the **SELMET** UMR to automatically detect heat in sheep, particularly in large-scale farming. In the plant field, the **ITAP** UMR and the **CAPTE** Mixed Technology Unit (UMT) are developing optical sensors for phenotyping or early detection of plant diseases. The **TETIS** UMR uses satellite remote sensing to detect plot defects. The acquisition of phenotypic data using sensors is being addressed by large-scale programmes and infrastructures such as **PHENOME** on the characterisation of crops grown in greenhouses and in the field and **IN SYLVA** on forests. The resulting data can be used to improve the predictive capacities of models and how they take into account interactions between genotypes and the environment. More broadly, high-speed phenotyping systems are also being developed in plant and animal experimental units at INRAE.

Data collection and transmission (What data to send, when and how)

Once the data has been acquired, it must be transmitted. Some systems use wired communication (Ethernet, serial, etc.), but this is not always possible and sensors have to be equipped for wireless communication, which poses different challenges. Data capture and transmission in agriculture increasingly use Internet of Things technology (Zhao et al., 2010), especially RFID and wireless sensor networks with specific features for agriculture.
<figure><img src="image_5.png" /><figcaption>Monitoring of animals in extensive breeding. © Selmet - CIRAD.</figcaption></figure>

Most wireless sensors rely on energy sources that are either limited (e.g. batteries) and/or variable (e.g. via a solar collector) and must therefore be preserved. Data transmission is often the most energy-consuming factor and presents a major challenge, so the aim is to limit the amount of data sent while maintaining the data accuracy required for the application to function. Research therefore focuses on data processing in the sensor, which itself is limited in computing and memory capacity, using spatial and/or temporal data aggregation (Salim et al., 2020) and simplified artificial intelligence methods. For example, researchers use the correlation between two quantities (such as temperature and humidity) to transmit just one of the two values and interpret the second. Another option is to locally predict the next value to be measured and only transmit the data if it does not match the predicted value. The more demanding an application is in terms of temporal resolution or accuracy, the more data transfer is required. This also requires a trade-off between efficiency, accuracy and cost.
<figure><img src="image_6.png" /></figure>

For example...

The **FUN** and **EVA** project-teams at Inria are working on data collection for agriculture using wireless sensor networks. Their work concerns both specific network protocols and the question of which data to transmit to avoid saturating the communication media and reduce the energy consumption of transmissions. The **FUN** project-team is installing sensors in vineyards in South Africa to improve watering and water management. They also collaborate with Sencrop, which uses sensors in cereal and potato farming. **EVA** installs sensors on peach trees in Argentina to protect against frost.

The choice of communication technology depends on the quantity of data to be retrieved as well as the distance to cover and location of the sensors. For data to be collected over long distance and requiring greater intervals between transmission (such as once-a-day temperature readings) one would uses long-range technology with low data rate and power consumption, whereas high-frequency readings (such as animal video tracking) require high data rate. The measurement points may be located in areas not covered by cellular technology (such as 3G/4G/5G or LPWAN – Low Power Wide Area Network), which would require specific network mechanisms to be put in place such as routing (relaying information to the destination station). This must take account of the constraints and requirements of the applications and the material limitations and characteristics of existing radio technology (Foubert and Mitton, 2021) and the environment in which the sensors are deployed (Ferreira et al., 2020). An additional difficulty is the heterogeneity of technologies required that must to coexist and sometimes cooperate, as well as the more general challenges of the Internet of Things (IoT), addressed in the Inria White Paper on the Internet of Things.<fnref n="29" />

Lastly, mobile data collection solutions are emerging for blackspots, ranging from simplified solutions (portable data devices that can be carried in a rucksack, as in the case of the COWSHED project in Africa<fnref n="30" />) to high-tech solutions with "aerial" devices (drones or nanosatellites). The latter can collect data from thousands of connected objects at a low data rate (LoRa protocol) or high data rate (i.e. 100 kB per transmission) using a smaller number of terminals on the

<footnote n="29">29. Scientific challenges of the Internet of Things. Inria White Paper. https://www.inria.fr/en/inria-white-paper-internet-things-iot</footnote>

<footnote n="30">30. https://hal.archives-ouvertes.fr/hal-03102190/document</footnote>
ground (around 100) (UHF protocol). Applications are developing in the field of agriculture, such as in Australia where farmers remotely monitor the level of irrigation tanks using nanosatellites.<fnref n="31" />

<figure><img src="image_7.png" /></figure>

## Definition

The Internet of Things (IoT) is the interconnection of the Internet with things, places and physical environments. The term refers to a growing number of objects that are connected to the Internet, allowing communication between the physical and digital aspects of our possessions. The IoT combines a wide range of technologies from simple RFID tags to mobile phone applications and wireless sensor networks. Radio communication technologies are diverse with different characteristics with regard to data rate, consumption, range, etc. Sensors can be equipped with microcontrollers with varying levels of power and energy consumption.

## Data storage and exchange, traceability

Once the data has been captured and transmitted, it can be used for a variety of purposes. Firstly, it can be stored and processed to extract knowledge, anticipate malfunctions, etc. This data can be very heterogeneous and of varying levels of quality. It can also have very different sampling rates due to it coming from different sources (physical sensors, "human" sensors or even simulation results) and can be very large in volume (many capture points, potentially high temporal frequency). Methods derived from multivariate data management and now big data offer a response to the challenges of volume, processing speed and the diversity of formats and sources (Bellon-Maurel et al., 2018). The prerequisite for successfully using this data is that it meets the guiding principles of "FAIR", which are Findability, Accessibility, Interoperability and Reusability,<fnref n="32" /> with minimal human intervention. There is therefore a demand for a new generation of information systems adapted to agriculture in order to manage and structure this complex mass of data using the FAIR principles. Metadata and data must be well described using semantic resources (ontology, taxonomy, thesaurus) to make them understandable and facilitate access via standardised protocols.

<footnote n="31">31. https://which-50.com/world-first-australian-iot-uses-satellites-to-monitor-farmers-tanks-rain-levels/</footnote>

<footnote n="32">32. Wilkinson et al. *The FAIR Guiding Principles for scientific data management and stewardship*. Scientific Data 3, 160018. doi:10.1038/sdata.2016.18.</footnote>
<figure><img src="image_8.png" /></figure>

For example...

At INRAE, automated data processing centres (CATI), federate and structure skills, methodologies and technologies to facilitate the reuse of data, such as the CATI SICPA (Information Systems and Computation for Animal Phenotyping), the CATI Codex (plant phenotyping) and the CATI GEDEOP (Management of Experimentation, Observation and Practice Data on Agro-socioecosystems).

Finally, the question of data validation is becoming a central issue with the increase in the quantities of data collected: is the measured value correct, for what applications and under what conditions?

A variety of DBMS (Database Management Systems) exist depending on the data model used, with the relational model currently being the most common. *Benchini et al.* (2007), for example, coupled a dynamic cropping system simulation model with a relational database to achieve efficient storage and analysis of model data at farm level. But when it comes to dealing with very large volumes (of the order of petabytes) or complex and heterogeneous data (graphs, documents, etc.) in highly distributed contexts (remote servers, Internet of Things, etc.), NoSQL-type databases using a different data model – which has fewer conceptual constraints than the relational model – are more pertinent.

In some cases, the primary aim is to share the data among multiple actors, for example to limit fraud or validate certain processes (correctly following a given route, compliance with the cold chain, local production or insecticide-free requirements). To achieve this, one of the most promising digital tools currently available is the blockchain (*Bermeo-Almeida et al.*, 2018). Blockchains, which use a distributed database that requires no control entity, date the components they contain and guarantee their unalterability. In agriculture, this technology makes it possible to record the life stages of a product and thus ensure traceability (*Kamilaris et al.*, 2019). It offers multiple advantages (see Chapter 4: Opportunities) including transparency and records of transactions between farmers, suppliers, buyers, consumers etc. In some sectors, using blockchains makes it possible to avoid long and costly certification processes (*Lin et al.*, 2017).

However, blockchains also pose new digital and organisational challenges. In any information system, there is always an inherent risk of hacking. Also, blockchain technology was initially designed for transferring and sharing intangible goods
(funds, certificates, diplomas) and is not infallible in cases where information flows must be coupled with physical flows, such as in agriculture and agrifood where the digital data must be an exact representation of the physical flow. Furthermore, the perceived security of a blockchain is based on a mechanism known as "proof of work" (of computer operations), which validates the new blocks to be incorporated. Public blockchains rely on large numbers of "proofs of work", which is very energy intensive, and research currently focuses on reducing the complexity of these cryptographic algorithms to reduce their energy consumption.

## 3.2 Modelling, simulation and optimisation

If data is one of the driving forces of digital agriculture, modelling is also essential for linking measurements and observations to interpretations and recommendations to help actors in the agricultural sector better understand, manage and improve their production systems.

In the field of agriculture and agronomic research, the scientific approach of modelling to predict harvests emerged as early on as the reign of Egyptian Pharaoh Sesostris I, when river levels were used to make crop forecasts (Gros de Belet, 1998), or among Inca farmers who knew several months in advance what agricultural cycles to expect by observing nature (Gutiérrez, 2008). Much later, the pioneering work of Mendel (Mendel, 1907) and then Fisher (Street, 1990) definitively legitimised the use of statistical models in the fields of genetics and agronomy. In the latter half of the 20th century, agricultural modelling developed in particular in rural economies to rationalise and optimise production, agronomy and zootechnics, crop management, animal nutrition and genetic selection in plants or animals. With the development of computers and the first calculators, modelling gradually went beyond statistics and operations research and increasingly used symbolic and algorithmic formalisms to produce models expressed in mathematical and computer terms and in which simulation plays a key role.

The general function of a model is often called a mediation function: "to an observer B, an object A* is a model of an object A to the extent that B can use A* to answer questions that interest him about A" (Minsky, 1965). This mediation can help meet different cognitive objectives: facilitating experience, intelligible formulation, theorisation, communication and the coconstruction of knowledge, decision and action (Varenne and Silberstein, 2013). Today, agricultural modelling concerns a very broad spectrum of objects and has four main purposes: analysis, communication, predicting and controlling the evolution of various components in an agricultural system and designing and optimising the system under consideration. In the rest of this section, we will present a few of the major types of models and how they can be used in digital agriculture thanks to simulation and optimisation.
## What to model, for what purposes and with which tools

**What to model?** – In agriculture, the objects of study – or subjects of the model – are anthropised natural systems that can involve multiple scales and levels of organisation. Modelling focuses on the components of these systems as well as the processes that govern their dynamics, the events that activate or inhibit these processes and the exogenous factors that influence them (such as weather conditions) (Martin et al., 2011). Some of the components are biophysical (such as crops with growth processes, diseases) (Kumar and Sinhg, 2003) while others are centred on the roles played by human actors. In the latter case, modelling can concern either a single individual (Martin-Clouaire and Rellier, 2004; 2009) or a group of individuals (such as the members of a cooperative) and the social processes of activity coordination carried out by different individuals in the collective (Drewniak et al., 2013; Manson et al., 2016).

<figure><img src="image_9.png" /></figure>

### For example...

Models of processes, flows and interactions are being developed by many teams and units. Below are a few examples at different scales.

At INRAE, several teams are working on crop or livestock modelling, whether on a plot-by plot basis (multispecies crop models such as STICS, which describe growth according to climate and environmental variables), at the individual level (animal growth models according to their diet and environment), or at larger scales (plant epidemiology models including inter-plot dispersal and animal epidemiology models describing inter-herd transmissions, etc.). These models are hosted by modelling platforms such as **RECORD**, **OPEN ALEA** and **OPEN FLUID** and form the basis of simulations run according to different climate and contextual scenarios.

The **STEEP** project-team (Inria, CNRS, Université Grenoble Alpes) develops mathematical models for analysing material flows (production, transformation, exchanges, consumption, waste) in agriculture and the forestry-wood sector in order to 1) understand the upstream/downstream vulnerabilities in the sectors, 2) question the use of natural resources and potential problems caused by competition for use and, finally, 3) assess environmental impacts. The tools developed are based on the modelling of chains in terms of products and sectors and the existing flows between them. One of the major difficulties is the particularly patchy and inconsistent nature of the data.
The **EASE** project-team (Inria, Ecole Nationale Supérieure Mines-Télécom Atlantique Bretagne Pays de la Loire, Université de Rennes 1) is developing a complete series of new interaction models, offering tools for augmenting and describing information from complex systems. The work has been applied to energy management in agriculture. In particular, the model helps define how to reduce the environmental impact of energy consumption when optimising an existing site or installing new facilities. It has shown that the optimisation of a single parameter alone (local production, storage or process transfer) is not enough to maximise self-consumption and minimise energy requirements.

**What purposes to model for?** – The level of detail of the modelling systems at the heart of a study is defined by the purpose of the study and the tool under consideration. The most commonly applied objectives range from identifying the aims and means of managing agroecosystems to predicting performance (Rio et al., 2019) in the light of different scenarios going through the identification of risks and the critical analysis of the functioning and conduct of agricultural production systems (Li et al., 2019). Modelling can also allow the design of new systems such as the configuration and sizing of a logistics chain (Taghikhah et al., 2021).

<figure><img src="image_10.png" /></figure>

For example...

Many models concern agroecosystem management. The joint **INRIA / INRAE** project-team **BIOCORE** (CNRS, Sorbonne Université-UPMC) focuses on modelling and control in epidemiology for tropical agriculture. At INRAE, the **MIAT UR** and **MISTEA** UMR develop simulation models and optimisation methods for managing agroecosystems at farm level. For animals, UMRs such as **BIOEPAR**, **SELMET**, **MoSAR**, **UMRH** and **PEGASE** are developing models on animal health and epidemiology, dynamic ingestive, digestive and metabolic phenomena and livestock farming systems. For example, the **PEGASE** UMR has developed new models for adjusting daily feed according to the nutritional needs of each animal for individualised feeding for pregnant and lactating sows (Gauthier et al., 2019).
<figure><img src="image_11.png" /><figcaption>Building an agronomic model. © INRAE</figcaption></figure>

**How to model?** – Computer modelling to support the analysis, design and management of agroecosystems is combined with approaches using either simulation or optimisation (Li et al., 2020). In dynamic simulation, the modelling of phenomena considered important for the objectives of the study is centred on realism (Kaghazchi et al., 2021), whereas optimisation involves algorithmic exploration of the space of alternatives that efficiently searches for an optimal solution according to one or more explicitly formulated criteria using reductionist mathematical models (Ezanno et al., 2020; Casagli et al., 2020). The two methods have relatively antagonistic objectives (modelling realism versus computational efficiency) and therefore generally use different modelling approaches.

## Representation frameworks

Agroecosystems are complex objects of which the models concern, on the one hand, the functioning of the biophysical entities that compose them (soil, plants, animals, mineral and water flows, etc.) and, on the other hand, human decision making and action on these biophysical entities (Zabala et al., 2021). Models convey knowledge that mainly comes from scientific disciplines such as agronomy, zootechnics, environmental science, management science and the humanities.

Biophysical models can be classified into three main fields: mechanistic, empirical and hybrid (Reyniers, 1996). Mechanistic modelling focuses on events, causal relationships and processes, whereas empirical models treat systems as
"black boxes" and only generally describe the underlying biophysical phenomena. These models represent the input-output dynamics of a system component in terms of observation data. In reality, there are few truly mechanistic or empirical models. Models are generally hybrids or classified in one or the other category according to whether they possess mainly mechanistic or empirical components. The overall understanding and level of information required to build these models increases as we move from empirical to mechanistic models. By making causality explicit, mechanistic models can be more complex, while empirical models are generally simpler but have a more limited scope of application due to statistical data availability issues.

Decision modelling varies depending on the modeller's hypothesis about the decision maker. In a first type of hypothesis, the decision maker is assumed to be perfectly rational (in the economist's sense) and, when making a decision, determines the mathematically optimal choice according to theoretically defined functions of utility. In a second type of hypothesis, known as a bounded rationality hypothesis, the agent makes a decision that leads to an outcome that they consider satisfactory given the information available and their level of aspiration. Mental models of varying levels of sophistication are often used, including models based on decision rules that associate situations with decisions or actions (Martin-Clouaire, 2017). To facilitate and standardise the development of these models, ontologies can be used to define the concepts, relationships and other distinctions relevant to the areas concerned (major crops, livestock production, etc.) (Roussey et al., 2011). An ontology (see Section 3.4) is an abstract model (metamodel) of the area and provides the representation primitives allowing the instantiation of models for specific systems in the form of knowledge bases (Martin-Clouaire and Rellier, 2004; Fishwick, 2007).

<figure><img src="image_12.png" /></figure>

## Definition

An ontology defines a vocabulary and the semantic links between the elements of the vocabulary. The vocabulary is composed of names of concepts (or "classes"), which are types of entities known by the system and names of possible relationships (or "roles") between these entities (for example, the relationship of "pest" links two "living organism"-type entities). The ontology is described in a logical computer language that expresses the representation of knowledge to different degrees of expression. It can range from a simple taxonomy (a set of concepts structured by progressive specialisation) to complex descriptions of vocabulary elements and their semantic links. The language used allows the implementation of automatic reasoning.
**Discrete event, discrete time and continuous time systems** – A discrete event simulation model allows the representation of a dynamic system using variables whose evolution depends entirely on the occurrence of asynchronous events over time. One particular case is when the time progression is in fixed increments. The discrete event approach contrasts with (but does not oppose) the “system dynamics” approach in which the state of the system is modified continuously over time based on a set of differential equations defining the rates of change and state variables. In each of these different cases, researchers are interested in the representation of causal relations (i.e., for the biophysical aspect, mechanistic models). One of the best known formalisms is Discrete Event System Specification – DEVS (Zeigler *et al.*, 2000), which is based on a generic framework allowing different adaptations to specific formalisms such as Petri nets, cellular automata and, more generally, models with fixed time intervals. Petri nets are a particularly popular mathematical formalism because of their ability to represent the synchronisation of processes running in parallel and offer possibilities for rigorous model analysis. A cellular automaton is built using a network of discrete cells and is well suited to representing spatial dynamics (such as the propagation of an infestation) and self-organising phenomena (such as the landscape dynamics of natural reforestation). Some formalisms such as statecharts (Léger and Naud, 2009), timed automata (Hélias *et al.*, 2008) and Petri nets (Guan *et al.*, 2008) can also suit processes for verifying the behaviour of the model (e.g. to ensure that it cannot lock up) or its temporal properties.

<figure><img src="image_13.png" /></figure>

Automatic detection of mangoes by artificial intelligence from a smartphone photo. Hortsys.

© Hortsys - CIRAD.
**Individual- or agent-based models** – These models focus on systems that can be broken down into a set of entities (such as plants, animals, zones) that act or interact (Daudé, 2004; DeAngelis and Diaz, 2019). When combined with a cellular automaton-based approach, the individual-based approach allows the spatial representation and simulation of biophysical processes on a territory divided into plots. When the modelled entities acquire more elaborate cognitive and decision-making capacities (Bahri et al., 2020), we speak of agent models that allow, for example, simulation of the decision-making behaviour of a group of agents (such as farmers) operating in a given territory (Huber et al., 2018). Farm management has often been modelled using simple mechanisms for triggering decision-making rules associated with possible situations. With this approach, however, it has proved difficult to control the order in which rules are used and to maintain the rule base once it reaches a certain size. An improvement was introduced by the BDI (Belief, Desire, Intention) approach (Georgeff et al., 1970; Bratman, 1987), which makes it possible to model the process by which an agent makes decisions based on a perception of the current situation (Belief), the declared objectives (Desire) and decisions on how to proceed toward the objectives (Intention).

<figure><img src="image_14.png" /></figure>

For example...

The **INRAE-MIAT** UR has developed several formalisms to represent and simulate the decision-making behaviour of farmers when managing their farms using the BDI approach, temporal planning and uncertainty in artificial intelligence. For example, Martin-Clouaire and Rellier address the problem of production management as one of coordinating a set of activities organised in flexible plans for which it is possible to simulate the implementation in a particular context (Martin-Clouaire and Rellier, 2009). For application examples in dairy farming see Martin et al. (2011) and Martin-Clouaire et al. (2016) in viticulture.

**Constraint-based models** – Constraint-based models use varied range of formalisms that are mainly based on the concept of graphs modelling binary relationships between variables (Hurley et al., 2016). These relationships can model correlations and causal influences, whether deterministic or probabilistic, as in the case of Bayesian networks and Markov chains. These networks can also describe constraints between variables in terms of combinations of acceptable or unacceptable values, leading to a Constraint Satisfaction Problem (CSP)
(Moummadi et al., 2011). In a similar vein, linear programming methods are based on the optimisation of a linear combination of multiple variables connected by linear relationships called constraints (Maqrot et al., 2017).

<figure><img src="image_15.png" /></figure>

For example...

At INRAE, the **BAGAP** UMR works on modelling the problem of dynamic crop allocation on a farm, based on the use of spatial and temporal constraints and the toolbar2 solver (Akplogan et al., 2013). For example, the team analysed the wooded countryside of the Charolais-Brionnais region to show the uniqueness of this landscape and its suitability for the different structures and functions of hedges. Thanks to this analysis, the countryside was added to the list of potential sites for submission to UNESCO for heritage protection.

Modelling and simulation

The primary advantage of modelling approaches is no doubt the ability to model and simulate complex behaviours in agricultural systems and, more broadly, socioecological systems such as agroecosystems (Peart and Curry, 1998). Models, especially agent-based ones, are often complex due to the number and heterogeneity of components and interactions and their sensitivity to variations affecting the systems. Their behaviour is difficult to study because the phenomena involved are non-linear with multiple discontinuities and feedback between levels of organisation and scales. Some of these models represent cognitive agents with bounded rationality behaviour. Numerous agricultural applications have been developed based on the CORMAS (Bommel et al., 2015) and GAMA (Taillandier et al., 2010) platforms, such as for studying water management, the reform of the Common Agricultural Policy, reducing the use of pesticides and developing organic farming.

<figure><img src="image_16.png" /></figure>

For example...

At INRAE, the **AGROECOLOGIE** UMR coordinates the development of the MAELIA platform for the integrated modelling and evaluation of socio-agroecological systems. It aims to produce knowledge on the structure, functioning and performance of these forms of agriculture at plot, countryside and/or territorial level.
In practice, modelling-simulation approaches offer a variety of uses, ranging from laboratory analysis by scientists, decision support (Huber et al., 2018), real-time decision making by farmers or agricultural advisors and support for negotiations between stakeholders (e.g. support models for joint water management in a territory) to the co-design of new production systems by a group of farmers and training. Individual or groups of farmers can thus improve their understanding of biophysical functioning and obtain ideas for improving the system studied in terms of product quality, system vulnerability, environmental consequences of the practices implemented, reduction of work overload and drudgery and, finally, economic performance linked to the application of agroecological principles.

<figure><img src="image_17.png" /></figure>

For example...

At INRAE, skills are grouped in **CATIs** for modelling large-scale systems, such as the **IMOTEP CATI** (Information, Models and Data Processing in Epidemiology and Population Dynamics) and the **IUMAN CATI** (Computerisation and Use of Models for Digital Agroecosystems). The work covers both modelling of the spread of epidemics in plants or animals and software development for platforms and proofs of concept allowing the sharing and computerisation of these new models at multiple scales.

Modelling and optimisation

By definition, optimisation explores possible solutions to a given problem using different methods to find an optimum or optima according to a criterion or set of criteria (Zelinka et al., 2013). It is used in different areas of agriculture and at different scales (Plà-Aragonés, 2015). At the farm level, optimisation is implemented either explicitly or implicitly, whether in feed formulation, herd management, animal slaughter planning, crop and land use planning or water management. It is also used at different scales, including groups of farms, territories, regions and countries, for managing land use, water and economic trade and market issues (Carpentier et al., 2015). In these cases, bioeconomic models are employed according to an analytical approach, in which the primary objective is to evaluate the impact of the applying constraints and criteria to optimal solutions.

Due to the complexity of agricultural systems and changes in questions relating to agriculture, optimisation has also evolved in agriculture (Jones et al., 2016). The early economic models of the 1950s focussed above all on maximising income.
Today, the formulation of low-cost feed still primarily aims to obtain the cheapest feed possible while meeting nutritional criteria. Optimisation has gradually become multi-objective to combine different aims: productive (e.g. animal or plant production, working time), economic (e.g. income, cost) or environmental (nutrient levels, environmental impact calculated by life cycle analysis, ecosystem services, etc.). In "constrained" optimisations, the constraints are also varied and can be biological, structural, regulatory, environmental or linked to decision-making.

Model optimisation in agriculture has also benefited from developments in optimisation methods, using a diverse range of methods. Deterministic linear programming methods are still very common, with adaptations to solve multi-objective problems. Stochastic metaheuristic methods are applied alone or in combination with the previous ones. These metaheuristic methods make it possible to address multicriteria optimisation and obtain a set of optimal solutions considered admissible in the context (called Pareto Front); they include, for example, evolutionary algorithms (such as genetic algorithms or differential evolution) that work on a population of solutions, particle swarm optimisation, taboo search, simulated annealing, etc. (Kaim et al., 2018; Memmah et al., 2015).

Current issues surrounding optimisation concern, in particular, how to adapt methods to increasingly complex models, and in particular how to take account of uncertainty (Crespo et al., 2010) and the temporal aspect in the formulation of the optimisation problem (Akplogan et al., 2013). These issues echo those traditionally addressed in the automatic control and optimal control community. Another major topic of research is the coupling between optimisation and simulation (Borodin, 2014), in particular in connection with reinforcement learning methods (Gosavi, 2015). Despite technological advances in computing power, the processing time of optimisation processes is still an important factor to be considered due to the increasing complexity of the models in question. Recent developments in metamodelling offer a possible simplification strategy to reduce these processing times.

<figure><img src="image_18.png" /></figure>

For example...

At INRAE, the **PEGASE** and **SMART-LERECO** UMRs develop multi-criteria optimisation approaches (zootechnical and economic performance, environmental impact) for feeding strategies in pig farming, based on a pig farm model.

At Inria, there are more than twenty project-teams working on the development of optimisation, operational research or control algorithms.
## 3.3 Multi-scale learning and knowledge extraction

The two previous subchapters presented the approaches used to collect data, followed by modelling techniques based mainly on human analysis. In this subchapter, we will focus on the main families of approaches to building models directly from data and thereby automatically extracting knowledge. The resultant knowledge can either be presented to human experts or remain within a learning context for prediction or identification tasks, for example.

We will first show that the “raw” data sent by the sensors cannot generally be used in its initial state and that its pre-processing represents a challenge in itself. Let’s start by presenting the types of data frequently used in digital agriculture, and which could constitute a Big Data.

### Massive data in agriculture

In agriculture, the most “massive” data comes from sensors with high temporal or spatial resolution, such as time series and remote sensing or mapping data from embedded sensors.

**Time series** – A time series is a sequence of numerical values representing the evolution of a variable measured on an individual over time. Such sequences of variables can be modelled individually to understand their past evolution and predict future behaviour using ARMA-type models (Box et al., 2015). Today, experiments in agronomy make it possible to observe the same variable on thousands of individuals (e.g. leaf area on thousands of plants in a greenhouse, the temperature of livestock) over long periods. The aim of analysing these time series has therefore evolved toward the search for common characteristics between series, major differences or the acquisition of more detailed knowledge about the internal (e.g. effect of genotypes) or external (e.g. linked to environmental variables) mechanisms that influence the observed variables. Time series are thus studied more generally as functions of time. Their data is also known as “functional” or “longitudinal” data.

**Remote sensing data** – Remote sensing data can be images of a given area, taken by satellite or by drones. Satellite images – which we will focus on next – can be recorded at different periods, these sequences constituting time series. They can also, for the same period, be taken from different satellites, each with a different radiometric content (i.e. radar information, optical information). Thanks to recent
space missions such as *Copernicus*, plant dynamics can now be monitored with a spatial resolution compatible with the size of the objects of interest and short revisit time intervals. The *Sentinel-1* satellite mission acquires radar information (two bands) every five or six days over the same area at a spatial resolution of ten metres. This source of data provides access to information on the structure of objects (i.e. forest or agricultural biomass) and makes it possible to monitor and assess wetland areas and the area of land that has been irrigated over a certain period. Another equally interesting satellite mission is *Sentinel-2*, which provides multispectral imaging information (thirteen bands), again delivered every five or six days and at a spatial resolution of ten metres. This optical sensor is particularly suitable for mapping land cover and land use, monitoring the biodiversity of natural states and for large-scale yield estimation over large areas (Lambert et al., 2018).

At the other end of the scale, at the microscopic level, metabarcoding<fnref n="33" /> metaomic data allows a better characterisation of the biological environment of crops or animals. This metadata is constructed by assembling the “fingerprints” of the genomes present or of their expressions (RNA, proteins), making it possible to analyse new dimensions of ecosystems, which can better explain the behaviour of crops or animals. We are still only at the beginning of exploring these new data sources, some of which remaining difficult to access (proteomics, metabolomics, etc.).

## Data pre-processing

The major challenges in data pre-processing are: i) identifying outliers or unreliable data: data collected during experiments or in the field is voluminous, very noisy and can be affected by errors from a variety of causes, such as a faulty sensor. Specific tools are therefore needed to annotate this data, rapidly detect faulty sensors and diagnose heterogeneity in the field or greenhouse to improve the quality of the data sets for future analyses; ii) linking data with expert knowledge, such as mimicking an expert's reasoning by an automaton when validating a "small" data set, or using the expert's knowledge to adjust curves (alignment of phenological stage dates).

One particular challenge is data fusion. Information that is difficult to obtain directly can be retrieved by combining data, whether of the same type (for example, the leaf area of a plant can be predicted from the analysis of

<footnote n="33">33. Metabarcoding is a method of identifying species from DNA or RNA segments. Instead of targeting different species, metabarcoding determines the composition of species in a sample, thereby allowing the identification of many taxa in an assembly of populations (of *bacteria* or other microorganisms) within an environmental sample (e.g. soil sample, *sediment*, *excrements*, etc.). It is thus one of the fastest methods for the *environmental assessment* of the biodiversity of ecological systems with a high number of unknown or difficult-to-identify species.</footnote>
fifteen images of this plant taken from different angles), or of different types. To do so, i.e. to monitor the same phenomenon or the same study area, an ever greater volume of heterogenous data of various types (called "multisource" data) is collected. The knowledge contained within this data is a real opportunity for improving our understanding of the complex phenomena associated with modern agricultural practices in order to better monitor and manage them. Within this general framework, one of the main challenges today is knowing how to make the best use of these heterogeneous and complementary sources of information to obtain the maximum amount of information (because, in the science of complexity, "the whole is greater than the sum of its parts"<fnref n="34" />). Depending on the typology of the sources involved in the process, two merging strategies can be used: early and late merging. In the first case, the data is combined at the beginning of the process to form a single new homogeneous dataset. This can be done, for example, by bringing all the available information to the same spatial or temporal resolution or unit of analysis. In this context, once the new dataset has been built, standard single-source analysis techniques can be used. In the second case (late merging), an analysis process is set up by each source specifically and the merge is carried out at the descriptor or decision-making level. For example, specific descriptors can be extracted from each source and then combined to exploit higher-level interactions between the different sources considered. Lastly, the different sources can be combined in what is known as an "end-to-end" process, in which the standard processing stages are replaced by a single system (usually a deep neural network) that takes the raw sources as input and returns the required decisions as output (Charvat et al., 2018; Plaisant, 2004; Tonda et al., 2018).

In the case of time series, merging series with different temporal resolutions is a major challenge, for example if the activity sensor in a collar worn by an animal sends information every five minutes, but the animal is only weighed once a day. In order to compare individuals, it may be necessary to interpolate the time series for the same time period (using linear or polynomial smoothing methods), possibly by matching similarities (dates of phenological stages, growth peaks, etc.). Dynamic Time Warping (Sakoe et al., 1978) is one of the well-known techniques for measuring similarities between two series. However, this technique does not provide answers to all the curve alignment issues encountered when dealing with living beings, when it is essential to take phenological time into account. These questions present challenges that are still largely unresolved in biology.

<footnote n="34">34. http://www.scilogs.fr/complexites/le-tout-est-il-plus-que-la-somme-des-parties/</footnote>
There are also methods able to extract several models from time series at different time scales and then select the most relevant ones using information theory approaches (principle of Minimum Description Length – MDL) (Vespier et al., 2012). The advantage of these approaches is that they allow us to focus on the temporal scale of the observed phenomena instead of the technical sampling value.

In the case of remote-sensing, with the explosion in the number of satellite missions (Sentinel, Spot, Pleiades and PleiadesNeo, PlanetScope, etc.), it is now possible to collect information describing a single study area at a lower cost in different spectral ranges (optical and radar) and at different spatiotemporal scales. This massive volume of multisource information requires new data management and analysis tools to be developed (Schmitt and Zhu, 2016). Typically, in a classical multisource fusion process for Earth observation data, sources are exploited through an early fusion process. For example, in the case of imagery at different spatial scales, a resampling stage is incorporated to bring all the images to the same spatial scale beforehand. Unfortunately, this type of process can introduce bias or error by generating new synthetic information. This is why late fusion approaches are now preferred wherever possible. Early examples in the context of land use mapping are starting to appear but we are still far from a generic solution that can be deployed systematically across different territories and adapted to different agricultural practices.

<figure><img src="image_19.png" /></figure>

For example...

At INRAE, the MISCA team of the TETIS UMR develop information management methods to meet the major societal challenges related to the environment, whether storing, managing, sharing or analysing large volumes of data. In particular, it contributes to soil mapping by applying Deep Learning techniques on very large datasets.

At Inria, several project-teams (GEOSTAT, TITANE, FLUMINANCE (Inria, INRAE, Université de Rennes 1), etc.) and the exploratory action AYANA are working on the analysis of satellite images.

In addition to purely satellite-based multisource information, other types of information are now combined with Earth observation data. For example, “spon- taneous” geolocation information or information from citizen science (Ienco et al., 2019) have much to offer to improve calibration and complement the purely physical information from satellite sensors.
## Supervised approaches

Supervised analysis consists mainly of two tasks: supervised classification and prediction of future values. Supervised classification consists in assigning, for a given time series and set of predetermined classes (e.g. "sick animal" and "healthy animal"), one of these classes to the time series. In practical terms, this can help determine the condition of an animal or plant from sensor data and information on the different conditions possible. Supervised classification methods need to be "trained"; to do this, they must be provided with a large number of correctly labelled examples indicating their class. Using these examples, the classification algorithm builds one (or more) model(s), to assign a class to an unlabelled time series according to its characteristics. The main differences between the major families of supervised classification approaches lie in the way the models are built. The simplest approaches, known as k-Nearest Neighbor (kNN), do not build a model but search for the k examples of the training set closest to the individual to be labelled, and return the majority label. The difficulty lies in choosing a suitable similarity method (Karlsson et al., 2016).

Finally, the very popular deep neural network methods can also be used to classify such data. The most successful method of this type is currently MLSTM-FCN (Karim et al., 2019), which combines a convolutional CNN (Convolutional Neural Network) block with a LSTM (Long Short Term Memory) block. The CNN block, widely used in image analysis, serves as a filter that traverses the time series or spectrum and extracts characteristic attributes at time t. It is combined with the LSTM block, which is widely used in the analysis of sequential data (especially text), and allows connections between past and present values to be made. This type of approach can produce excellent results (Kamilaris and Prenafeta-Boldú, 2018). However, it requires an even greater volume of labelled training data (which can be difficult to acquire in some agronomic contexts), and its parameters can be tricky to define (Zhu et al., 2017).

<figure><img src="image_20.png" /></figure>

### For example...

At Inria, the **STATIFY** project-team (Inria, CNRS, Institut Polytechnique de Grenoble) focuses on the statistical modelling of systems involving complex-structure data. The team develops statistical methods for capturing the variability of the systems studied while ensuring a good level of precision and taking into account extreme values that generally reflect rare phenomena. In particular, they model weather events for agroecology.
An example of the use of supervised classification methods in agriculture can be found in Fauvel *et al.* (2019), in which the authors work with precision breeding sensor data from dairy cows. The cows are equipped with thermometers and collars with an accelerometer. The time series of temperature and physical activity are analysed to allow more accurate oestrus detection than with existing methods or visual observation, even in the frequent cases where cows do not express any particular behaviour in the pre-oestrus phase (30%).

## Unsupervised approaches

Unsupervised approaches are used to reveal certain structures in data, whether groupings with clustering or recurring patterns with pattern mining.

**Clustering** – The aim of the clustering (or unsupervised classification) learning method is to identify relevant classes in the data. Data is grouped by similarity or proximity within each class. To achieve a good classification, it is necessary to minimise the intra-class inertia (to obtain homogeneous classes) and maximise the inter-class inertia (to obtain well differentiated classes). Two main families of methods are commonly used: i) hierarchical ascending classification (HAC), which seeks to group individuals iteratively, starting at the bottom (the two closest) and gradually building a tree, or dendrogram, to finally group all the individuals into a single class, at the root; ii) classification by dynamic reallocation (the k-means algorithm is a well-known example of this). The number k of classes is fixed a priori. After initialising k class centres, all individuals are assigned to the class whose centre is closest in the sense of the chosen distance. The algorithm then calculates the barycenters of these classes which become the new centres. The process (assignment of each individual to a centre, determination of the centres) is iterated until convergence to a fixed (local) minimum or maximum number of iterations.

The main issues to overcome when clustering multivariate data are identifying the "right" number of classes and defining a distance that is adapted to the data, sometimes implying the need to reduce the dimension. One common technique involves performing principal component analysis on the data and then apply clustering on the coordinates of the data in the eigenbasis, with all the difficulties of choosing dimensions that this entails. Clustering by combining Dirichlet processes (Coquet *et al.*, 2002) offers a way to get around these difficulties.
**Patterns mining** – Patterns correspond to implied regularities/irregularities or specificities of the data or subparts of the data. In agronomic applications, an individual can be described through a sequence of characteristics or events. For example, a plot can be described by a sequence of cultivation operations, a plant can be described by a DNA sequence, etc. one of the major challenges with this type of data is the extraction of frequent or rare subsequences.

<figure><img src="image_21.png" /></figure>

For example...

At INRAE, the **TETIS** UMR focuses on extracting frequent/rare subsequences in this type of data and frequent patterns in the form of items and sequences (sequences of events ordered in time) in order to characterise the difference in vegetation growth between different spatial areas. Their work is particularly relevant for wetland area estimation and biodiversity monitoring.

Other approaches aim to highlight sub-parts of the data with very different characteristics to the rest of the data (distribution differences for certain attributes, etc.). For example, in *Millot et al.* (2020), the authors use the notion of discriminating patterns to characterise, from simulation data, sub-families of crop protocols in urban farms where part of the attributes (temperature, light, CO₂, etc.) show an interesting distribution with respect to a given measure of interest.

Unfortunately, these methods are often faced with a number of patterns that prove to be too large to be easily used by experts. A promising and currently much studied avenue is the selection of the most relevant pattern subset. Patterns can be extracted from time series after a pre-processing phase in which the sequence of numerical values is transformed into a sequence of symbolic values, allowing classical pattern discovery methods to then be applied. When the numerical data is kept, methods for extracting representative subsets, called "shapelets", can be used.

<figure><img src="image_22.png" /></figure>

For example...

INRAE units such as the **PEGASE** UMR, **UMRH**, the **Toxalim** UMR and the **GenPhyse** UMR use these different learning approaches for precision feeding, early detection of anomalies in the activity of dairy cows in a herd, detection of pathologies in piglets or analysis of sow behaviour respectively.
Reinforcement learning

Like many types of data, agricultural data is often uncertain (see 3.1). Reinforcement Learning (RL) is concerned with learning to operate in an uncertain environment. One example of a modern use of RL for crop management planning is *Déciblé* (*Chatelin et al.*, 2005), originating from *Garcia* (1999) and based on interaction using a decision rule model for wheat cultivation. This empirical crop simulator is used to evaluate policies expressed as sets of decision rules. In *Ndiaye* (1999), model-free methods – namely Q-learning and R-learning – are mixed with genetic algorithms, decision trees and fuzzy logic to find optimal decision rules for crop management coupled with *Déciblé*. The result was considered to be not as good as the decisions that an expert would expect to see. These early approaches were interesting in that they introduced modern RL techniques for crop management while considering a range of actions. They also expressed an optimised policy in a natural way, i.e. in the form of a set of simple decision rules corresponding to farmers’ reasoning. However, the solutions in Garcia (1999) and *Ndiaye* (1999) are limited in that learning is offline, using an empirical decision model simulator with its own biases and field of validity. Because learning is not in real time, the systems do not use farmers’ feedback to improve the policy learned from the simulator.

These methods were later applied in a more complex context, incorporating an economic model for oilseed rape management and a pest and disease component in crop modelling (Trépos et al., 2014). RL methods have been successfully applied in irrigation planning when water availability is limited (Bergez et al., 2001). Nevertheless, each management decision must take into account the whole sequence of choices. Different crop varieties have different water requirements, so there will be different irrigation costs. Bu and Wang (2019), proposed a general computer architecture for intelligent decision making in agriculture based on deep Q-Learning. In practice, deep Q-Learning requires billions of instances of trial and error. Furthermore, no proposal has been made to integrate specialist knowledge (e.g. knowledge of plant physiology) into this system; approaches using expert knowledge could therefore be considered (model-based learning), allowing the amount of examples needed for training to be reduced by several orders of magnitude.
<figure><img src="image_23.png" /></figure>

The SCOOL project-team (Inria, CNRS, Université de Lille) specialises in reinforcement learning and is studying the recommendation of practices in agriculture for very small farms, especially in developing countries, and in gardening. The research is carried out with a focus on sustainable development.

Different machine learning and data science methods are implemented in the scikit-learn library, which was principally developed at Inria and is one of the three most downloaded artificial intelligence libraries in the world.

The INRAE MIAT unit is also working on the development of methods based on Markov decision processes and reinforcement learning applied to the management of agroecological systems, with particular focus on issues related to the spatial dimension of problems.

## Data warehouses and OLAP analysis

Data warehouses (DWs) were designed to handle very large volumes of data from heterogeneous sources (Chandra and Gupta, 2018). Multidimensional modelling (where data is characterised across multiple axes of analysis) and hierarchical modelling (where an axis of analysis can be associated with different levels of granularity) form the basis of DWs and multidimensional analysis. For example, the analysis of the amount of pesticides or nitrogen used by farmers can be characterised according to several dimensions (or axes of analysis): temporal, spatial and at crop level (Bouadi et al., 2017). This allows quantities to be represented by crop type, season and plot. These dimensions can be expressed in different levels of detail. For example, spatial information can be defined at the scale of a single plot or at a larger scale such as the watershed, region, etc., since each plot belongs to a watershed, which in turn belongs to a region, which in turn belongs to a country.

Multidimensional analysis uses OLAP (On-Line Analytical Processing) to aggregate, visualise and interactively explore data. If we take the previous example, we could analyse the quantity of pesticides or nitrogen at plot level or at a more aggregated level of spatial dimension such as the watershed. OLAP processing is used to navigate between different granularities of one or more dimensions in a very efficient way (i.e. navigation is instantaneous).
Users can use the data warehouse by combining the different dimensions and different levels of granularity of the corresponding hierarchies. To select the appropriate data at the right scale, users express and submit queries to the data warehouse.

Other works (Palpanas, 2000) describe the coupling of multidimensional analysis with data mining methods (e.g. pattern mining), with the aim of proposing hybrid methods that combine the exploratory and analytical capacities of OLAP with the descriptive capacities of data mining. For example, the ADSS-OLAP tool (Abdullah and Hussain, 2006) combines OLAP and data mining (clustering) and was developed to analyse the impact of mealy bug on cotton crops. To further enhance OLAP analysis and allow geographic data mining, the idea emerged to couple OLAP and GIS (Geographic Information System) technologies. Thus, the new concept of Spatial-OLAP (SOLAP) (Bédard et al, 2007) was introduced to jointly exploit OLAP tools (decision, graphs, etc.) and geographic tools (cartographic representation, geographic aggregators, etc.).

<figure><img src="image_24.png" /></figure>

For example...

At Inria, the **LACODAM** project-team (Inria, Institut national des sciences appliquées de Rennes, Institut national supérieur des sciences agronomiques, agroalimentaires, horticoles et du paysage, Université de Rennes 1) has modelled and built a data warehouse to analyse/explore, in space and time, the effects of agricultural practices on nitrogen emissions to water and the air (Bouadi et al., 2017). The team is also studying the use of machine learning to improve animal welfare (dairy cow health and sow feeding).

At INRAE, the **TSCF** unit focuses on spatial OLAP. Among other things, it contributes to storing and analysing biodiversity data online, in particular through the VGI4BIO project (www.vgi4bio.fr) which proposes methods for analysing biodiversity indicators in an agricultural context centred around data and VGI users.
<figure><img src="image_25.png" /></figure>

<figure><img src="image_26.png" /></figure>

Optidose® a tool to adapt the dose according to the parameters of the crop and the epidemic risk.
© Le Mas Numérique.

## 3.4 Knowledge management and engineering for decision support in agriculture

The previous sections provided an overview of the state of the art in data collection, management and processing; we also saw how modelling makes it possible to manage and represent knowledge in a mathematical way using measurements and observations to help with interpretation and recommendation. Another important facet of digital agriculture is knowledge management, i.e. higher-level information including both general scientific knowledge (e.g. plant or animal physiological processes) and methods specific to certain actors in the agricultural sector (e.g. a livestock farmer's herd management, methodology for making certain cheeses, etc.). In recent years, significant effort has been made to formalise this knowledge and organise it in ontologies that provide a structured access. Ontologies are a component of computer systems that help users accomplish a task. This assistance can take many forms, from automating an irrigation decision to finding information to help make a decision. Knowledge can also be generated by the analyses presented above. In this case, the difficulty lies in presenting these analyses to human actors in the most intelligible way. Again, recent developments in data analysis are of particular interest to agriculture, whether via visualisation approaches or methods for interpreting learning models. Lastly, the aim of
everything presented in this section is to enable human actors to make better
decisions. Specific tools known as DSS (Decision Support Systems) that use all or
some of the techniques presented in this chapter are available to these actors and
are constantly evolving. This section will conclude with an overview of these tools.

<figure><img src="image_27.png" /></figure>

For example...

The **BIOEPAR** UMR contributed to the development of **EMULSION**, an open source program and DSS based on artificial intelligence. **EMULSION** allows modellers to develop stochastic mechanistic models of complex systems in epidemiology at different scales and using different paradigms, while reducing the amount of computer code that has to be written (Picault *et al.*, 2019). Based on this, the **ATOM** (Automation of decision support Tools based On epidemiological Models) project aims to develop a process for industrialising the DSS generation using mechanistic epidemiological models (https://www6.angers-nantes.inrae.fr/bioepar/Recherche/Projets-en-cours/ATOM).

Knowledge-based systems in agriculture

**From the first expert systems to knowledge-based systems** – The first expert systems emerged in the 1970s as a result of research in artificial intelligence. These systems were dedicated to resolving a specific problem by using the knowledge of one or more experts and mimicking their reasoning, with the ultimate aim of replacing them. In one approach, called the symbolic approach, expert knowledge is formalised using a knowledge representation language based on logical reasoning. This is in contrast to the connectionist approach, which mimics the functioning of the human brain using neural networks.

One particularity of the agricultural sector is that the inherent problems in
crop or herd management require expert knowledge in several fields (soil science,
meteorology, chemistry, biology, etc.). To meet this demand for multidisciplinary
expertise, some expert systems incorporate simulation models as components,
such as those described in Section 3.2. Such an example is the expert system
"CrOp MAnagement Expert" (COMAX), dedicated to cotton cultivation, which aims
to maximise yields while minimising inputs (McKinion and Lemmon, 1985) and
encapsulates a simulation model of cotton development (GOSSYM). Because the
acquisition of expert knowledge is crucial for the development of expert systems,
knowledge engineering has focused on methods for acquiring such knowledge.
These methods have been used to guide cognitive scientists through the complex
tasks of identifying, extracting and formalising expert knowledge from a variety
of sources (expert interviews or other documents describing the task of solving
the problem).

Very popular in the 1980s, expert systems have also been severely criticised
for not being adaptable to applications other than the one they were designed
for and for offering poor potential for development. In the 1990s, expert systems
gradually evolved into knowledge-based systems. The notion of ontology then
appeared in computer science. Ontologies are designed to formalise consensual
and relatively stable knowledge in a given domain to allow it to be reused in other
knowledge-based systems.

A knowledge-based system is composed of two distinct parts: firstly, a
knowledge base including an ontology that structures the knowledge from
the domain, with a fact base that instantiates the ontology to describe specific
situations and sometimes a rule base that enriches the ontology; secondly, a
reasoning engine associated with the knowledge representation language but
independent of any particular knowledge base.

**Evolution of knowledge acquisition and capitalisation methods** – The shift toward knowledge-based systems was accompanied by a change in the conception of the relationship between humans and machines. Knowledge-based systems and their associated intelligent computer system aim to cooperate with the user to help them perform a task requiring different types of knowledge by supplementing the user's knowledge, revealing the consequences of their choices and proposing alternative options to those they would have imagined. Knowledge engineering then evolved into a form of knowledge modelling mediation, producing "knowledge models" – a model here taking on a different and more global meaning than in Section 3.2 since it no longer represents phenomena but knowledge. These models allow the cognitive scientist, in charge of implementation in a computer system, to dialogue with experts to enrich and validate the knowledge to be represented. To help this mediation, several methods have been developed, the best known being "Knowledge Acquisition and Documentation Structuring" (KADS, from which commonKADS was later developed). For example, a system for recommending irrigation dates for mango trees was developed using the commonKADS method (Nada et al., 2014).
<figure><img src="image_28.png" /></figure>

For example...

The **GRAPHIK** project-team (Inria, INRAE, CNRS and Université de Montpellier) studies knowledge representation. Among other things, the team is working on a method for collecting, modelling and formalising knowledge to improve the quality of local cheeses. Data collection is carried out through questionnaires; modelling is done by creating mind maps to facilitate validation by experts and formalisation is carried out using conceptual graph language.

Knowledge models were deployed in other computer systems such as information (source) search systems. This development led to the creation of "organisational memories". An organisational memory is the set of human and material resources – knowledge carriers – which allow an organisation to carry out its tasks. A memory can be composed of a set of text documents, videos, lists of employee skills and one or more knowledge models. The formalisation of these models allows them to be used automatically to help the circulation of resources and knowledge among the members of the organisation. This formalisation often takes the form of a thesaurus: a structured list of standardised terms organised into three types of relationship (equivalence, hierarchy, association), with the aim of indexing and helping to search for different content.

<figure><img src="image_29.png" /></figure>

Find out more

The FAO (Food and Agriculture Organisation of the United Nations) has been instrumental in producing organisational memories in the agricultural field (O'Leary, 2008). Its bibliographic database AGRIS offers different types of resources (scientific documents, data sets, etc.) in several languages. These resources are indexed using the **AGROVOC** thesaurus, which is now one of the most important tools in the agricultural field (Sini et al., 2008). This thesaurus encompasses many aspects of the agricultural domain and can be adapted to the needs of a new organisation. An example of an organisational memory is the Agropedia project, led by a number of Indian agricultural institutes in collaboration with the FAO (Pappu et al., 2010) to transform scientific knowledge from universities into practical knowledge of use to farmers. Agropedia uses Topic Map knowledge models, which aggregate all the knowledge needed for a given crop.
**Semantic access to information sources** – The birth of the Semantic Web in the early 2000s had a strong impact on the field of knowledge representation. Semantic Web technology is a set of standardised languages, protocols and tools under the aegis of the W3C to enable the automated exploitation of Web resources according to their content. Web resources (such as HTML documents or, more broadly, any data available on the Web) are annotated with metadata describing their content in a formal language, constituting a fact base that can be enriched with a thesaurus or ontology that specifies its semantics.

<figure><img src="image_30.png" /></figure>

## Definition

The main formal languages of the Semantic Web are:

* Resource Description Framework (RDF): the language for describing Web resources in the form of a graph made up of triples (subject, predicate, object);

* RDF Schema (RDFS): an extension of RDF that allows a vocabulary to be defined in terms of classes and properties (or binary predicates) organised by specialisation;

* SPARQL Protocol and RDF Query Language (SPARQL): the RDF(S) description query language;

* Ontology Web Language (OWL): the most commonly used language for describing Semantic Web ontologies;

* Semantic Web Rule Language (SWRL): a rule language that can be used to enrich OWL descriptions;

* Simple Knowledge Organization System (SKOS): RDFS specification for formalising terminologies, thesauri, classifications and other vocabularies used in information retrieval systems.

Web ontologies are modular and focus on a specific need to facilitate their reuse and combined use. By making them available on the web, the interoperability between knowledge-based systems can be improved. Of particular interest is the RDA *Agrisemantics* working group initiative which has been exploring the use of this technology and associated resources to improve the exchange and sharing of agricultural data (Aubin *et al.*, 2017). This technology has allowed organisational memories to be transferred to the Web.
<figure><img src="image_31.png" /></figure>

For example...

INRAE is also developing organisational memories on the impact of climate change on agricultural practices and agroecology. An archive of French agricultural information bulletins, the Plant Health Bulletins (BSV), was compiled during the VESP project, which studied epidemiosurveillance networks (Roussey et al., 2017).

The GECO collaborative web portal (https://geco.ecophytopic.fr/) was developed to improve knowledge sharing around integrated crop protection and agroecology. This portal manages a set of explanatory text sheets to propose means of controlling pests (Soulignac et al., 2017). GECO allows users to perform searches regardless of their level of expertise.

Web ontologies and SKOS thesauri have become reusable resources. Specific portals have been developed for searching for all these resources..

<figure><img src="image_32.png" /></figure>

For example...

At INRAE, the computer science department of the MISTEA UMR has, among other things, developed the AgroPortal (http://agroportal.lirmm.fr/) which lists ontologies and thesauri related to agronomy and agriculture and makes them openly available. AgroPortal also provides services to help annotate text documents and detect links between concepts in two ontologies (ontology alignment). It also contributes to advances in high-throughput plant phenotyping.

**Semantic integration of structured data** – Linked Data refers to a network of linked sets of resources. It was developed in the 2010s and marked a new stage in data sharing using Semantic Web technology by considering a network of interconnected sets of resources. This network is based on the use of shared vocabularies (thesauri, ontologies, etc.), used to describe the data. This development goes hand in hand with the generalisation of the concept of data and encompasses all data, including structured data from different databases.
<figure><img src="image_33.png" /></figure>

For example...

Examples of structured data include:
Weather data from an INRAE station is available as Linked Data ([http://meteo.clermont.cemagref.fr/](http://meteo.clermont.cemagref.fr/)). A central Semantic Sensor Network (SSN) ontology provides a design pattern for describing the measurements.

The European project SmartOpenData ([http://www.smartopendata.eu/](http://www.smartopendata.eu/)) proposed an infrastructure (and a SmartOpenData (SMOD) schema) for managing Open and Linked Data in the field of biodiversity and the environment (e.g. in agroforestry data management).

The Agronomic Linked Data project (AgroLD - [http://www.agrold.org](http://www.agrold.org)) integrates 50 databases into a single RDF database. Its objective is to jointly question and link different points of view on cultivated plants (genomic, proteomic and phenomenic) formalised by at least one of the ten Web ontologies used (Gene Ontology, Plant Trait Ontology, etc.).

Hybrid architectures such as OpenSilex ([http://www.opensilex.org/](http://www.opensilex.org/)) that incorporate ontologies, an inference engine and different formats of databases (relational, NoSQL, RDF) are used to develop multiple information systems for high-throughput phenotyping. In this architecture, a scientific object (plant, pot, field, etc.) is identified by a web identifier (URI) and typed by an element of one of the associated ontologies. The RDF database stores the static descriptive metadata while the NoSQL database stores the raw data streams: drone photos, time series from field sensors, etc.
<figure><img src="image_34.png" /></figure>

For example...

The GnpIS information system stores all the structured data from experiments carried out on plant phenotyping (Pommier et al., 2019). The ontologies proposed by the Crop Ontology network (https://www.cropontology.org/) are used as dictionaries for all observable traits in the experiments.
For animals, the descriptions of animal experiments carried out in different research centres can be made compatible using a web ontology network developed at INRAE. The network currently consists of three web ontologies (Salaun et al., 2018): Animal Trait Ontology for Livestock (ATOL, on phenotypic traits in livestock), Environment Ontology for Livestock (EOL, on environmental parameters in livestock farming), and AHOL (for livestock animal health).

**Emerging architectures** – Beyond the Web of Data, the problem of the intelligent exploitation of increasingly large and heterogeneous data has led to very active research combining knowledge representation, data management, the Semantic Web, data mining and learning, etc. It is in this context that a new architecture has been proposed called Ontology-Based Data Access (OBDA) (Xiao et al., 2018), which combines a specific approach to data integration, called mediation, with the concept of the knowledge-based system. OBDA systems are structured in three levels: the conceptual level, organised around an ontology (described for example in RDFS or OWL); the data level, composed of various pre-existing and independent databases; and the mapping level, which translates the data relevant to the target application into a fact base using the ontology vocabulary. Queries to the system (e.g. in SPARQL) use this vocabulary, with the user expressing himself at a conceptual level with no knowledge of the data storage system (for example, a query such as ""*what auxiliaries can control pest X and what are the associated techniques that would limit competition with the main crop?"" would be completely dissociated from the underlying database schemas).

<figure><img src="image_35.png" /></figure>

For example...

At INRAE, the Ecology of Mediterranean Forests URFM unit of the **ECODIV** department conducts multidisciplinary research in ecology. In particular, it implements mature OBDA systems such as Ontop (https://ontop-vkg.org/) and MASTRO (https://www.obdasystems.com/mastro) for the sustainable management of Mediterranean forest ecosystems.
In the context of the Internet of Things, some of these systems also use Semantic Web technologies (OWL ontology, SWRL rules, RDF annotation base).

Standardisation bodies such as W3C and ETSI are currently working on the validation of new standards and ontologies to combine the Internet of Things and the Semantic Web: these are the SAREF ontology by ETSI and the Web of Things (WoT) ontology by W3C. These ontologies have not yet reached a sufficient level of maturity to be used in real-life applications.

<figure><img src="image_36.png" /></figure>

For example...

At INRAE, the TSCF unit has proposed a translation of the IRRINOV manual irrigation method into SWRL rules and a web ontology network to represent knowledge for automating irrigation.

Questions remain about the compatibility of ontologies built on different principles: different uses, different authors, different foundational ontologies etc. Some web ontologies propose data schemas corresponding to reusable patterns (design patterns) centred on a specific need. Other ontologies offer reference classifications to qualify data. Data managers must therefore build a network of ontologies to structure their data, checking that these ontologies remain compatible with each other. Are they based on the same patterns? Do they allow correct inferences to be made? Lastly, current research topics focus on questions concerning the distribution of reasoning over all the components of an “Internet of Things”-type system.

## Knowledge restitution, visualisation and human-machine interaction in agriculture

Data-driven knowledge production methods (section 3.3) have produced results that are not only increasingly accurate and reliable but also increasingly difficult to understand, to the extent that most of these approaches are now described as “black boxes”, of which the user is unable to understand the determinants of the result produced (e.g. a decision on the technical process). One solution to this problem consists in using local interpretability approaches such as LIME (Ribeiro et al., 2016) or SHAP (Lundberg and Lee, 2017). Instead of aiming to explain the learned model as a whole, which is too complex, these approaches explain the reasons that led the model to produce such a decision in the specific case provided by the user, such as the attributes that contributed most (positively or
negatively) to the decision. For example, SHAP, used in oestrus detection as seen
above (Fauvel et al., 2019), provides explanations of the type: "*an oestrus was
predicted today based on temperature changes over the last three days and a
significant rest period three days ago.*"

In parallel to the issue of interpretability, the visual representation of data and
information is essential for any computer system designed with user interaction
in mind. "Visualising" consists in producing visual elements (graphs, curves,
maps, images) to help users understand, explore and analyse to make sense of
data, models and information, sometimes present in large quantities and often
complex (Kubicek et al., 2013). Smooth and efficient human-machine interfaces
and visualisations are often essential to the success of digital systems intended
for the general public. The field of agriculture is no exception to the rule.

There is a high demand for visualisation in this sector due to a combination of
several factors: significant growth in the masses of data collected, the existence of
users who are not computer scientists but are often technophiles, and the need
for visibility on private and public data at different spatial and temporal scales.
Visualisation is sometimes even seen as a strategic matter, because mastering
these techniques can offer a competitive advantage or afford a certain power to
some actors in the agrifood chain. Private players (equipment manufacturers)
are heavily involved in the field, but there are also initiatives by universities and
institutes, including INRAE and Inria, available under free (e.g. AQUAPONY,<fnref n="35" />
GeoVisage,<fnref n="36" /> PARCHEMIN)<fnref n="37" /> or participatory licenses (I-EKbase)<fnref n="38" /> (Wachowiak
et al., 2017).

<figure><img src="image_37.png" /></figure>

For example...

At INRAE, the Ecology and Evolution of Zoonoses group of the **CBGP UMR** analyses the viral diversity of hantaviruses and the evolutionary processes that shape it. Among other things, they piloted the development of AQUAPONY, a web-based viewer that allows interactive navigation through a phylogenetic tree and facilitates the objective interpretation of evolutionary scenarios.

<footnote n="35">35. http://www.atgc-montpellier.fr/aquapony/</footnote>

<footnote n="36">36. http://geovisage.nipissingu.ca/</footnote>

<footnote n="37">37. http://www.parchemins.bzh/index.php/outil-de-visualisation-donnees-lagriculture-littorale-bretagne/</footnote>

<footnote n="38">38. http://iekbase.com/hot-spots-monitoring</footnote>
The tools currently deployed for the agricultural world are based on traditional visualisation paradigms: cartography (GIS, or Geographical Information Systems), sensor data visualisation interfaces, collections of linked visualisations (multi-faceted) and reactive tools (dynamic queries). Semantic interfaces and related visualisation strategies (linking views) are major topics of interest, as well as 3D visualisation, which provides a view of the topology of geographical areas (down to field profiles), and the use of image synthesis or even augmented reality.

<figure><img src="image_38.png" /></figure>

For example...

At INRAE, the SAS UMR supports the agroecological transition of livestock systems and territories. In particular, it participated in the *PARCHEMINS* project on the visualisation of coastal agricultural data in Brittany, completed in March 2021. This project developed a map viewer, a web-based map consultation tool that allows the user to interact with geographical data. Designed for users who are not geomatics experts or computer scientists, it allows spatial information to be represented and analysed in an intuitive and easy-to-use way.

The interactivity and speed of response of visualisation tools are key in agriculture (and elsewhere!) as these visualisations must be adapted to lightweight devices (smartphones, tablets) or on-board terminals (connected tractors). The fluidity of data visualisation is closely linked with technical solutions, data exchange protocols and system architecture. Adaptive visualisation, which is currently a major research topic, makes it possible to adapt the visualisation to the context, such as the user's profession, the visualisation terminals or the nature of the data available.

The issue of visualisation and knowledge sharing is not widely addressed in practice and thus remains more in the field of research than application for the time being. In agriculture, however, human expertise traditionally plays a very important role, creating a particularly favourable context for the development of interactive techniques: it is indeed tempting to combine human expert capacities with learning, optimisation or modelling algorithms (Boukhelifa et al., 2018). Depending on the strategy and the system, Human-Computer Interaction (HCI) can be either explicit (the user is regularly asked questions via an interface or visualisation system) or implicit (unbeknownst to the user or non-verbal, with the machine capturing information and using it as a learning base). Current research in visualization and HCI focuses primarily on questions concerning the interpretability, explicability, causality and transparency of interactions.
The HCI/visualisation tandem is also a factor in frameworks in which human expertise is required to manage data uncertainty and decision-making on multi-criteria issues: qualitative approaches are complementary to automatic/statistical approaches (e.g. choice of criteria) for managing ambiguities, knowledge gaps or extrapolations to different crop types. Examples include uses in agroecological zoning - based on clustering and segmentation techniques - or the Crop-GIS web application combining modelling and visualisation for maize crop management.<fnref n="39" /> However, interactive systems can be difficult to evaluate because while the algorithm learns and adapts to the human, the reverse is also true: the user learns to use a system. Understanding these subtle mechanisms of co-adaptation and co-evolution requires the use of experimental science approaches (test plans, reliability of results, biases) and testing on cohorts of volunteers.

In conclusion, the topics of visualization and HCI applied to agriculture are relatively rarely addressed in both agronomic and visualization scientific literature. And yet, the issue is a key factor in the adoption of technology because farmers prefer tools that are less accurate but easy to use to high-performance ones that are difficult to use (Pierpaoli et al., 2013).

## Decision Support Systems (DSS)

In the 1980s, computer programs and electronics began to be used to improve efficiency in agriculture and reasoning in agricultural activities. This saw the emergence of the first digital DSS (Decision Support Systems). This revolutionary development has been fairly well received both by farmers (79% of farmers who use new technologies recognise their usefulness, source: Rapport agriculture et innovation 2025)<fnref n="40" /> and in society where there is a demand for digital innovations for the protection of the environment (47% of those questioned, OpinionWay survey, 2016). Digital DSS are based on "simple" computer programming combined with a relatively small body of reference data, and can be installed on personal computers or used in a web interface that allows access to the application. They are most often developed by research or technical institutes. This generation of DSS includes software such as INRAtion<fnref n="41" /> and InraPorc.<fnref n="42" /> These programs, designed by INRA, are French benchmark tool in terms of assistance for defining feed rations for ruminants and pigs. Many software programs have also been developed in the plant sector to help farmers plan and manage crop fertilisation, pest control or irrigation. Today, with the upsurge in digital technology, a new generation of DSS

<footnote n="39">39. https://www.cropgis.com/</footnote>

<footnote n="40">40. https://agriculture.gouv.fr/sites/minagri/files/rapport-agriculture-innovation2025.pdf</footnote>

<footnote n="41">41. https://www.inration-ruminal.fr/</footnote>

<footnote n="42">42. https://inraporc.inra.fr/inraporc/</footnote>
has emerged that uses contemporary digital technology such as remote sensing, GPS, the Internet of Things, artificial intelligence, etc. These DSS are designed and produced by the AgriTech sector, which involves major agribusinesses and numerous start-ups (Padhy and Satapathy, 2020).

<figure><img src="image_39.png" /></figure>

## Definition

Agritech is a generic term for agricultural technology. It includes four main areas: 1/ biocontrol, 2/agricultural big data, 3/ robotics and 4/ plant genetics and biotechnology. These four elements are often closely linked and many agricultural technologies are derived from them.

The integration of new technologies into DSS allows the range of services offered by these tools to be expanded as farmers seek to make the most appropriate farm management decisions (Spanaki et al, 2021). Precise knowledge of the state of agricultural plots or herds is essential for the farmer, who can now use data (images, biophysical measurements, etc.) from connected sensors to obtain more information than can be perceived by the naked eye. After various digital processes, the farmer can access this information via a dedicated application online or on a smartphone. In the livestock sector, these new digital tools are readily adopted by farmers if they promise technical and economic gains and can reduce the arduousness of work. First of all, there are DSS based on sensors worn by the animals (externally or internally), which provide real-time measurements of the physiological characteristics of the animal and its activity (temperature, abdominal pressure sensors, movement, etc.). In dairy farming, the farmer can use these tools to monitor the animal's reproductive cycle and reliably detect heat or parturition or health problems, even before any external signs can be detected by a professional. We are also seeing the emergence of DSS prototypes based on image recognition (from cameras installed on the farm) using artificial intelligence methods (deep learning). These allow animals' behaviour and health to be monitored and can even go so far as facial recognition. If an anomaly is detected in a group or animal, an alert can be sent to the farmer's smartphone. Despite the number of initiatives underway, certain issues, which are crucial for making a DSS used and usable by professionals, are still the subject of research carried out in collaboration with the latter. In particular, these concern precision, pertinence (a DSS that provides too many false alerts, for example, risks being rejected), the adequacy and form of the information made available to the farmer according to his expertise and needs, and the ergonomics of the tool, in connection with the notions seen in the visualisation and HCI section (Li et al., 2020). The way in which user knowledge is used is also the topic of ethical questions raised in connection with open innovation more generally.
<figure><img src="image_40.png" /></figure>

Automation, control and robotics

As highlighted above, digital farming is far from being limited to data acqui-
sition and processing. The aim is to use this data in decision making and deter-
mine actions to be taken, both spatially and temporally, to optimise cultivation
techniques capable of reconciling high levels of production, crop quality and
environmental conservation. In this sense, precise and potentially frequent work
will be required in order to meet such specifications, which is not always possible in
terms of human resources and capacities. This is especially true since agricultural
tasks are often tedious and sometimes dangerous. Exploiting the full potential
of the principles of digital farming could therefore lead to task automation.
Today, robotics technology is taking the developments already implemented
in the context of automated tools or driver assistance systems for agricultural
machinery even further. But, beyond the automation of certain tasks, advances
in the field of robotics in the agricultural world must pave the way for a change
in practices to accompany the ecological transition.

Farmstar is a DSS based on subcellular spatial images. It was developed by Airbus in collaboration with agricultural technical institutes. The complex processing chain combines the use of spatial images and other data sources such as climatic data, and uses computer simulation for agronomic models. The result can be accessed via APIs (Application Programming Interfaces) which are queried by the user application. The farmer can thus obtain useful information in the form of maps and "dashboard" indicators via an integrated web application that hides the complex computer architecture and data flows involved.

<figure><img src="image_41.png" /><figcaption>Figure 1: Farmstar, from high-resolution spatial imagery to advice maps.</figcaption></figure>
<figure><img src="image_42.png" /></figure>

The **VALSE** project-team (Inria, Ecole Centrale de Lille, Université de Lille) studies problems arising from the analysis of distributed, uncertain and interconnected dynamic systems. Its aim is to design estimation and control algorithms for different fields. In particular, in the field of oyster farming, these algorithms have enabled the design of a biosensor based on the measurement and interpretation of bivalve mollusc behaviour, for the remote detection of coastal water pollution and the consequences of climate change.

## Structured environments: allies of robots

The rise of robotics has historically been rooted in industrial applications, especially automotive, for the automation of production lines (Bahrin et al., 2016). From this, it is possible to design infrastructures that allow robots to be referenced and operate in perfectly known and unchanged environments, as well as to control the conditions of interaction (lighting conditions, handling known objects, creation of specific zones). This greatly helps the design of robust perception and control algorithms based on robot operation models that require strong assumptions (rolling without slipping, object or scene recognition, accurate localisation, etc.). As a result, robotics applications in agriculture have primarily focused on the indoor environment, particularly for livestock production (Bergerman et al., 2016). In this sense, the biggest market for robotics in agriculture is currently in the livestock sector, with feeding and milking robots. These are able to operate using a number of reference points and benefit from special arrangements to maintain high repeatability. They can thus perform demanding tasks (such as milking or feeding animals) and free up the farmer's time. Such developments are increasingly common in agricultural practices, and today half of new French dairy farm facilities are equipped with milking robots (Tse et al., 2018).

In cropping, such infrastructures are more difficult to put in place, with the structure of crop production being inherently changeable and posing detection and referencing issues. Nevertheless, the automation of certain tasks, particularly driving farm machinery, has greatly benefited from the advent of GPS, especially centimetre-precision models which offer absolute referencing. Many devices aimed at automating machine operation under the supervision of a "driver" have thus emerged, sharing a certain number of research challenges with advances in driverless vehicles.
However, the use of GPS sensors alone remains limited for the production of fully autonomous robots (i.e. without on-board human supervision) for several reasons. Firstly, the potential loss of satellite signals near buildings, in greenhouses or near tall vegetation, would require manual intervention. Secondly, farming requires referencing and interaction with plants and not absolute references, even if planting is carried out using GPS referencing. Lastly, the absence of an on-board supervisor means that autonomous machines must be equipped with a means of perception to ensure their safety (avoiding obstacles, traversability management).

<figure><img src="image_43.png" /><figcaption>Study of a robotized electric tractor for agroecology. © INRAE.</figcaption></figure>

Thus, several other strategies including vision (Stefas et al., 2019) and laser technology (Tourrette et al., 2017) are substituting or complementing absolute referencing to achieve autonomous navigation. This is already being used commercially in robots, mainly for mechanical weeding, mowing and surveillance. However, the task efficiency of these robots is currently limited and performance is closely correlated to detectability conditions.

Before envisaging more complex work (pruning, harvesting in the field) performed in a fully autonomous way, there are several scientific and technological obstacles that must be overcome in order to deal with the variability of the environments and the diversity and complexity of the tasks to be carried out while preventing any damage to the robot(s).
## From adaptation to reconfiguration

Unlike mobile robotics in industrial environments or road traffic, mobile robots in natural environments require specific adaptive abilities to deal with the diversity of interaction conditions and their variability (Bergerman et al, 2016). This involves the online modification of perception and control parameters (such as modifying response times as a function of speed (Hill et al, 2020) or adapting detection thresholds to light conditions. Several adaptation and anticipation or robust control mechanisms have been proposed to deal with the variation in these environments and maintain a high level of accuracy, while protecting the robot from damage (Krid et al., 2017, Yandun et al., 2017). This last functionality is defined in a relatively binary way in structured environments: avoid collisions with geometric obstacles, do not operate in out-of-bounds areas, etc. In natural environments, the notion of "obstacle" is less well defined and solutions are more complex. Firstly, encountering an obstacle is not necessarily a failure, as robots do not have to be stopped when passing over vegetation or if they have to push aside a branch. Secondly, some areas can be traversed under certain conditions (speed or load limitation) and the crossing also depends on the ground conditions (especially adhesion) and the properties of the robot (Guastella, 2018). Lastly, operating in some areas may lead to a loss of control or physical stability of the robot (Wolf et al., 2019).

<figure><img src="image_44.png" /></figure>

### For example...

At INRAE, the TSCF UR designs reconfigurable and shared autonomy systems to enhance the performance and safety of machines operating in natural environments, particularly those found in agriculture. For example, the team designs adaptation mechanisms to deal with the diversity of interaction conditions and their variability.

Several approaches allow this complexity to be taken into account through the concept of traversability (the set of conditions allowing a given area in front of the robot to be crossed). Nevertheless, work on this concept illustrates the difficulty of defining a single perception and control approach to allow a robot to perform complex agricultural tasks. Many studies currently focus on the real-time selection or fusion of typical behaviour (see the INRAE Adap2E project^43), which addresses the problem of scene interpretation and behaviour evaluation.

<footnote n="43">43. https://adap2e.inrae.fr/</footnote>
In addition, many strategies in agricultural robotics are based on cooperation between less complex robots able to work together or in the same area. This reduces the risks in terms of the operation of each robot (limited kinetic energy in the case of impact) and their cost, but shifts the issue of complexity to the association and synchronisation of the group (Blender et al., 2016).

Conclusion

In this chapter, we have browsed at the different areas of research addressing the use of digital technology in agriculture. They mainly concentrate on data at all levels of the data cycle, from capture to exploitation via collection, traceability, processing, storage, interpretation, restitution and use in automated or robotic systems. Different skills involving networking, modelling, learning, knowledge management, control and security are used to provide efficient, safe and secure solutions. The key aims are to assist farmers in difficult tasks, allow better management of our resources and promote exchanges and expert knowledge, all while respecting the environment as much as possible.