# On-farm experimentation practices and associated farmer-researcher relationships: a systematic literature review

Quentin Toffolini, Marie-Hélène Jeuffroy

► To cite this version:

Quentin Toffolini, Marie-Hélène Jeuffroy. On-farm experimentation practices and associated farmer-researcher relationships: a systematic literature review. Agronomy for Sustainable Development, 2022, 42 (6), pp.114. 10.1007/s13593-022-00845-w. hal-03879583

HAL Id: hal-03879583

https://hal.inrae.fr/hal-03879583

Submitted on 4 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# On-farm experimentation practices and associated farmer-researcher relationships: a systematic literature review

Quentin Toffolini<fnref n="1" /> · Marie-Hélène Jeuffroy<fnref n="1" />

Accepted: 4 November 2022 / Published online: 30 November 2022
© The Author(s) 2022

## Abstract

The convergence among the rise of digital technologies, the attention paid to the localized issues of transitions in practices toward agroecology, and the emergence of new open innovation models are renewing and reviving the scientific community's interest in on-farm experimentation (OFE). This form of experimentation is claimed to be enhanced by digital tools as well as being an enabler of production of credible, salient, and legitimate science insofar as it embraces a farmer-centric perspective. However, the forms of research in which some experimental activities on farms are anchored vary greatly, notably with regard to the actual forms that interventions on farms take, the legitimacy of the actors involved and their roles, or the observations and instruments applied for interpretation. We propose a systematic review of the literature and an analytical framework in order to better understand this diversity of practices behind on-farm experimentation. Our analysis segregated six major publication clusters based on themes appearing in titles and abstracts. These themes guided a more in-depth analysis of representative articles, from which we identified seven types of OFE practices that are described and discussed here with regard to the knowledge targeted, roles of the various actors, and on-farm experimental space. Our typology provides an original basis for supporting reflexivity and building alignment between the above-mentioned dimensions and the ways in which new tools can support the experimental process.

**Keywords** On-farm experimentation · On-farm research · Participatory trial · Digitalization · Semantic networks

## 1 Introduction

Agricultural research and extension activities that rely on experimentation have been embedded in various ways in physical and institutional spaces (including laboratories, research stations, and farms). They have been associated with various methodological developments or research fronts in disciplines such as agronomy, genetics, crop physiology, livestock science, and ecology (Sumberg et al. 2003; Maat 2011; Parolini 2015). Recently, an explicit interest in experiments that take place on farms in collaboration with farmers has led to a renewed and specific definition proposal for “on-farm experimentation” (OFE) (Lacoste et al. 2021).

The respective roles and legitimacies of experimental research stations and farmers’ fields as appropriate spaces to

produce knowledge and innovations and their evolution, as changes in agronomy-centered research occurred, have been partially documented. Historical studies have already shown how experimentation practices are influenced by developments in methodologies and basic disciplines. For example, in the mid-nineteenth century, the rise of chemistry and inferential statistics was decisive in the development of agro-nomic research stations (Jas 2000). While research stations were preferred spaces for the development of generalizable knowledge and rules that were to be disseminated to farmers, the rise of more systemic and context-dependent approaches (for instance the Farming System Research movement) has led to a greater focus on farmers’ practices and has given some legitimacy back to experiments in less-controlled environments such as that of commercial farms (Lockertetz 1987; Jouve 2007). These types of experiment were, for instance, proposed for more adequately taking into consideration particular local circumstances in the implementation of techniques or with regard to specific farmers’ objectives, or in making necessary final adaptations of new technologies (Sumberg et al. 2003). In fact, agronomists’ ways of

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<footnote>Quentin Toffolini
quentin.toffolini@inrae.fr</footnote>

<footnote n="1">¹ Université Paris-Saclay, INRAE, AgroParisTech, UMR Agronomie, 22 place de l'Agronomie, 91123 Palaiseau Cedex, France</footnote>
designing new practices and systems, and of disseminating
them and the associated knowledge, entailed a new relation-
ship to farms as places where experiments are carried out
and to farmers' practices in general (Salembier et al. 2018).
Interactions between on-station and on-farm experiments
have consequently been examined from different perspec-
tives. Some researchers focus on the balance between the
scientific legitimacy of knowledge production and the rel-
evance of the proposed techniques in real farming conditions
(e.g., Henke 2000). Others underline the reciprocal inspira-
tion among farmers' own experiments, observations made at
stations, and on-farm experimentation in which farmers and
researchers collaborate (Maat and Glover 2012; Périnelle
et al. 2021). Along the same lines, some authors directly
associate the saliency of OFE with the research questions
to be answered, namely, questions concerning agricultural
techniques that are particularly sensitive to management
skills and conditions of implementation in particular con-
texts (Lockert et al. 1987). These various approaches to OFE in
research activities exhibit a diversity of applied practices and
tools, regarding what the actual form of intervention on the
farm is, who the involved actors are as well as their roles and
legitimacy, and what observations and instruments applied
for interpretation are (Fig. 1). We argue that there is still a
lack of studies providing descriptions and distinctions within
this diversity to inform agricultural research communities.

These descriptions of the realities of the OFE prac-
tices are all the more salient now as the literature shows
a recent revival of interest in OFE (Lacoste et al. 2021)
at the intersection between several agricultural research
fronts. First, enhancing agroecological transitions calls for
“option by context” approaches (Sinclair and Coe 2019),
taking into account the particularities of farming situations
and the complexities and uncertainties in their evolution
as practices change (Meynard et al. 2012; Tittonell et al.
2020). Bringing experimental processes into these diverse
situations is one of the means explored to allow for the
complexity, uncertainty, diversity, and variability linked
with these realities to be addressed (Lacoste et al. 2021).
Second, the development of digital tools to acquire, store,
analyze, and share a wide variety and large quantities of
data and knowledge has increased the interest in on-farm
experimentation (Bullock et al. 2019; Lacoste et al. 2021).
Digitalization is often related to the development and
improvement of precision agriculture methods for moni-
toring and managing variability (Kyveryga 2019). How-
ever, digitalization also supports the development of tools
and platforms for sharing qualitative knowledge, situated
experiences, and action strategies in more contextualized
and illustrated ways (Compagnone et al. 2018; Girard and
Magda 2020; Salembier et al. 2020). Third, the increasing
development of open innovation theories and approaches
within the agricultural sector also lays the foundations for

<figure><img src="image_4.png" /><figcaption>b)</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 1** *a* Two treatments are applied on 0.25-ha farm to compare different ways of growing camelina (monocropping and intercrop with barley). *b* Yields of an on-farm trial being measured with harvest equipment rather than with microplot sampling (credit: Margot Leclère).</figcaption></figure>

OFE practices (Cook et al. 2021). In fact, OFEs are part
of many coordinated activities comprising innovation pro-
cesses in which their functions and outputs depend on the
actual experimental practices applied. For example, new
innovative processes and arrangements (e.g., living labs
and transition experiments) link OFE practices to the rec-
ognition of the value of combining diverse interests (public
and private) and to the fostering of trusting and produc-
tive relationships among actors (Schäpke et al. 2018). The
renewal of OFE favors “joint exploration whereby research-
ers and others engage closely with farming realities to align
with the ways farmers learn” (Lacoste et al. 2021). Over the
past few decades, various on-farm experimentation prac-
tices have thus emerged in accordance with and as part of
distinct approaches to innovation. These range from tech-
nology transfer (where farmers are considered as adopters
and scientists innovate from the knowledge they produce)
to the farming system research approach (where farmers
provide scientists with knowledge and information about

<figure><img src="image_6.png" /></figure>
particular contexts and needs) and, recently, the agricul-
tural innovation system (where scientists and farmers are
partners in co-innovation processes along with other actors
in various political, agro-climatic, economic, and institu-
tional contexts) (Hall 2007; Klerkx et al. 2012). These three
archetypal movements, coexisting but anchored in different
innovation paradigms, are inviting agricultural researchers
to build reflexivity about what the various characteristics
of OFE practices are and how they contribute to different
innovation dynamics.

In this paper, our objective is to facilitate this reflexivity
and transparency on how OFE is conducted by benchmark-
ing existing practices. Through a literature review, we high-
light the diversity of ways of defining and performing OFE
with a focus on the characteristics of experimental practices.
Regarding this diversity, we also aim to identify and discuss
the various forms of digital development that support these
experimental practices or that could do so. More specifically,
we answer the following questions: can we identify diverse
types of experimental practices within the OFE-related lit-
erature? If so, how do these different forms of OFE rely
on digital devices and what would be the various roles that
digital technologies could acquire in these OFE practices?
We first present our two-step method, based first on the sci-
entometric analysis of a large corpus of publications, and,
second, on a more in-depth and qualitative analysis of sub-
sets of publications spread across the topics identified. This
led us to identify seven types of OFE practices based on a
specifically proposed analytical framework. After describ-
ing these types of OFE practices, we discuss in the final
section how this typology of approaches can support the
deeper analysis of their intertwining with the development
of digital technologies and, more generally, different OFE
development perspectives.

## 2 Material and methods

Our research process was two-fold. First, we built a corpus of
literature addressing theoretical aspects of OFE or presenting
research based on OFE practices. We applied scientometric
analysis based on methods developed for mapping networks
and characterizing the socio-semantic dynamics from publica-
tion corpuses (Cointet 2009). This type of analysis combines:
i) the construction of a specific corpus, ii) lexical extractions
that make it possible to build lists of terms characterizing
diverse practices, iii) indexation of the corpus with these cus-
tom lists of terms, and iv) analysis of the frequencies and co-
occurrences of these terms within the corpus. In the second
step, we used the resulting structure of the scientific corpus to
build an analytical framework. This served to describe experi-
mental practices that enabled the steering of a more in-depth
qualitative review of a sample of representative articles.

## 2.1 Construction of the corpus and semantic analysis

We performed a query on the SCOPUS database, com-
bining variants of terms referring to the location of farm
experiments or farmer-centric approaches, with the terms
related to experimentation activities. Terms related to
farmer participation in research activities were considered
in constructing the query (e.g., “participatory research,”
“participatory experiment,” and “collaborative research”),
but they resulted in corpuses that went well beyond the
literature, allowing us to describe specifically experimen-
tation practices and their relations to farms. Furthermore,
we ensured that a query without these terms would still
return articles related to participatory research by checking
that specific articles we knew should have been captured
were indeed present in the corpus. We focused the query
in order to capture mainly the predominantly English-lan-
guage, academic-focused literature. This made the textual
analysis possible on the whole resulting corpus (since the
tools applied for the scientometric analysis cannot handle
several languages simultaneously). This was in alignment
with our objective to explore the mainstream practices in
the leading internationally renowned literature. Finally,
the following query was applied on titles, abstracts, and
keywords that combine the targeted terms at a maximum
distance of 5 terms: TITLE-ABS-KEY ([“on farm” OR
“on-farm” OR “farmer-centric” OR “farmer? led” OR
“farmer? field” OR “farmer? managed”] W/5 [experi-
ment* OR trial OR demonstration OR test OR survey OR
research OR evaluation]).

This query returned 3955 publications (on August 16,
2021).

The 2000 most frequent terms (including monograms)
were extracted from titles and abstracts with the CorText
Manager Platform (http://manager.cortext.net/). After
removing duplicates and grouping together closely related
terms, we refined the list to conserve only terms related to
experimentation (for instance we removed all terms describ-
ing agronomic objects such as species names but kept terms
like *control treatment*, *crop response*, and *cropping sys-
tems*). We thus obtained a list of 926 terms. The corpus was
indexed with this custom list of terms based on the title,
abstract, and keywords fields. Co-occurrence counts and net-
work mapping were then performed with the new indexation
of the corpus.

The network mapping relied on the calculation of dis-
tances between terms, for which we applied the distribu-
tional measure as it is commonly used for homogenous net-
works (i.e., for calculating distances between terms from
the same fields of publication descriptions) (Weeds and
Weir 2005). Closest subgroups of terms were then clustered
based on Louvain's algorithm, limiting the number of closest
neighbor terms to seven to clarify the readability of the gen-
erated map.

The network of co-occurrences of terms was first ana-
lyzed to identify the types of methodologies applied in OFE
practices and the scientific communities to which these uses
refer (namely, by identifying journals related to each clus-
ter). For instance, we analyzed the terms used in connection
with “on-farm research,” “on-farm experiment,” and “on-
farm trials,” as these three expressions are grouped in dif-
ferent clusters. This first analysis was based on publication
metadata (namely authorship, publication journals, discipli-
nary areas, keywords, and titles) and the network of terms,
without including entire publications. Reading entire sub-
groups of publications and analyzing them in-depth served
as the second step of our analysis. The mapping of term co-
occurrences made it possible to identify clusters and associ-
ate the publications of the corpus with each cluster. Between
5 and 10 of the most cited publications from each cluster
were selected for the second step of our analysis.

## 2.2 Framework to analyze the specificities of experimental practices

To decipher practices associated with the various dominant
terms in the corpus and to better understand their diversity
as reflected by the clusters obtained, we applied a specific
analytical framework.

Our analysis of experimental practices described in the
selected articles was based on relevant analytical categories
inspired by preceding works on experimentation either in
urban and public spaces (e.g., Laurent and Tironi 2015) or
in rural contexts (e.g., Lovell et al. 2018), and on studies
on experimentation in sustainable transition contexts (e.g.,
Caniglia et al. 2017).

We connected four analytical categories that were appro-
priate for further understanding the experimental logic
behind the analyzed cases of OFE practices in each cluster:

i) Construction of a space for experimentation: this refers
to aspects that are considered in order to characterize a par-
ticular experimentation site. As Engels et al. (2019) point
out, “what counts as real-world conditions for testing are
never just ‘out there,’ but always subject to interpretation
and occasionally highly contested.” In fact, we focus specifi-
cally on two aspects: a) the delineation of what is consid-
ered as part of the experiment and what is not (e.g., to what
extent are complete plots or cropping systems included in
the analysis? Are the adaptations applied by the farmers con-
sidered? To what extent are the existing farmers’ practices
intertwined with the applied treatments and analyzed?) and
b) the dimensions of on-farm contexts that are included in
the characterization, selection or comparison of experimen-
tal sites (e.g., are socioeconomic contexts considered as part
of what characterizes experimental situations?).

ii) Specificities of interventions: what are the practices effectively implemented on experimental sites? For how long? Who acts and how? In which parts of the experimental process? Is there an iterative adjustment of protocols during the course of the experiment and across various sites? How are practices monitored?

iii) Observations and measurements: for the same inter-
vention, the ways in which experimenters capture the out-
comes of actions (around which the uncertainty justifies the
experimental process) can take multiple forms. What types
of data are obtained? What instrumentation is deployed?
Which observations and analyses are included in the valori-
zation of the experiment? How are unplanned observations
or events handled when these are mentioned?

iv) Mention of digital technologies: mention of tools or of
the opportunities to develop them even when these are not
explicitly related to a digitalization process.

This framework made it possible to identify that each
cluster derived from the scientometric analysis covers
sometimes largely differing on-farm experimental prac-
tices. We thus progressively defined different types of on-
farm experimentation by iteratively allocating to each one
the most cited publications of each cluster. The qualitative
identification of these types of OFE practices resulted from
common features based on the four analytical categories
described above, encountered in the publications across
the different clusters. A minimum of five most cited arti-
cles associated with each cluster of terms (excluding pub-
lications finally considered to be outside the scope of the
inquiry) were included in this second part of the analysis.
We selected the most cited articles over a random selec-
tion, in spite of risk of bias toward publication years, in
order to favor influential publications reflecting mainstream
research practices and thoughts on the topic.

## 3 Results

The query returned 3955 documents published from 1940
but with a steep and constant increase in numbers of publica-
tions from 1980. The five most frequent journals were *Field
Crops Research* (157 articles), *Experimental Agriculture*
(118), *Agronomy Journal* (102), *Agricultural Systems* (88),
and *Acta Horticulturae* (85).

### 3.1 Analysis of the structure of the corpus: Seven thematic clusters related to OFE

The co-occurrences of the most frequent terms appearing
in titles, abstracts, and keywords were mapped according
to their frequencies and proximity (Fig. 2). Six clusters of
terms were clearly identified, as well as a peripheral one
consisting of only three terms (in red in Fig. 2), across a total

<figure><img src="image_7.png" /></figure>
<figure><img src="image_8.png" /><figcaption>**Fig. 2** Co-occurrences of terms extracted from titles, summaries, and keywords. The 200 most frequent co-occurring terms are shown (distribu-
tional proximity, proximity threshold 0.43, filtering of the seven closest neighbor terms).</figcaption></figure>

of 3955 publications. These clusters were named with the
two terms presenting the most numerous connections with
other terms within the same cluster.

The largest cluster concerned “grain yields and fertilizer
management” (light green, 1206 publications associated,
from journals such as *Field Crops Research* and *Indian
Journal of Agronomy*) corresponding to terms related to
yield (most frequent associated terms: increased yields and
yield gap) and fertilization management within cropping

systems (NPK fertilizer, fertilizer application, fertilizer
management, nutrient management, cropping systems,
and crop production). This cluster corresponded to stud-
ies generally focused on single species (e.g., rice, wheat,
and maize), associated with research-based fertilization
strategies to be tested or compared to traditional ones (Cui
et al. 2008; Peng et al. 2008). This research on fertilization
strategies includes specific types of OFE practices, as we
will describe below.

<figure><img src="image_9.png" /></figure>
An opposite cluster, “knowledge and innovations” (dark green, 755 publications), comprised terms related more to farmers’ participation and knowledge (work, farmer participation, collaboration, perceptions, knowledge, role, and decision) and to farmers’ perceptions of innovations and implementation (demonstration, adoption, perceptions, and innovations). Contrary to the preceding cluster, the term on-farm research was more common in the associated publications than on-farm trials or experiments. Associated publications dealt with farmers’ learning, risk perceptions (e.g., Ghadim et al. 2005), and complete farming systems or consistent combinations of practices (e.g., conservation agriculture, agroforestry, and system of rice intensification), as well as participatory research (e.g., Chambers and Ghildyal 1985; Carberry et al. 2002).

A smaller cluster, both in number of terms and associated articles, concerned “model and error” (orange, 216 publications). This cluster was connected to precision agriculture and sensor technologies (site specific, sensor, variability, precision, and estimate, model). The most frequent journals were Precision agriculture and Computers and Electronics in Agriculture. Articles from this cluster focused on the characterization of on-farm variability to refine new technologies or models (e.g., hyperspectral canopy sensing of paddy, Gnyp et al. 2014), or proposed plans for on-farm precision experiments (Alesso et al. 2021).

Two related clusters were articulated around the term on-farm evaluation. One of these clusters, “laboratory and samples” (blue, 816 publications), corresponded to evaluations based on collection of information and observations on farms (assessment, farm surveys, measures, and value), as in the case of unpredictable events occurring in farms and difficult to reproduce experimentally (e.g., a case control study on on-farm risk factors for tail biting in pigs, Moinard et al. 2003). The other cluster, named “station and weight” (yellow, 606 publications), concerned relations between experimental stations and farms for evaluation of varieties and crop traits and preferentially applied the term on-farm trial (varieties, environment, performance, replication, and on-station). For instance, Casler et al. (1998) evaluated perennial forage grass varieties for management intensive grazing (MIG) systems by replicating trials on three dairy farms in southern Wisconsin.

The sixth and last cluster of interest, “smallholder and food security” (light orange, 306 publications), concerned farming system analyses and impacts of policies and climate change. Different approaches were used in this variegated cluster (e.g., questionnaires and surveys, on-farm experiments, and action research), but it specifically comprised works that address households with regard to labor, income, and livelihoods. These dimensions of analysis led to particular methods for characterizing contexts in which experiments were to take place (e.g., surveys on labor

and perceived constraints by farmers, and socioeconomic diagnoses).

The described clusters showed distinct orientations in the use of farms as spaces where experiments are performed. Some objects appeared to be studied with specific OFE approaches. “On-farm evaluation”, for example, seemed to be related to the measurement of traits, performances, weights of dry matter, namely, applied to the study of varieties, and in connection with research stations (“station and weight” cluster, Fig. 2). However, these objects are more closely associated with works in a distinct subfield of agricultural research (that which is dedicated to biological measurements that happen to be taken on farms) than works that define how the corresponding methodologies apply (or use) experimentation on farms. This distinction can, for instance, be illustrated by the following: how the relation between on-farm evaluation and on-station trials with varieties is constructed, how various environments are characterized, and who acts in these experimental processes. The second step of analysis, presented in the next section, developed this more in-depth qualitative description of OFE practices.

## 3.2 Analysis of variety in experimentation practices associated with OFE

The identification of clusters made it possible to select publications for each one that would help refine the understanding of the associated research through in-depth reading. We applied the analytical framework to subgroups of the 5 to 10 most cited publications from each cluster in order to analyze the construction of the experimentation space, interventions, measures and observations, and the role and importance of digital devices. These four analytical categories brought out important variations in practices between clusters and within clusters that we inductively classified into 7 distinct “types of OFE practice.” These types and their main distinctive characteristics are summed up in Table 1. We describe each type in the following sections.

### 3.2.1 Type 1: Exploring and explaining a phenomenon through the diversity of a farmers’ circumstances and practices

Type 1 corresponded to experimentation on a large number of farms in which biophysical and farming situations were carefully assessed to further the understanding of their variability and how it affected the biophysical processes investigated. Assessments of diverse situations regarding the biophysical processes impacting a phenomenon under study were used to analyze variations and better understand the phenomenon itself. This was, for instance, typically the case in the study “The effect of shade structure on coffee grain yields” (Soto-Pinto et al. 2000) and in the research

<figure><img src="image_10.png" /></figure>
<table><caption>**Table 1** Summary of the distinct types of OFE practices according to our analytical framework, with potential digital uses</caption><thead><tr><th>Types of OFE practice</th><th>Construction of the experimental space</th><th>Interventions</th><th>Measures</th><th>Digital aspects</th><th>Examples of publications</th></tr></thead><tbody><tr><td><strong>1. Explore and explain phenomena</strong></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>- Explore a large range of agroecological and climatic conditions and characterized well in order to build understanding</td><td>- Farmers carry out usual practices, or slightly adapted.</td><td>- Quantitative data on the agronomic process of interest (e.g., yields, N use efficiency, water flow)</td><td>- Mentioned: no<br/>- Potential: user-friendly interfaces to collect information on practices?</td><td>Boiffin et al. 1981,<br/>Soto-Pinto et al. 2000<br/>Peoples et al. 2001,<br/>Sims and Luka-McCaufferty 2002,<br/>Rockström and Barron 2007</td></tr><tr><td></td><td>- Apply the same treatments and measures on stations and on farms</td><td>- Researchers perform targeted measures and exploratory diagnosis of situations</td><td>- Contextual qualitative data (e.g., farming systems, climates)</td><td></td><td>Fermont et al. 2009<br/>Clusters "grain yields and fertilizer management", "small holder and food security", "knowledge and innovations", and "station and weight".</td></tr><tr><td></td><td>- Monitor herds or farms for phenomena that are difficult to reproduce on stations (e.g., pig tail biting, litter properties)</td><td></td><td>- Incidence of observed events explained by farms' characteristics gathered through interviews or observations</td><td></td><td></td></tr><tr><td></td><td>- Compensate for farm variability with treatment modulation</td><td></td><td>- Farmers' evaluations are sometimes mentioned</td><td></td><td></td></tr><tr><td></td><td>- Diversity of biophysical contexts to validate or refine model-based techniques (e.g., fertilization strategies) or yield gap analysis</td><td>- Same protocols are applied (often by researchers or advisors) at all sites</td><td>- Mostly quantitative data (notably yields, N use efficiency)</td><td>- Mentioned: simulation models, databases on broad range of environments</td><td>Peoples et al. 1995,<br/>Witt et al. 1999,<br/>Savary et al. 2000,<br/>Chen et al. 2011,<br/>Van Intersum et al. 2013,<br/>Gnyp et al. 2014,</td></tr><tr><td></td><td>- Aim to be representative of farmers' practices, which are not analyzed or described in detail.</td><td>- Annual scale, no system approach</td><td>- Profitability at farm level or interactions with other practices are not assessed</td><td></td><td>Van Intersum et al. 2013,<br/>Gnyp et al. 2014,<br/>Dominant in clusters "grain yields and fertilizer management" and "laboratory and samples"</td></tr><tr><td><strong>2. Validate models in diverse contexts</strong></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>- Both on-station and on-farm sites with diverse biophysical contexts and representative of farmers' practices</td><td>- All management aspects that are not part of the treatments (e.g., fertilizer use, seeding rate, pest control) follow local farming practices</td><td>- Quantitative performances<br/>- Profitability in farm contexts<br/>- Adoption of the tested technology is the main stated objective</td><td>- Mentioned: models for calculation of fertilizer quantities; sensors such as SPAD or field sensors to calculate gaseous emissions</td><td>Tabbal et al. 2002,<br/>Dobermann et al. 2002,<br/>Fox et al. 2005,<br/>Peng et al. 2006,<br/>Cui et al. 2008,<br/>Sapkota et al. 2015,</td></tr><tr><td></td><td>- Sometimes, the local and most prevalent farmers' constraints are assessed</td><td>- Two to four multi-year treatments applied by farmers<br/>- Comparison of new sets of techniques (e.g., fertilization strategies), with mean farmers' practices (as control)</td><td></td><td></td><td>In clusters "small holder and food security" and "grain yields and fertilizer management"</td></tr><tr><td><strong>3. Compare new strategies with farmers' practices</strong></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>- Problems, goals, and activities planned by research institutions</td><td>- Problems, goals, and activities planned by research institutions</td><td>- Quantitative performances<br/>- Profitability in farm contexts<br/>- Adoption of the tested technology is the main stated objective</td><td>- Mentioned: sensors and precision farming tools</td><td>Fernandez et al. 1988,<br/>King et al. 2000,<br/>Dominant in cluster "station and weight" and "model and error"</td></tr><tr><td></td><td>- Randomized complete block designs</td><td>- Randomized complete block designs</td><td></td><td></td><td></td></tr><tr><td></td><td>- Same technology implemented on various farms</td><td></td><td></td><td></td><td></td></tr><tr><td><strong>4. Demonstrate a technology in farm fields to convince farmers</strong></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>- "Real-world" context</td><td></td><td></td><td></td><td></td></tr><tr><td></td><td>- Variety of (uncharacterized) contexts to validate robustness</td><td></td><td></td><td></td><td></td></tr></tbody></table>

<figure><img src="image_11.png" /></figure>
Table 1 (continued)

<table><thead><tr><th>Types of OFE practice</th><th>Construction of the experimental space</th><th>Interventions</th><th>Measures</th><th>Digital aspects</th><th>Examples of publications</th></tr></thead><tbody><tr><td>5. Use farm fields as research fields</td><td>- On-farm fields are used as a research site<br>- Farmers (or their practices) are not mentioned</td><td>- Implementation of techniques or devices for validation or improvement<br>- Randomized complete block designs with replicates</td><td>- Quantitative observation of the phenomena with focus on specific processes (reductionism)</td><td>- Mentioned: sensors and monitoring tool (e.g., hand-held refractometer to monitor DM and metabolite concentrations)</td><td>Graefe et al. 2004,<br>Larkin and Griffin 2007,<br>Daniel et al. 2014,<br>No predominance in a specific cluster.</td></tr><tr><td>6. On-farm research with multi-year trials</td><td>- Surveys and monitoring of on-farm activities to describe the essential characteristics of farming systems in the region<br>- Comparison of models with on-farm trials for learning purposes rather than extending the validity of simulations</td><td>- Multi-year trials, farmer-led, and negotiated<br>- Randomized block designs<br>- Protocol adapted to emerging farms' characteristics</td><td>- Quantitative and qualitative<br>- Adapted to emerging problems and farmers' needs<br>- Evaluations of learning and changes in practices</td><td>- Mentioned: simulation models for learning</td><td>Cooper et al. 1987,<br>Herweg et al. 1999,<br>Carberry et al. 2002,<br>Drewry 2006<br>In clusters "grain yields and fertilizer management" and "knowledge and innovations"</td></tr><tr><td>7. Participatory and farmer-managed trials</td><td>- Individual analysis of farming systems and practices<br>- Agronomic and socioeconomic diagnostics</td><td>- Participatory trials (farmers contribute to treatment design)<br>- Co-design of differing treatments adapted to some sites</td><td>- Diverse socioeconomic aspects of newly tested practices<br>- Description of observed and adapted farmers' practices</td><td>- Potential: experience sharing tools? Participatory assessment tools?</td><td>OUédraogo 2001,<br>Rockström et al. 2009,<br>In clusters "grain yields and fertilizer management" and "knowledge and innovations"</td></tr></tbody></table>

by Rockström et al. (2007), who applied measurements and assessed variations in widely contrasting situations to demonstrate that “crop transpiration and yield show nonlinearity under on-farm and low yield conditions.” The research methods often combined surveys (for instance to identify various constraints that farmers face, Fermont et al. 2009) and diagnoses with treatment comparisons on farms. Sometimes, various experimental treatments were applied on farms, with usual farmers’ practices as one of them. Mostly, however, the treatments applied corresponded to slight modifications of one aspect of farmers’ practices (e.g., fertilization practices and cultivar choice) in order to explore the variability of impacts of actions and of the targeted phenomenon in diverse situations. In fact, farm plots were sometimes associated with the term “environmental sampling” (Meynard et al. 1981).

In this type of OFE practice, digital tools were very seldom mentioned or applied.

### 3.2.2 Type 2: Validating models or technologies in a large range of biophysical contexts through standardized protocols

In type 2, the experimental sites on farms were selected to obtain the greatest and most representative variety of biophysical and agronomic conditions for assessing the robustness of technologies or technical models when applied in those diverse conditions. These experiments were not designed to answer new research questions but to validate previous research-based developments (models and technologies). For instance, Kanampiu et al. (2003) verified “that the herbicide seed coating technology is successful in multi-site on-station studies and especially in farmers’ fields in different conditions and environments.” Similar treatments were usually applied during a single study both at experimental stations and on farms, often on randomized block or split-plot designs, proposed and mostly managed by agronomists. This is consistent with interventions on farms to test the robustness of a defined technology. For instance, Khan et al. (2008), in their on-farm evaluation of the “push-pull” technology for the control of stemborers and striga weed on maize in western Kenya, explained that “farmers are guided by the Ministry of Agriculture and ICIPE [International Centre of Insect Physiology and Ecology] field staff [in order] to ensure that the ‘push-pull’ plots are properly laid out and companion plots properly established and managed since the effectiveness of the technology is dependent on these two.”

Digital tools were seldom mentioned unless the tested technology corresponded to a set of practices resulting from a model (Chen et al. 2011) or when on-farm data contributed to the calibration of models based on specific sensors (e.g., hyperspectral canopy sensors, Gnyp et al. 2014).

<figure><img src="image_12.png" /></figure>
### 3.2.3 Type 3: Comparing new strategies and combinations of techniques with farmers' practices

In this type of OFE practice, the trials implemented on farms were intended to promote the adoption and adaptation of proven techniques, namely, by testing them in diverse contexts under farmers' own constraints. On-farm trials ranged from researcher-led/researcher-managed to farmer-led/ farmer-managed interventions. The object of experiments on stations and farms was more often a set of combined techniques forming a strategy (e.g., site-specific nutrient management for rice fertilization, Dobermann et al. 2002) rather than isolated technologies. These OFE practices were sometimes related explicitly to an "on-farm evaluation" (Cui et al. 2008) or a "farm-scale evaluation" (Perry et al. 2003). Thus, in contrast with the preceding type of OFE, they usually included particular attention to the profitability of tested strategies or techniques in farmers' contexts and under specific constraints. Unlike type 1, the situations in which experiments took place were not characterized further than with biophysical aspects related to the strategies tested and with the most dominant farming practices. Most often, several new strategies or specific sets of practices were applied and compared (both at stations and on farms) using the equivalent of the predominant or representative farmers' practices as the control treatment. This was typically the case for several studies comparing N fertilization strategies proposed by researchers (namely, "real-time N management" and "fixed-time adjustable-dose N management", both based on researchers' monitoring with chlorophyll meters) with those of farmers in specific regions and for particular crops (Peng et al. 2006). These objectives of comparison and demonstration of clearly identified treatments were consistent with the experimental settings, which mainly corresponded to a randomized complete block design with replicates. Mentions of farmers appeared in most studies. They were reported as users of "traditional practices" as well as potential adopters of the new strategies.

The digital tools mentioned in relation to these experimental processes were mainly models for calculation of fertilization strategies and sensors either for monitoring the implementation of strategies (e.g., chlorophyll meters) or for measuring processes of interest for evaluating strategies (e.g., oxygen/carbon dioxide analyzer in an experiment on maize storage bags, Ng'ang'a et al. 2016).

### 3.2.4 Type 4: Demonstrating or testing new technologies on farm fields to convince future adopters

As with the previous type, the objective of demonstrating the value of a technique to promote farmer adoption steered the experimental practices. However, the main difference here was that the promotion goal was so emphasized that farmers'

practices or constraints were little investigated. Instead, researchers insisted on the idea of testing technologies on demonstration farms, in "real-world situations," or on "real farms" to ensure a robust assessment but without scrutinizing particular interactions with existing practices or building persuasiveness from acute comparisons with farmers' usual practices. Measurements were thus usually restricted to the technology under assessment, such as NH3 emissions for "air scrubbing techniques for ammonia and odor reduction in livestock operations" (Melse and Ogink 2005), and the saliency of their results was mostly argued in relation to the fact that they were obtained on real farms. Researchers and advisors usually designed and managed most experimental settings; as Frank et al. (2018) commented on in the case of a demonstration farm approach for pastoral livestock production systems, "farmers who own the fields often only participate passively." Much like the preceding type of OFE practice, farmers were mostly considered as adopters of demonstrated technologies.

### 3.2.5 Type 5: Considering farm fields as the locus of experiments without mentioning farmers

The experimentation took place partly on farmers' fields, but neither farmers' practices nor "real world situations" motivated the choice for this type of OFEs. Rather, the aim was to access conditions that were difficult to reproduce or obtain with reliability and relevant diversity at research stations. This was the case for disease conditions explored by Larkin et al. (2007), who chose potato farms with a history of soil-borne disease problems to experiment with brassica species as green manure. The space for experimentation was thus, above all, a relevant biophysical space for the phenomenon studied, where experimentation was feasible. Randomized complete block designs were applied in most cases. Measures and observations, as in the previous type, considered only the agronomic process under study, while farmers' constraints and difficulties to implement the experimental treatments or technologies were never mentioned.

### 3.2.6 Type 6: Developing on-farm research based on multi-year trials and surveys

In these cases, the experiments described were more explicitly embedded within long-term interventions combining different means of knowledge production. These combined, for instance, model development and implementation as learning and decision-support tools with research and development on experimental sites, interactions with farmers' collectives and advisors, and surveys. Categories of situations were often constructed to capture diversity and structure corresponding interventions. These categorizations relied mostly on surveys and analyses of main farm characteristics

<figure><img src="image_13.png" /></figure>
(e.g., size, main practices, and mean yields, Cooper et al. 1987) or of biophysical contexts (e.g., rainfall and erosivity, Herweg and Ludi 1999). The selection of experimental treatments and the design of on-farm trials were typically negotiated with local farmer groups and their private consultants or with local public extension officers, as in the case of the Farmscape program in Australia (Carberry et al. 2002). Contrary to types 3 and 4, farmers were thus much more engaged in the experimental processes, contributing either to the choice of treatments to be applied or to the implementation and observations at various sites. The trials usually lasted several years, and farmer groups engaged in monitoring, for example, of soil parameters under different treatments (e.g., different grazing systems, Drewry et al. 2006; soil and water conservation techniques, Herweg et al. 1999). This monitoring often made it possible to consider emergent issues or characteristics of particular situations during the multi-year experimental process. For instance, during the Farmscape program, the identification of “deep N bulge” throughout on-farm trial monitoring led to the reconfiguration of the purposes of these trials and of the use of simulations. Researchers consistently paid attention to the farmers’ and other actors’ learning throughout the experimental process (Carberry et al. 2002). To this end, they relied on informal interviews and on a combination of quantitative measures with “qualitative observations and statements of farmers from within and around the research sites” (Herweg et al. 1999).

Mentions of digital tools thus regarded mainly simulation models dedicated to the agronomic processes of interest and those used to favor co-learning and shared interpretations of on-farm trials to complement the qualitative observations.

### 3.2.7 Type 7: Adapting participatory and farmer-managed trials to individual farms

This type differs from the previous one regarding the extent to which farmers were co-designers of the experimental choices and settings and the attention paid to particular adaptations required in various situations for the investigated practices to be satisfying. The balance between treatments and replicates was secondary compared to the emphasis on adaptive and collaborative design of relevant treatments, as in the case of Rockström et al. (2009), where “each combination of tillage, timing, weeding, fertilization, and crop choice was agreed in farmer groups, as was the set of comparative treatments.” Blocks were still randomized but with farms considered as replicates. As adaptation to a particular situation was part of the object under investigation, experimental situations were characterized extensively. This included the analysis of biophysical, economic, and social conditions, as well as existing farmers’ practices (e.g., Ouédraogo 2001; Rockström et al. 2009). As in type 6 and unlike the other

types, adjustments of the experiments were recognizable and often made explicit. This included adjustments to both particular farming situations and those in order to take first observations or outcomes on board in adaptations of the applied protocols. New knowledge per se was not always underlined as an explicit goal of the process, but evaluations of results, adaptations in protocols, and orientations for new trials were often described as being the output of joint workshops (i.e., with farmers, advisors, and researchers).

Digital tools were seldom mentioned, and when cited mainly corresponded to crop models supporting the exploration and interpretation of practices (e.g., Stoop et al. 2002).

# 4 Discussion

## 4.1 A novel framework to analyze the diversity of scientific OFE practices

Studies on experimentations in the agricultural sector have often separated the experimental practices of researchers and those of farmers mostly in a comparative way (e.g., Catalogna et al. 2018; Hansson 2019). The descriptive tools applied in these cases are mostly well established in experimental agronomy, such as the presence or absence of “controls”, the possibility to isolate the effects of different variables, and the degree of randomization and replication. The differences between these aspects of experimentation are then often linked to distinct intents: “epistemic experiments” that are intended to produce knowledge and further the understanding of specific processes and that are opposed to “direct action-guiding experiments” that inform the effectiveness of certain actions (Hansson 2019). In contrast, we proposed here to analyze and distinguish between experiments according to the very processes of their implementation, which include the activities they require or combine from various actors involved. Our analysis shows that OFE practices, as reported in the academic literature, are widely diverse even when similar terms such as “on-farm trials” or “on-farm research” are used to define them. This was particularly apparent by the fact that we could identify very distinct “types of OFE practice” within each sub-corpus of publications associated with the clusters established during our first phase of analysis (Table 1, column “Examples of publications”). For instance, publications associated with the cluster *Grain yields and fertilizer management* included many examples of work corresponding to type 2 (*Validate models in diverse contexts*, e.g. Chen et al. 2011; Van Ittersum et al. 2013), as well as other works corresponding more closely to the contrasting type 7 (*Participatory and farmer-managed trials*, e.g., Ouédraogo et al. 2001; Stoop et al. 2002). The analytical framework we applied to publication samples across these clusters enabled the identification

<figure><img src="image_14.png" /></figure>
of major distinctive characteristics beyond these common
terms. Similarly, Salembier et al. (2018) have shown that
agronomists relied on farmers’ actual farming practice
situations in very different ways over the course of time
as agronomy as a scientific discipline evolved itself. The
types of OFE practice that we have described show that
the same diversity of approaches appears behind the OFE
umbrella. Furthermore, publications associated with the
different clusters of terms were concomitant (median year
of publication ranged between 2009 and 2014), which sug-
gests that these differing approaches have continued to
coexist in agricultural research.

Combining the analysis of the experimental space, the interventions (in terms of instruments, actions, and actors involved), observations, and measurements made it possible to identify coherence in the features of the experiments themselves. Such coherence is usually built between one aspect of the experimental process and its outcome, or based on the general aims of the experimenters. Lockeretz (1987) linked the relevance of OFE with certain objectives or requirements met by agronomists, such as to cover a range of particular soil types or other physical conditions that are not available at the experiment station, to analyze systems that involve interactions among several individual enterprises or that intrinsically are of a whole-farm nature, to evaluate production techniques that are particularly sensitive to management skills, or to analyze a production method or management system that is already practiced by some farmers but has not received attention from researchers. Sumberg et al. (2003) more particularly associated the legitimacy and utility of farmers’ participation in an experimental process with the type of technology targeted as its outcome: either commercial high tech, where farmers may have a limited role in problem identification, or “systems technologies” where farmers have important roles in problem identification and assessment of technologies early in the process in accordance with their needs and situations. Our redefined description of on-farm experimentation reverses the viewpoint, as it relates the possible outcomes of the experimentation process to the type of practice it relies on.

The intention supporting the definition of types is to better understand the diversity of logic supporting the experimental process rather than to provide criteria to judge what may (or may not) correspond to good OFE practices. The different types of OFE practice we have described show how the same kinds of devices, tools, and methods are, in fact, applied in very different ways with different objectives. However, as Lacoste et al. (2021) commented, “theoreti-cians and practitioners need to align their work conceptually, methodologically, and empirically to provide a solid and unified foundation for future efforts.” We argue that clarify-ing the various practices and approaches within the OFE community is an important step in that direction.

An example of such clarification that could be grounded using the types of OFE that we propose concerns farm-ing situations and their characterization as “contexts of on-farm experimentations.” We have described vari-ous ways in which these farming contexts are assessed, either with a focus on restricted biophysical conditions of interest (types 2 and 3) or through extended diagnoses of socioeconomic aspects and practices (types 1, 6, and 7). The latter type of description supports the understand-ing of various approaches to the variability of situations, whereas within experimental stations, this variability is most often handled as a bias reduced through replicates, and the spreading of experimental interventions on mul-tiple farms transforms the uses of induced variability in multiple ways. In type 1 (explore and explain phenom-ena), such variability is an asset for better understanding the functioning of a phenomenon occurring in each situ-ation. Thus, Meynard et al. (1981) argued, “the study of farming situations is a central part of the scientific field of agronomy; it extends and enriches the development of theoretical models used in this discipline.” In types 3 and 4, the variability within the same farming context is often what a strategy or technology is supposed to adapt to (e.g., a fertilization strategy to adapt to particular soil fertility states and their dynamics), based on the given formal-ism and model of the involved biophysical processes. This resonates with the “option by context” approach proposed by Sinclair and Coe (2019). In type 2, the variability is maximized as a support for testing the robustness of a technology or practice without the need for characterizing situations individually or linking specific results of trials with specific situations (unless it is to identify unexpected data points), whereas adaptation of technologies to local conditions is a principle directly associated with agroeco-logical approaches (Bell et al. 2008; Tittonell et al. 2020), and the concrete and illustrated ways to handle it are too seldom discussed in the literature (Nelson et al. 2019; Sin-clair and Coe 2019; Salembier et al. 2021). However, the various types of OFE practice require this.

Finally, the types of OFE practice can help research-
ers analyzing and supporting farmers’ own experiments.
As Kummer et al. (2017) commented, farmers’ experi-
ments have received little attention from agronomists, and
mostly in countries of the Global South. Researchers may
describe and formalize farmers’ experimental practices so
as to stimulate experimentation in another farmers’ activity
(Catalogna et al. 2018). Types 6 and 7 also show how more
attention to these experiments can be paid in combination
with a researcher-initiated experimental process either by
including socioeconomic dimensions and practices in initial
diagnoses preceding OFE (Cooper et al. 1987) or by includ-
ing independent farmers’ evaluations in the assessments of
experiments (Rockström et al. 2009) (Table 1).
## 4.2 Diverse forms of digitalization suggested by the various types of OFE practice

Digital technologies sometimes associated with the term “Agriculture 4.0” include many different kinds of devices such as drones, the internet of things (IoT), robotics, and sensors connected to precision farming technology, artificial intelligence, machine learning, and blockchains (Klerkx and Rose 2020). The application of such tools in the seven types of OFE practice was most often implicit, except in types 4 and 5 where sensors and precision farming technologies were tested and developed further. The development of simulations and models was closely related to various OFE practices but with distinct objectives: either to improve robustness through application in a wide range of environments (e.g., type 2 or 4) or to support learning (e.g., type 6). The relatively weak resonance of digital transformations in agriculture in our corpus is probably to be attributed to the time period and selection criteria for in-depth analysis, which excluded the most recent publications (only 3 publications from 2015 or later). There is no doubt, however, that the digitalization of agriculture is closely connected to developments in on-farm experimentations (Piepho et al. 2011; Laurent et al. 2020; Lacoste et al. 2021). Digitalization still refers mostly to big data technologies and precision agriculture (Rotz et al. 2019; Ingram et al. 2022). In fact, the development of tools derived from information and communication technologies (e.g., virtual spaces for information exchanges and media for recording observations) has long been associated with experiments on farms (Wolfert et al. 2011). Smart farming has renewed the potential for a range of tools currently in use, such as smart sensing and monitoring (i.e., acquiring more numerous and accurate data points on farms for better decision making), smart analysis and planning (i.e., management and decision tools that ground calculations on more interconnected and enriched information on the farm’s biophysical and economic data), and smart control (i.e., precision farming) (Wolfert et al. 2017).

In contrast, the diverse types of OFE practice we identified invite us to focus awareness on two major issues. First, while the main developments of digital tools for OFE are based on the assumption that all variables of interest to be monitored by digital tools should be known, along with the best data to inform them, this does not fit OFE practices where some of the variables to be explored emerge during the experimental process (namely in types 6 and 7). During the multi-year experiments that support a step-by-step redesign of cropping systems, for instance, the most useful observations for interpreting the effects of actions often emerge from the first outcomes of new practices and are re-assessed after connecting several observations (e.g., the vegetation architecture of peas in the flowering stage is only interpreted after having progressively established several

relationships with sowing density, fertility, and physical states of soils with different preceding crops or yields finally reached) (Toffolini et al. 2015). This may occur in relation to the exploration of a phenomenon in contrasting situations (type 1, Soto-Pinto et al. 2000) or in relation to the adaptation of protocols and their adjustments to situations with farmers (types 6 and 7, e.g., Carberry et al. 2002). More generally, it points to the risk associated with the paradoxically reduced exploration of reality through experimentation: what is not included in digitally targeted data falls out of the scope of emerging sources of knowledge.

Second, very few studies and reviews on the development of digital tools in the agricultural sector highlight the possibility for new digital tools to support social interactions and learning among the diverse actors involved in on-farm experimentation processes (Leveau et al. 2019). For instance, digital tools could be tailored to store and provide access to the serendipity of collective activities (e.g., analyzing and visualizing social and interpretative interactions during a workshop), visualize qualitative data and situated interpretations (e.g., concerning the sharing of individual experiments on farms), and connect existing information resources based on a query by farmers rather than fine-tuning the individualized advice. Some examples have appeared recently, such as digital platforms for sharing maps and descriptions of on-farm innovations regarding equipment or buildings (Chance and Meyer 2017), or for sharing techniques and experiences related to the valorization of natural vegetation in production (Girard and Magda 2020).

These digital tools need to integrate diverse dimensions to support meaningful comparisons and analogies across farming situations if they are to derive generic knowledge from individual and anecdotic situations. This calls for specific research on their design (Quinio et al. 2022). For instance, the tools should offer support for more heterogeneous databases (including qualitative observations). They could also offer media that enhance the exchanges and collective interpretations of situations experienced, with a view to supporting innovation in other contexts (Elzen et al. 2017). More specifically, digital tools that support farmers’ interactions and exchange of observations made through OFE are part of these potential developments and could draw on recent works on farmers’ use of online communities and social media (Prost et al. 2017).

## 4.3 Alignments between OFE practices and agricultural innovation approaches

We observed some alignment between the types of OFE practice identified through the present analysis and various approaches to agricultural innovation that emerged over time. On the one hand, types 3 and 4 (*Comparing new strategies and combinations of techniques with farmers*

<figure><img src="image_15.png" /></figure>
practices, and *Demonstrating or testing new technologies in farm fields to convince future adopters*) could be related to a diffusion model or technology transfer approach (Hall 2007; Klerkx et al. 2012). On the other hand, the emphasis on collecting farm data including socio-technical information in types 1 and 6 (*Exploring and explaining a phenomenon through a diagnosis of diverse farmers' practices*, and *Developing on-farm research based on multi-year trials and surveys*) illustrate the Farming Systems Research stream, which purposively placed farms and farmers groups within their direct biophysical and socioeconomic contexts in order to develop social learning. Such approaches draw on the idea that to achieve agroecological transitions, the technologies designed must fit specific farming situations. Questions arise regarding means for sharing situated knowledge and experiences and associated data privacy and intellectual property issues. These questions are, all the more, acute that open innovation approaches develop in the agricultural sector and within the OFE research community (Berthet et al. 2018; Salembier et al. 2020; Lacoste et al. 2021).

Finally, OFE practices that correspond to farmer participatory research with agronomic and socioeconomic diagnoses (type 7) may be related to an agricultural innovation system approach that is more oriented toward the development of capacities for innovation (ibid.).

The analysis performed here is, however, not sufficient to fully relate the seven types of OFE practice to innovation system theories and approaches and would require a wider analysis of OFE practices in their institutional contexts. First, OFE practices could be more widely situated with larger corpuses of literature, including more research using terms related to participation rather than being limited to the on-farm locus of experimental interventions. This enlargement could also target less academic literature, in languages other than English, and include more development practices. Second, a more institution-focused analysis of mobilization and application of OFE concepts would require the collection of more and different information than that provided by the reviewed articles, for instance, information on institutional arrangements and actors' interventions in the experiments and around their implementation and use, with a view to better understand the various contributions of experiments to innovation, as proposed by Salembier et al. (2021). This is a sound research perspective for further mapping the realities of OFE practices in various innovation settings. Deciphering OFE practices, focusing on their pragmatic realities, and focusing on literature expressly referring to the on-farm location for experimental processes are the first step that calls for broadening the inquiry and discussion on how OFE is institutionalized and to refine or renew research practices and innovation policies that contribute to shaping innovation processes.

# 5 Conclusion

Our aim was first to characterize the wide variety of practices gathered under the banner of on-farm experimentation. The literature review process and analytical framework presented here provide a synthetic understanding of a wide range of practices and how these are organized. The two-step methodology, joining a scientometric approach with a qualitative analysis of the literature, provided a comprehensive and original deciphering of seven types of on-farm experimentation practices based on the treatments applied, ways to consider farmers' existing practices and socioeconomic contexts, distribution of responsibilities among the actors involved, and resulting learning, whether targeted or not. It appeared that digital technologies other than those related to precision agriculture and simulation models were not often discussed or envisioned, whereas these could support participatory and long-term on-farm experimentation practices (e.g., knowledge exchange support tools, repertories of experiences, and designs applied on farms and their situated evaluations). Further refinements to describe OFE practices need to be developed to inform a collective reflection within emerging research communities on the appropriate positioning and types of digital technologies to support and, especially, to engage in collective inquiry into these issues with broader communities of stakeholders and citizens. Specifying how and what we name behind the keywords associated with OFE should help to keep the wide variety of approaches in the debate, maintaining all possibilities open and legitimate instead of closing innovation paths around the most advanced digital technologies. The contributions to agroecological transitions of the different OFE practices identified also need to be discussed.

**Acknowledgements** We are grateful to colleagues from the CorText team (https://www.cortex.net/) for the previous help and training on corpus analysis, as well as Liz Carey Libbrecht for language editing the English version. This work has also benefited from discussions within the IDEAS (Initiative for DEsign in Agrifood Systems) research community.

**Authors' contributions** Conceptualization, Q.T. and M-H.J.; methodology, Q.T.; data acquisition, Q.T.; writing (first draft) Q.T. All the authors commented on several versions of the manuscript and approved the final version of the manuscript.

**Funding** The publication of this article was funded by "#OFE2021, the first Conference on farmer-centric On-Farm Experimentation – Digital Tools for a Scalable Transformative Pathway" through contributions made by the OECD Co-operative Research Program for "Biological resource management for sustainable agricultural systems – Transformational technologies and innovation", INRAE, the Digital Agriculture Convergence Lab #DigitAg (ANR-16-CONV-0004), and the OFE-C (on-farm experimentation community) of the International Society for Precision Agriculture.

**Data availability** The datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request.

<figure><img src="image_16.png" /></figure>
**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

## References

Alesso CA, Cipriotti PA, Bollero GA, Martin NF (2021) Design of on-farm precision experiments to estimate site-specific crop responses. Agron J 113:1366-1380. https://doi.org/10.1002/agj2.20572

Bell MM, Lyon A, Gratton C, Jackson RD (2008) Commentary: the productivity of variability: an agroecological hypothesis. Int J Agric Sustain 6:233-235. https://doi.org/10.3763/ijas.2008.c5004

Berthet ET, Hickey GM, Klerkx L (2018) Opening design and innovation processes in agriculture: insights from design and management sciences and future directions. Agric Syst 165:111-115. https://doi.org/10.1016/j.agsy.2018.06.004

Boiffin J, Caneill J, Meynard JM, Sebillotte M (1981) Elaboration du rendement et fertilisation azotée du blé d'hiver en Champagne crayeuse I. - Protocole et méthode d'étude d'un problème technique régional. Agronomie 1:549

Bullock DS, Boerngen M, Tao H, et al (2019) The data-intensive farm management project: changing agronomic research through on-farm precision experimentation. Agron J 111:2736-2746. https://doi.org/10.2134/agronj2019.03.0165

Caniglia G, Schäpke N, Lang DJ et al (2017) Experiments and evidence in sustainability science: a typology. J Clean Prod 169:39-47. https://doi.org/10.1016/j.jclepro.2017.05.164

Carberry PS, Hochman Z, McCown RL et al (2002) The FARMSCAPE approach to decision support: farmers', advisers', researchers' monitoring, simulation, communication and performance evaluation. Agric Syst 74:141-177. https://doi.org/10.1016/S0308-521X(02)00025-2

Casler MD, Undersander DJ, Fredericks C et al (1998) An on-farm test of perennial forage grass varieties under management intensive grazing. J Prod Agric 11:92-99. https://doi.org/10.2134/jpa1998.0092

Catalogna M, Dubois M, Navarrete M (2018) Diversity of experimentation by farmers engaged in agroecology. Agron Sustain Dev 38:50. https://doi.org/10.1007/s13593-018-0526-2

Chambers R, Ghildyal BP (1985) Agricultural research for resource-poor farmers: the farmer-first-and-last model. Agric Adm 20:1-30. https://doi.org/10.1016/0309-586X(85)90063-9

Chance Q, Meyer M (2017) L'agriculture libre. Les outils agricoles à l'épreuve de l'open source. Techniques & Culture Revue semestrielle d'anthropologie des techniques

Chen X-P, Cui Z-L, Vitousek PM et al (2011) Integrated soil-crop system management for food security. Proc Natl Acad Sci 108:6399-6404. https://doi.org/10.1073/pnas.1101419108

Cointet J-P (2009) Dynamiques sociales et sémantiques dans les communautés de savoirs, morphogenèse et diffusion. Phdthesis, Ecole Polytechnique

Compagnone C, Lamine C, Dupré L (2018) La production et la circulation des connaissances en agriculture interrogées par l'agro-écologie. Revue d'anthropologie des connaissances Vol. 12, N°2:111-138

Cook S, Jackson EL, Fisher MJ et al (2021) Embedding digital agriculture into sustainable Australian food systems: pathways and pitfalls to value creation. Int J Agric Sustain 0:1-22. https://doi.org/10.1080/14735903.2021.1937881

Cooper PJM, Gregory PJ, Tully D, Harris HC (1987) Improving water use efficiency of annual crops in the rainfed farming systems of West Asia and North Africa. Ex Agric 23:113-158. https://doi.org/10.1017/S001447970001694X

Cui Z, Zhang F, Chen X et al (2008) On-farm evaluation of an in-season nitrogen management strategy based on soil Nmin test. Field Crop Res 105:48-55. https://doi.org/10.1016/j.fcr.2007.07.008

Daniel C, Mathis S, Feichtinger G (2014) A new visual trap for Rhagoletis cerasi (L.) (Diptera: Tephritidae). Insects 5:564-576. https://doi.org/10.3390/insects5030564

Dobermann A, Witt C, Dawe D et al (2002) Site-specific nutrient management for intensive rice cropping systems in Asia. Field Crop Res 74:37-66. https://doi.org/10.1016/S0378-4290(01)00197-6

Drewry JJ (2006) Natural recovery of soil physical properties from treading damage of pastoral soils in New Zealand and Australia: A review. Agric Ecosyst Environ 114:159-169. https://doi.org/10.1016/j.agee.2005.11.028

Elzen B, Janssen A, Bos A (2017) Portfolio of promises: designing and testing a new tool to stimulate transition towards sustainable agriculture. AgroEcological transitions Wageningen University & Research

Engels F, Wentland A, Pfotenhauer SM (2019) Testing future societies? Developing a framework for test beds and living labs as instruments of innovation governance. Res Policy 48:103826. https://doi.org/10.1016/j.respol.2019.103826

Fernandez BB, Tumapon AS, Duna LV, Balanay NM, Kloos JP, Zaag PV (1988) On-farm evaluation of true potato seed in the Philippines. American Potato Journal 65:457-461. https://doi.org/10.1007/BF02854360

Fox P, Rockström J, Barron J (2005) Risk analysis and economic viability of water harvesting for supplemental irrigation in semi-arid Burkina Faso and Kenya. Agric Syst 83:231-250. https://doi.org/10.1016/j.agsy.2004.04.002

Fermont AM, van Asten PJA, Tittonell P et al (2009) Closing the cas-sava yield gap: an analysis from smallholder farms in East Africa. Field Crop Res 112:24-36. https://doi.org/10.1016/j.fcr.2009.01.009

Frank M, Easdale MH, Kaufmann B (2018) Assessing a demonstration farm approach for technological innovations in pastoral livestock production systems of northern Patagonia: participants' perceptions of stakeholder roles and innovations. Exp Agric 54:774-793. https://doi.org/10.1017/S0014479717000448

Ghadim AKA, Pannell DJ, Burton MP (2005) Risk, uncertainty, and learning in adoption of a crop innovation. Agric Econ 33:1-9. https://doi.org/10.1111/j.1574-0862.2005.00433.x

Girard N, Magda D (2020) The interplays between singularity and genericity of agroecological knowledge in a network of livestock farmers. J Rural Stud 73:214-224. https://doi.org/10.1016/j.jrurs.tud.2019.11.003

<figure><img src="image_17.png" /></figure>
Gnyp ML, Miao Y, Yuan F et al (2014) Hyperspectral canopy sensing of paddy rice aboveground biomass at different growth stages. Field Crop Res 155:42–55. https://doi.org/10.1016/j.fcr.2013.09.023

Graefe S, Hermann M, Manrique I, Golombek S, Buerkert A (2004) Effects of post-harvest treatments on the carbohydrate composition of yacon roots in the Peruvian Andes. Field Crop Res 86:157–165. https://doi.org/10.1016/j.fcr.2003.08.003

Hall A (2007) Challenges to strengthening agricultural innovation systems: where do we go from here? United Nations University - Maastricht Economic and Social Research Institute on innovation and technology (MERIT).

Hansson SO (2019) Farmers’ experiments and scientific methodology. Euro Jnl Phil Sci 9:32. https://doi.org/10.1007/s13194-019-0255-7

Henke CR (2000) Making a place for science: the field trial. Soc Stud Sci 30:483–511. https://doi.org/10.1177/030631200030004001

Herweg K, Ludi E (1999) The performance of selected soil and water conservation measures—case studies from Ethiopia and Eritrea. CATENA 36:99–114. https://doi.org/10.1016/S0341-8162(99)00004-1

Ingram J, Maye D, Bailye C et al (2022) What are the priority research questions for digital agriculture? Land Use Policy 114:105962. https://doi.org/10.1016/j.landusepol.2021.105962

Jas N (2000) Au Carrefour de la chimie et de l'agriculture: les sciences agronomiques en France et en Allemagne, 1850[sic]-1914. In: Éditions des archives contemporaines (EAC), Paris

Jouve P (2007) In: Aeschlimann JP, Feller C, Renoir S (eds) Périodes et ruptures dans l'évolution des savoirs agronomiques et de leur enseignement. Histoire et agronomie: entre ruptures et durée P Robin. IRD, Paris, pp 109–120

Kanampiu FK, Kabambe V, Massawe C et al (2003) Multi-site, multi-season field tests demonstrate that herbicide seed-coating herbicide-resistance maize controls Striga spp. and increases yields in several African countries. Crop Prot 22:697–706. https://doi.org/10.1016/S0261-2194(03)00007-3

Khan ZR, Midega CAO, Amudavi DM et al (2008) On-farm evaluation of the “push-pull” technology for the control of stemborers and striga weed on maize in western Kenya. Field Crop Res 106:224–233. https://doi.org/10.1016/j.fcr.2007.12.002

King LM, Kirby JD, Froman DP, Sonstegard TS, Harry DE, Darden JR, Marini PJ, Walker RM, Rhoads ML, Donoghue AM (2000) Efficacy of sperm mobility assessment in commercial flocks and the relationships of sperm mobility and insemination dose with fertility in turkeys. Poult Sci 79:1797–1802. https://doi.org/10.1093/ps/79.12.1797

Klerkx L, Rose D (2020) Dealing with the game-changing technologies of agriculture 4.0: how do we manage diversity and responsibility in food system transition pathways? Global. Food Security 24:100347. https://doi.org/10.1016/j.gfs.2019.100347

Klerkx L, van Mierlo B, Leeuwis C (2012) Evolution of systems approaches to agricultural innovation: concepts, analysis and interventions. In: Darnhofer I, Gibbon D, Dedieu B (eds) Farming systems research into the 21st century: the new dynamic. Springer, Netherlands, Dordrecht, pp 457–483

Kummer S, Leitgeb F, Vogl CR (2017) Farmers’ own research: organic farmers’ experiments in Austria and implications for agricultural innovation systems. SAR 6:103. https://doi.org/10.5539/sar.v6n1p103

Kyveryga PM (2019) On-farm research: experimental approaches, analytical frameworks, case studies, and impact. Agron J 111:2633–2635. https://doi.org/10.2134/agronj2019.11.0001

Lacoste M, Cook S, McNee M, et al (2021) On-farm experimentation to transform global agriculture. Nat Food 1–8. https://doi.org/10.1038/s43016-021-00424-4

Larkin RP, Griffin TS (2007) Control of soilborne potato diseases using brassica green manures. Crop Prot 26:1067–1077. https://doi.org/10.1016/j.cropro.2006.10.004

Laurent B, Tironi M (2015) A field test and its displacements. Accounting for an experimental mode of industrial innovation. CoDesign 11:208–221. https://doi.org/10.1080/15710882.2015.1081241

Laurent A, Miguez F, Kyveryga P, Makowski D (2020) Going beyond mean effect size: presenting prediction intervals for on-farm network trial analyses. Eur J Agron 120:126127. https://doi.org/10.1016/j.eja.2020.126127

Leveau L, Bénel A, Cahier J-P et al (2019) Information and communication technology (ICT) and the agroecological transition. In: Bergez J-E, Audouin E, Therond O (eds) Agroecological transitions: from theory to practice in local participatory design. Springer International Publishing, Cham, pp 263–287

Lockertetz W (1987) Establishing the proper role for on-farm research. Am J Altern Agric 2:132–136. https://doi.org/10.1017/S088918930000179X

Lovell H, Hann V, Watson P (2018) Rural laboratories and experiment at the fringes: a case study of a smart grid on Bruny Island, Australia. Energy Res Soc Sci 36:146–155. https://doi.org/10.1016/j.erss.2017.09.031

Maat H (2011) The history and future of agricultural experiments. NJAS-Wageningen J Life Sci 57:187–195. https://doi.org/10.1016/j.njas.2010.11.001

Maat H, Glover D (2012) Alternative configurations of agronomic experimentation. In: Contested agronomy. Routledge

Melse RW, Ogink NWM (2005) Air scrubbing techniques for ammonia and odor reduction at livestock operations: review of on-farm research in the Netherlands. Trans ASAE 48:2303–2313. https://doi.org/10.13031/2013.20094

Meynard J-M, Boiffin J, Caneill J, Sebillotte M (1981) Elaboration du rendement et fertilisation azotée du blé d'hiver en champagne crayeuse II. - types de réponse à la fumure azotée et application de la méthode du bilan prévisionnel. Agronomie 1:795–806. https://doi.org/10.1051/agro:19810912

Meynard JM, Dedieu B, Bos AP (Bram) (2012) Re-design and co-design of farming systems. An overview of methods and practices. In: Darnhofer I, Gibbon D, Dedieu B (eds) Farming systems research into the 21st century: the new dynamic. Springer Netherlands, pp 405–429

Moinard C, Mendl M, Nicol CJ, Green LE (2003) A case control study of on-farm risk factors for tail biting in pigs. Appl Anim Behav Sci 81:333–355. https://doi.org/10.1016/S0168-1591(02)00276-9

Nelson R, Coe R, Haussmann BIG (2019) Farmer research networks as a strategy for matching diverse options and contexts in small-holder agriculture. Exp Agric 55:125–144. https://doi.org/10.1017/S0014479716000454

Ng'ang'a J, Mutungi C, Imathiu SM, Affognon H (2016) Low permeability triple-layer plastic bags prevent losses of maize caused by insects in rural on-farm stores. Food Sec 8:621–633. https://doi.org/10.1007/s12571-016-0567-9

Ouédraogo E (2001) Use of compost to improve soil properties and crop productivity under low input agricultural system in West Africa. Agric Ecosyst Environ 84:259–266. https://doi.org/10.1016/S0167-8809(00)00246-2

Parolini G (2015) Charting the history of agricultural experiments. Hist Philos Life Sci 37:231–241. https://doi.org/10.1007/s40656-015-0079-5

Peoples MB, Bowman AM, Gault RR, Herridge DF, McCallum MH, McCormick KM, Norton RM, Rochester IJ, Scammell GJ, Schwenke GD (2001) Factors regulating the contributions of fixed nitrogen by pasture and crop legumes to different farming

<figure><img src="image_18.png" /></figure>
systems of eastern Australia. Plant Soil 228:29-41. https://doi.org/
10.1023/A:1004799703040

Peng S, Buresh RJ, Huang J et al (2006) Strategies for overcoming low agronomic nitrogen use efficiency in irrigated rice systems in China. Field Crop Res 96:37-47. https://doi.org/10.1016/j.fcr. 2005.05.004

Peng S, Khush GS, Virk P et al (2008) Progress in ideotype breeding to increase rice yield potential. Field Crop Res 108:32-38. https://doi.org/10.1016/j.fcr.2008.04.001

Périnelle A, Meynard J-M, Scopel E (2021) Combining on-farm innova-
tion tracking and participatory prototyping trials to develop
legume-based cropping systems in West Africa. Agric Syst
187:102978. https://doi.org/10.1016/j.agsy.2020.102978

Perry JN, Rothery P, Clark SJ et al (2003) Design, analysis and statistical power of the farm-scale evaluations of genetically modified herbicide-tolerant crops. J Appl Ecol 40:17-31. https://doi.org/10.1046/j.1365-2664.2003.00786.x

Piepho H-P, Richter C, Spilke J et al (2011) Statistical aspects of on-farm experimentation. Crop Pasture Sci 62:721-735. https://doi.org/10.1071/CP11175

Prost M, Prost L, Cerf M (2017) Les échanges virtuels entre agri-
culteurs: un soutien à leurs transitions professionnelles? Raisons
éducatives:129-154

Quinio M, Guichard L, Salazar P et al (2022) Cognitive resources to promote exploration in agroecological systems design. Agric Syst 196:103334. https://doi.org/10.1016/j.agsy.2021.103334

Rockström J, Barron J (2007) Water productivity in rainfed systems: overview of challenges and analysis of opportunities in water scarcity prone savannahs. Irrig Sci 25:299-311. https://doi.org/10.1007/s00271-007-0062-3

Rockström J, Kaumbutho P, Mwalley J et al (2009) Conservation farming strategies in east and southern Africa: yields and rain water productivity from on-farm action research. Soil Tillage Res 103:23-32. https://doi.org/10.1016/j.still.2008.09.013

Rotz S, Duncan E, Small M et al (2019) The politics of digital agri-
cultural technologies: a preliminary review. SociologiaRuralis
59:203-229. https://doi.org/10.1111/soru.12233

Salembier C, Segrestin B, Berthet E et al (2018) Genealogy of design reasoning in agronomy: lessons for supporting the design of agricultural systems. Agric Syst 164:277-290. https://doi.org/10.1016/j.agsy.2018.05.005

Salembier C, Segrestin B, Sinoir N et al (2020) Design of equipment for agroecology: coupled innovation processes led by farmer-designers. Agric Syst 183:102856. https://doi.org/10.1016/j.agsy.2020.102856

Salembier C, Segrestin B, Weil B et al (2021) A theoretical framework for tracking farmers’ innovations to support farming system design. Agron Sustain Dev 41:61. https://doi.org/10.1007/s13593-021-00713-z

Sapkota TB, Jat ML, Aryal JP, Jat RK, Khatri-Chhetri A (2015) Climate change adaptation, greenhouse gas mitigation and economic profitability of conservation agriculture: Some examples from cereal systems of Indo-Gangetic Plains. J Integr Agr 14:1524-1533. https://doi.org/10.1016/S2095-3119(15)61093-0

Savary S, Willocquet L, Elazegui FA, Castilla NP, Teng PS (2000) Rice pest constraints in Tropical Asia: quantification of yield losses due to rice pests in a range of production situations. Plant Dis 84:357-369. https://doi.org/10.1094/PDIS.2000.84.3.357

Schäpke N, Stelzer F, Caniglia G et al (2018) Jointly experimenting for transformation? Shaping real-world laboratories by comparing them. GAIA - ecological perspectives for. Sci Soc 27:85-96. https://doi.org/10.14512/gaia.27.S1.16

Sims JT, Luka-McCafferty NJ (2002) On-Farm Evaluation of Alu-
minum Sulfate (Alum) as a Poultry Litter Amendment. J Environ
Qual 31:2066-2073. https://doi.org/10.2134/jeq2002.2066

Sinclair F, Coe R (2019) The options by context approach: a paradigm shift in agronomy. Exp Agric 55:1-13. https://doi.org/10.1017/S0014479719000139

Soto-Pinto L, Perfecto I, Castillo-Hernandez J, Caballero-Nieto J (2000) Shade effect on coffee production at the northern Tzel-tal zone of the state of Chiapas, Mexico. Agric Ecosyst Environ 80:61-69. https://doi.org/10.1016/S0167-8809(00)00134-1

Stoop WA, Uphoff N, Kassam A (2002) A review of agricultural research issues raised by the system of rice intensification (SRI) from Madagascar: opportunities for improving farming systems for resource-poor farmers. Agric Syst 71:249-274. https://doi.org/10.1016/S0308-521X(01)00070-1

Sumberg J, Okali C, Reece D (2003) Agricultural research in the face of diversity, local knowledge and the participation imperative: theoretical considerations. Agric Syst 76:739-753. https://doi.org/10.1016/S0308-521X(02)00153-1

Tabbal DF, Bouman BAM, Bhuiyan SI, Sibayan EB, Sattar MA (2002) On-farm strategies for reducing water input in irrigated rice; case studies in the Philippines. Agric Water Manag 56:93-112. https://doi.org/10.1016/S0378-3774(02)00007-0

Tiltonell P, Piñeiro G, Garibaldi LA et al (2020) Agroecology in large scale farming—a research agenda. Front Sustainable Food Syst 4. https://doi.org/10.3389/fsufs.2020.584605

Toffolini Q, Jeuffroy M-H, Prost L (2015) Indicators used by farmers to design agricultural systems: a survey. Agron Sustain Dev 36:1-14. https://doi.org/10.1007/s13593-015-0340-z

Van Ittersum MK, Cassman KG, Grassini P, Wolf J, Tiltonell P, Hochman Z (2013) Yield gap analysis with local to global relevance—A review. Field Crop Res 143:4-17. https://doi.org/10.1016/j.fcr.2012.09.009

Weeds J, Weir D (2005) Co-occurrence retrieval: a flexible framework for lexical distributional similarity. Comput Linguistics 31:439-475. https://doi.org/10.1162/089120105775299122

Witt C, Dobermann A, Abdulrachman S, Gines HC, Guanghuo W, Nagarajan R, Satawatananont S, Thuc Son T, Sy Tan P, Van Tiem L, Simbahan GC, Olk DC (1999) Internal nutrient efficiencies of irrigated lowland rice in tropical and subtropical Asia. Field Crop Res 63:113-138. https://doi.org/10.1016/S0378-4290(99)00031-3

Wolfert J, Kruize JW, Verdouw CN, Beulens AJM (2011) Agri-food living lab: the virtual meeting place for open innovation on farm information management and ICT development. In: Conference Proceedings 18th International Farm Management Congress, Christchurch, New Zealand, 20-25 March, 2011. pp 496-504

Wolfert S, Ge L, Verdouw C, Bogaardt M-J (2017) Big data in smart farming – a review. Agric Syst 153:69–80. https://doi.org/10.1016/j.agsy.2017.01.023

**Publisher's note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_19.png" /></figure>