# Co-production opportunities seized and missed in decision-support frameworks for climate-change adaptation in agriculture – How do we practice the “best practice”?

Aleksandra Dolinska, Emeline Hassenforder, Ana Maria Loboguerrero, Benjamin Sultan, Jérôme Bossuet, Jeanne Cottenceau, Michelle Bonatti, Jon Hellin, Insaf Mekki, Alexis Drogoul, et al.

► To cite this version:

Aleksandra Dolinska, Emeline Hassenforder, Ana Maria Loboguerrero, Benjamin Sultan, Jérôme Bossuet, et al. Co-production opportunities seized and missed in decision-support frameworks for climate-change adaptation in agriculture – How do we practice the “best practice”?. Agricultural Systems, 2023, 212, pp.103775. 10.1016/j.agsy.2023.103775. hal-04279659

HAL Id: hal-04279659

https://hal.inrae.fr/hal-04279659

Submitted on 25 Apr 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_1.png" /></figure>

Check for
updates

# Co-production opportunities seized and missed in decision-support frameworks for climate-change adaptation in agriculture – How do we practice the “best practice”?

Aleksandra Dolinska<sup>a,b,*</sup>, Emeline Hassenforder<sup>c,d,e</sup>, Ana Maria Loboguerrero<sup>f</sup>, Benjamin Sultan<sup>b</sup>, Jérôme Bossuet<sup>a,g</sup>, Jeanne Cottenceau<sup>h,i</sup>, Michelle Bonatti<sup>j,k</sup>, Jon Hellin<sup>l</sup>, Insaf Mekki<sup>m</sup>, Alexis Drogoul<sup>h,i</sup>, Vincent Vadez<sup>a,n</sup>

## HIGHLIGHTS

* Knowledge co-production is largely suggested as the best practice to bridge the gaps between science, policy and practice.

* Evidence is lacking for the actual use of co-production in decision-support for climate-change adaptation in agriculture.

* We identify useful entry points for applying co-production principles from already used decision-support frameworks.

* Researchers are rarely selected to build decision-support frameworks based on their ability to support co-production.

* When designing decision-support frameworks co-production principles can be used as a guiding tool.

## GRAPHICAL ABSTRACT

Co-production opportunities seized and missed in decision-support frameworks for climate-change adaptation in agriculture – how do we practice the “best practice”?

<figure><img src="image_2.png" /></figure>

* Co-production in presented as the best practice to arrive at sustainable solutions to adapt to climate change

* Co-production is recommended to avoid usability gap in decision-support

<figure><img src="image_3.png" /></figure>

Is co-production applied in multi-stakeholder decision-support frameworks for climate-change adaptation? How exactly?

<figure><img src="image_4.png" /></figure>

Literature review for decision-support frameworks for climate-change adaptation actively involving non-academic stakeholders

18 applied frameworks analyzed in detail for entry-points for co-production principles

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

### examples of identified / underrepresented or missing entry-points

* <u>coastal</u> - exchange and use, endogenous adaptation strategies...

* <u>plurispheric</u> - responding to the direct local demands...

* <u>interregional</u> - having a selection strategy aiming at diversity, creating interaction spaces for groups with competing views...

* <u>non-downloading altitude, etc.</u> - taking into account farmers' specific knowledge when selecting farmers when building a research team, also using selection criteria (reliability for mutual learning...)

* <u>gadolin</u> - discussing values underlying participatory goals...

* <u>co-creation</u> - developing a theory of change with the participants...

* <u>co-education</u> - discussing values underlying participatory goals...

* <u>co-implication</u> - discussing other assumptions at all stages of the process...

* <u>co-impact</u> - inviting participants to produce their own tools of analysis, maps, calendars, matrices...

* <u>co-implication</u> - if the project involves citizen science, putting traditional/indigenous knowledge at the same level as scientific knowledge, for example by creating spaces to upload it on the on-line tools setting the participants determine their own role/level of implication in the process

### CONCLUSIONS
Members of the research team should be perceived as stakeholders in the co-production process with similar expectations and roles applied to them as to the other stakeholders

* The four co-production principles proposed by Nøstrom can be used as a check-list to guide the process of co-construction of decision support

* The potential of decision-support frameworks to tackle the three dimensions: science, policy and practice should be explored

<footnote><sup>a</sup> French National Research Institute for Sustainable Development (IRD), UMR DIADE, University of Montpellier, Montpellier, France</footnote>

<footnote><sup>b</sup> French National Research Institute for Sustainable Development (IRD), UMR ESPACE-DEV, University of Montpellier, Montpellier, France</footnote>

<footnote><sup>c</sup> CIRAD, UMR G-EAU, 01800 Tunis, Tunisia</footnote>

<footnote><sup>d</sup> G-EAU, Univ Montpellier, AgroParisTech, CIRAD, INRAE, Institut Agro, IRD, Montpellier, France</footnote>

<footnote><sup>e</sup> Institut National Agronomique de Tunisie (INAT), Tunisia</footnote>

<footnote><sup>f</sup> Bioversity International, Via di San Domenico 1, 00153 Rome, Italy</footnote>

<footnote><sup>g</sup> Science Communication Consultant, Exeter, GB, United Kingdom</footnote>

<footnote><sup>h</sup> ACROSS International Joint Laboratory IRD/Thuyloi University, 175 Tay Son, Hanoi, Viet Nam</footnote>

<footnote><sup>i</sup> UMI UMMISCO (IRD/Sorbonne Université), 32 Av. H. Varagnat Bondy Cedex, France</footnote>

<footnote><sup>j</sup> Leibniz Centre for Agricultural Landscape Research (ZALF e. V.), Müncheberg, Germany</footnote>

<footnote><sup>k</sup> Department of Agricultural Economics, Humboldt University of Berlin, Berlin, Germany</footnote>

<footnote><sup>l</sup> International Rice Research Institute (IRRI), Philippines</footnote>

<footnote><sup>m</sup> INRGREF, University of Carthage, B.P. 10, 2080 Ariana, Tunisia</footnote>

<footnote><sup>n</sup> Centre d'Etude Régionale sur l'amélioration de l'Adaptation à la Sècheresse (CERAAS), Laboratoire Mixte International, Campus ENSA, Senegal</footnote>

<footnote>* Corresponding author at: 11 rue de la Fontaine, 30440 Saint Laurent le Minier, France.
E-mail address: ola.dolinska@protonmail.com (A. Dolinska).</footnote>
<figure><img src="image_2.png" /></figure>

Check for
updates

# Co-production opportunities seized and missed in decision-support frameworks for climate-change adaptation in agriculture – How do we practice the “best practice”?

Aleksandra Dolinska<sup>a,b,*</sup>, Emeline Hassenforder<sup>c,d,e</sup>, Ana Maria Loboguerrero<sup>f</sup>, Benjamin Sultan<sup>b</sup>, Jérôme Bossuet<sup>a,g</sup>, Jeanne Cottenceau<sup>h,i</sup>, Michelle Bonatti<sup>j,k</sup>, Jon Hellin<sup>l</sup>, Insaf Mekki<sup>m</sup>, Alexis Drogoul<sup>h,i</sup>, Vincent Vadez<sup>a,n</sup>

## HIGHLIGHTS

* Knowledge co-production is largely suggested as the best practice to bridge the gaps between science, policy and practice.

* Evidence is lacking for the actual use of co-production in decision-support for climate-change adaptation in agriculture.

* We identify useful entry points for applying co-production principles from already used decision-support frameworks.

* Researchers are rarely selected to build decision-support frameworks based on their ability to support co-production.

* When designing decision-support frameworks co-production principles can be used as a guiding tool.

## GRAPHICAL ABSTRACT

Co-production opportunities seized and missed in decision-support frameworks for climate-change adaptation in agriculture – how do we practice the “best practice”?

<figure><img src="image_3.png" /></figure>

* Co-production in presented as the best practice to arrive at sustainable solutions to adapt to climate change

* Co-production is recommended to avoid usability gap in decision-support

<figure><img src="image_4.png" /></figure>

Is co-production applied in multi-stakeholder decision-support frameworks for climate-change adaptation? How exactly?

<figure><img src="image_5.png" /></figure>

Literature review for decision-support frameworks for climate-change adaptation actively involving non-academic stakeholders

18 applied frameworks analyzed in detail for entry-points for co-production principles

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

### examples of identified / underrepresented or missing entry-points

* <u>coastal</u> - exchange and use, endogenous adaptation strategies...

* <u>plurielastic</u> - having a selection strategy aiming at diversity, creating interaction spaces for groups with competing views

* <u>non-downloading altitude</u>, etc...

* <u>gadolin</u> - discussing values underlying participatory goals

* <u>interactive</u> - developing a theory of change with the participants, discussing other perspectives at all stages of the process

* <u>interactive</u> - inviting participants to produce their own tools of analysis, maps, calendars, matrices... If the project includes citizen science, putting traditional/indigenous knowledge at the same level as scientific knowledge, for example by creating spaces to upload it on the on-line tools setting the participants determine their own role/level of replication in the process

### CONCLUSIONS
Members of the research team should be perceived as stakeholders in the co-production process with similar expectations and roles applied to them as to the other stakeholders

* The four co-production principles proposed by Nøstrom can be used as a check-list to guide the process of co-construction of decision support

* The potential of decision-support frameworks to tackle the three dimensions: science, policy and practice should be explored

<footnote><sup>a</sup> French National Research Institute for Sustainable Development (IRD), UMR DIADE, University of Montpellier, Montpellier, France</footnote>

<footnote><sup>b</sup> French National Research Institute for Sustainable Development (IRD), UMR ESPACE-DEV, University of Montpellier, Montpellier, France</footnote>

<footnote><sup>c</sup> CIRAD, UMR G-EAU, 01800 Tunis, Tunisia</footnote>

<footnote><sup>d</sup> G-EAU, Univ Montpellier, AgroParisTech, CIRAD, INRAE, Institut Agro, IRD, Montpellier, France</footnote>

<footnote><sup>e</sup> Institut National Agronomique de Tunisie (INAT), Tunisia</footnote>

<footnote><sup>f</sup> Bioversity International, Via di San Domenico 1, 00153 Rome, Italy</footnote>

<footnote><sup>g</sup> Science Communication Consultant, Exeter, GB, United Kingdom</footnote>

<footnote><sup>h</sup> ACROSS International Joint Laboratory IRD/Thuyloi University, 175 Tay Son, Hanoi, Viet Nam</footnote>

<footnote><sup>i</sup> UMI UMMISCO (IRD/Sorbonne Université), 32 Av. H. Varagnat Bondy Cedex, France</footnote>

<footnote><sup>j</sup> Leibniz Centre for Agricultural Landscape Research (ZALF e. V.), Müncheberg, Germany</footnote>

<footnote><sup>k</sup> Department of Agricultural Economics, Humboldt University of Berlin, Berlin, Germany</footnote>

<footnote><sup>l</sup> International Rice Research Institute (IRRI), Philippines</footnote>

<footnote><sup>m</sup> INRGREF, University of Carthage, B.P. 10, 2080 Ariana, Tunisia</footnote>

<footnote><sup>n</sup> Centre d'Etude Régionale sur l'amélioration de l'Adaptation à la Sècheresse (CERAAS), Laboratoire Mixte International, Campus ENSA, Senegal</footnote>

<footnote>* Corresponding author at: 11 rue de la Fontaine, 30440 Saint Laurent le Minier, France.
E-mail address: ola.dolinska@protonmail.com (A. Dolinska).</footnote>
ARTICLE INFO

Editor: Laurens Klerkx

Keywords:

Co-production

Climate-change adaptation

Decision-support framework

Usability-gap

Non-academic actors

ABSTRACT

**CONTEXT:** To contribute to building sustainable and effective climate change adaptation solutions avoiding usability gap, it is largely recommended to engage in the process of co-production, integrating expertise and knowledge from various academic and non-academic actors.

**OBJECTIVE:** We want to learn if and how co-production, believed to effectively link knowledge and decision-making, and thus suggested as the best practice in building decision-support frameworks, is really applied in the frameworks that are being implemented on the ground.

**METHODS:** A literature review allowed us to identify integrated decision-support frameworks for climate-change adaptation in agriculture developed and used over the period of the last 10 years and involving non-academic stakeholders. To analyse them, we chose as an assessment tool the four co-production principles proposed by Norström and colleagues: context-based, pluralistic, goal-oriented and interactive.

**RESULTS AND CONCLUSIONS:** The useful entry points for incorporating co-production in the design of decision-support that we found in the reviewed frameworks include among the others adequate participants selection strategy, building on existing interaction spaces, developing a theory of change with the participants, and involving participants in the design of different elements of the method. The architectures of the analyzed frameworks contained more elements that responded to pluralistic and interactive principles than to context-based and goal oriented principles, we have also identified gaps in the design, such as taking into account the personal characteristics of researchers that could strengthen a framework's implementation and its impact, or attempts at bridging different levels of decision making, to cover the triad of science, policy and practice. A detailed look at the decision-frameworks that are actually being applied allows for a critical reflection whether and how we as researchers use what we preach as an effective way of responding to sustainability challenges in agriculture. Co-production principles turn out to be a useful tool for analysis and we suggest they can be used as a check-list when designing decision-support frameworks for climate-change adaptation.

**SIGNIFICANCE:** This papers offers useful examples of how to shift the research-led processes of decision-support towards more co-production with non-academic actors, to increase chances of bridging the gaps between science, policy and practice.

# 1. Introduction

The growing pressure of climate change on agricultural systems and the resulting threat to food security and livelihoods of farming communities create a sense of urgency in the domain of climate change adaptation (Ford et al., 2011; Biesbroek et al., 2010; Rosenzweig et al., 2021). The need for successful adaptation actions is particularly pressing in regions where both impact of climate change and climate-related vulnerability are disproportionally high, such as sub-Saharan Africa (Steynor et al., 2020). According to projections, by 2050 the majority of African countries will experience novel climates (Challinor et al., 2016). This is expected to strongly impact their crop production, dependent mainly on rain-fed smallholder agriculture (Adiku et al., 2014; Nya-mekye et al., 2018; MacCarthy et al., 2021), and affect populations that are already vulnerable due to their pre-existing poverty (Ribot, 2014).

As a complex “wicked” problem (Boon et al., 2014), climate change needs to be addressed by a collaborative effort of various actors operating at different levels (Neil Adger et al., 2005; Dilling and Lemos, 2011; Cundill et al., 2019a; Cundill et al., 2019b). This places climate change adaptation at the cross-section of science, policy and practice – the space typically described as full of “gaps”. The metaphorical gaps (for a critical look at the “gap” metaphor see Van Kerkhoff and Lebel, 2015) include science-policy gap, usability gap (between science and practice) and policy-practice gap. The science-policy gap is related to the differences in trust towards scientific evidence between scientists and policy-makers (Bradshaw and Borchers, 2000; Hegger et al., 2012; Serrao-Neumann et al., 2015; Lacey et al., 2018). The usability gap results from a mismatch between what science proposes as solutions and what is needed and considered relevant by societal actors operating in their complex realities (Feldman and Ingram, 2009; Lemos and Rood, 2010; Lemos et al., 2012). In policy-practice gap, the mismatch exists between what policymakers put in place as regulations and the practical considerations of concerned actors (Micha et al., 2020).

There is a common understanding that to contribute to bridging these gaps, research should be practiced in a certain way – one that entails giving up the “sacred” position of scientific expertise and instead relies on the contributions from various academic and non-academic

actors, jointly engaging in production of knowledge (Boon et al., 2011; Hegger et al., 2012; Klenk and Meehan, 2015; Hellin et al., 2022). This process is often referred to as co-production. It can be more precisely defined as an “iterative and collaborative processes involving diverse types of expertise, knowledge and actors to produce context-specific knowledge and pathways towards a sustainable future” (Norström et al., 2020). Applying such processes in research transforms it into more issue-driven (Robinson, 2008; Romero-Lankao et al., 2013), solution-oriented (Hinkel and Bisaro, 2016; Kates, 2011) and genuinely collaborative activity, generating more socially robust and actionable outcomes (Turnhout et al., 2020; Rossing et al., 2021) and ensuring greater ownership of involved actors over the process. It also creates an opportunity to integrate different knowledge systems, such as scientific and indigenous (Berkes et al., 2000), or different kinds of knowledge, such as experiential and localized (Nowotny et al., 2003; Pohl, 2008), functional (Jentoft, 2000), or tacit (Warner et al., 2018), to name a few. The concept of co-production, developed simultaneously in different literatures (Wyborn et al., 2019; Bremer and Meisch, 2017; Norström et al., 2020), found its place in mainstream research on sustainability and climate change adaptation, gaining a status of the best practice (Jagannathan et al., 2020).

One of the promises of co-production is to more effectively link knowledge and decision-making (Wyborn et al., 2019). It makes it particularly relevant for agricultural decision-support, the field that deals with the issue of improving farmers' decision-making by drawing on scientific knowledge with the help of systems and tools designed for this purpose. There has been a long-lasting concern about these tools not being sufficiently used by actors they were meant for (Hochman and Carberry, 2011; Giupponi, 2014; Rose et al., 2016; Antle et al., 2017) and a growing body of works suggests co-production approaches with their potential of bridging the usability gap as a solution for improving decision-support. Whereas various authors provide theoretical insights, describe constraints and challenges of using co-production and propose “how to” guides (Hegger et al., 2012; Prokopy et al., 2015; Barnhart et al., 2018; Walling and Vaneeckhaute, 2020), we think that whether and how it is actually being used in practice is not sufficiently reported.

Under the growing pressure to find effective climate change
ARTICLE INFO

Editor: Laurens Klerkx

Keywords:

Co-production

Climate-change adaptation

Decision-support framework

Usability-gap

Non-academic actors

ABSTRACT

**CONTEXT:** To contribute to building sustainable and effective climate change adaptation solutions avoiding usability gap, it is largely recommended to engage in the process of co-production, integrating expertise and knowledge from various academic and non-academic actors.

**OBJECTIVE:** We want to learn if and how co-production, believed to effectively link knowledge and decision-making, and thus suggested as the best practice in building decision-support frameworks, is really applied in the frameworks that are being implemented on the ground.

**METHODS:** A literature review allowed us to identify integrated decision-support frameworks for climate-change adaptation in agriculture developed and used over the period of the last 10 years and involving non-academic stakeholders. To analyse them, we chose as an assessment tool the four co-production principles proposed by Norström and colleagues: context-based, pluralistic, goal-oriented and interactive.

**RESULTS AND CONCLUSIONS:** The useful entry points for incorporating co-production in the design of decision-support that we found in the reviewed frameworks include among the others adequate participants selection strategy, building on existing interaction spaces, developing a theory of change with the participants, and involving participants in the design of different elements of the method. The architectures of the analyzed frameworks contained more elements that responded to pluralistic and interactive principles than to context-based and goal oriented principles, we have also identified gaps in the design, such as taking into account the personal characteristics of researchers that could strengthen a framework's implementation and its impact, or attempts at bridging different levels of decision making, to cover the triad of science, policy and practice. A detailed look at the decision-frameworks that are actually being applied allows for a critical reflection whether and how we as researchers use what we preach as an effective way of responding to sustainability challenges in agriculture. Co-production principles turn out to be a useful tool for analysis and we suggest they can be used as a check-list when designing decision-support frameworks for climate-change adaptation.

**SIGNIFICANCE:** This papers offers useful examples of how to shift the research-led processes of decision-support towards more co-production with non-academic actors, to increase chances of bridging the gaps between science, policy and practice.

# 1. Introduction

The growing pressure of climate change on agricultural systems and the resulting threat to food security and livelihoods of farming communities create a sense of urgency in the domain of climate change adaptation (Ford et al., 2011; Biesbroek et al., 2010; Rosenzweig et al., 2021). The need for successful adaptation actions is particularly pressing in regions where both impact of climate change and climate-related vulnerability are disproportionally high, such as sub-Saharan Africa (Steynor et al., 2020). According to projections, by 2050 the majority of African countries will experience novel climates (Challinor et al., 2016). This is expected to strongly impact their crop production, dependent mainly on rain-fed smallholder agriculture (Adiku et al., 2014; Nya-mekye et al., 2018; MacCarthy et al., 2021), and affect populations that are already vulnerable due to their pre-existing poverty (Ribot, 2014).

As a complex “wicked” problem (Boon et al., 2014), climate change needs to be addressed by a collaborative effort of various actors operating at different levels (Neil Adger et al., 2005; Dilling and Lemos, 2011; Cundill et al., 2019a; Cundill et al., 2019b). This places climate change adaptation at the cross-section of science, policy and practice – the space typically described as full of “gaps”. The metaphorical gaps (for a critical look at the “gap” metaphor see Van Kerkhoff and Lebel, 2015) include science-policy gap, usability gap (between science and practice) and policy-practice gap. The science-policy gap is related to the differences in trust towards scientific evidence between scientists and policy-makers (Bradshaw and Borchers, 2000; Hegger et al., 2012; Serrao-Neumann et al., 2015; Lacey et al., 2018). The usability gap results from a mismatch between what science proposes as solutions and what is needed and considered relevant by societal actors operating in their complex realities (Feldman and Ingram, 2009; Lemos and Rood, 2010; Lemos et al., 2012). In policy-practice gap, the mismatch exists between what policymakers put in place as regulations and the practical considerations of concerned actors (Micha et al., 2020).

There is a common understanding that to contribute to bridging these gaps, research should be practiced in a certain way – one that entails giving up the “sacred” position of scientific expertise and instead relies on the contributions from various academic and non-academic

actors, jointly engaging in production of knowledge (Boon et al., 2011; Hegger et al., 2012; Klenk and Meehan, 2015; Hellin et al., 2022). This process is often referred to as co-production. It can be more precisely defined as an “iterative and collaborative processes involving diverse types of expertise, knowledge and actors to produce context-specific knowledge and pathways towards a sustainable future” (Norström et al., 2020). Applying such processes in research transforms it into more issue-driven (Robinson, 2008; Romero-Lankao et al., 2013), solution-oriented (Hinkel and Bisaro, 2016; Kates, 2011) and genuinely collaborative activity, generating more socially robust and actionable outcomes (Turnhout et al., 2020; Rossing et al., 2021) and ensuring greater ownership of involved actors over the process. It also creates an opportunity to integrate different knowledge systems, such as scientific and indigenous (Berkes et al., 2000), or different kinds of knowledge, such as experiential and localized (Nowotny et al., 2003; Pohl, 2008), functional (Jentoft, 2000), or tacit (Warner et al., 2018), to name a few. The concept of co-production, developed simultaneously in different literatures (Wyborn et al., 2019; Bremer and Meisch, 2017; Norström et al., 2020), found its place in mainstream research on sustainability and climate change adaptation, gaining a status of the best practice (Jagannathan et al., 2020).

One of the promises of co-production is to more effectively link knowledge and decision-making (Wyborn et al., 2019). It makes it particularly relevant for agricultural decision-support, the field that deals with the issue of improving farmers' decision-making by drawing on scientific knowledge with the help of systems and tools designed for this purpose. There has been a long-lasting concern about these tools not being sufficiently used by actors they were meant for (Hochman and Carberry, 2011; Giupponi, 2014; Rose et al., 2016; Antle et al., 2017) and a growing body of works suggests co-production approaches with their potential of bridging the usability gap as a solution for improving decision-support. Whereas various authors provide theoretical insights, describe constraints and challenges of using co-production and propose “how to” guides (Hegger et al., 2012; Prokopy et al., 2015; Barnhart et al., 2018; Walling and Vaneeckhaute, 2020), we think that whether and how it is actually being used in practice is not sufficiently reported.

Under the growing pressure to find effective climate change
adaptation measures, we set out to explore to what extent the co-production approaches are reflected in the design of decision-support frameworks developed over past 10 years (matching the concept's growing popularity) in the field of climate change adaptation for agriculture. Through a literature review, we identify examples of integrated decision-support frameworks that involve multiple-stakeholders, and we assess whether and in what way co-production is used. The aim is to explore which elements and characteristics of decision-support frameworks provide useful entry-points for applying in practice co-production principles and to find some guidance on how these applications can be better replicated elsewhere.

We first shortly describe how decision-support evolved towards co-production. Secondly, we present a review of decision-support frameworks and then we present and – in parallel – discuss the results of this review. We conclude by proposing some paths for further reflection.

## 2. Towards co-production of decision-support systems

Decision-support systems (DSS) are designed to help in both structuring and resolving problems (Walling and Vaneeckhaute, 2020), through improving the process, the outputs or the outcomes of decision-making (Moser, 2009). A DSS will typically make use of a data processing tool (model, spatial analysis tool, etc.) combined with an evaluation routine (multi-criteria analysis, cost-benefit analysis, etc.) and have an interactive interface (Giupponi, 2014). In agriculture, DSS are aimed at supporting actors in dealing with challenges of agricultural management of various kind such as crop, land, pest, natural resources, farm economy or livestock. Seen as tools of both analysis and communication (Smits, 2002; Barnhart et al., 2018), agricultural DSS are often meant to be used by technical advisors allowing them to make better site-specific recommendations to farmers (Rose et al., 2016).

Literature provides various explanations for the observed usability gap in the field of agricultural DSS: the tools do not sufficiently match the actual needs in the field (Allen et al., 2017), they do not address actual farmers' concerns (Derner et al., 2012), do not reflect the ways targeted farmers make decisions (McCown, 2002; Carberry et al., 2002; Jakku and Thorburn, 2010), or are designed based on erroneous assumptions by scientists about the future users (Lemos and Rood, 2010; Bruine de Bruin and Bostrom, 2013; Sofoulis, 2011; Porter and Dessai, 2017). An early response to the usability gap in the DSS field was to increase the participation of users and to improve both quantity and quality of interaction between users and developers (Lowrey et al., 2009; Barnhart et al., 2018). However, more dialogue has been initially incorporated mostly at the stage of using rather than producing DSS (Carberry et al., 2002), with more emphasis on joint learning (as opposed to the knowledge transfer) between the advisors and the farmers (Duru et al., 2012). Further, voices have been raised to turn to dialogue also at the stage of designing DSS (Cerf et al., 2012; Martin, 2015). Finally, the increasingly popular participatory design approaches, such as for example user-centered design of IT products (Ortiz-Crespo et al., 2021; Steinke et al., 2022) found its place in the DSS development and users are more often invited to provide input into the construction of tools with the hope of creating engagement that would result in better use of those tools (see for example Sultan et al., 2020).

Another step towards co-production in DSS was taken when the underlying assumption that decision making is primarily limited by the quantity and quality of available information was called into question. Several authors suggested broadening decision-support beyond the information processing tools by factoring in the broader decision environment (Lawrence et al., 2001). This addresses institutional, political and communication processes relative to decisions (Pyke et al., 2007), supplements technical data with local knowledge (Nay et al., 2014) and finally, supports models with additional frameworks that would bring stakeholders closer to the process and allow to include uncertainty and risk, that models alone fail to incorporate (Doukas and Nikas, 2020). In this perspective decision-support is regarded more as a process than a

tool. More integrated decision-support frameworks are proposed, that structure different elements needed to support decision-making and allow for exploring impacts of decisions and evaluating options against the objectives of those invited to participate (for example, those who make or are impacted by decisions) (Fürst et al., 2010). As a result, involving multiple stakeholders in building decision-support becomes a primary issue. Walling and Vaneeckhaute (2020) formulate recommendations for dealing with stakeholder-oriented challenges in creating environmental DSS, including involving stakeholders early in the process and giving them a real role to play in the development of the tool. Other authors focus on additional elements of decision-support provision: Guido et al. (2020) explore different dimensions of context in co-production of Climate Information Services (CIS) - services offering information about the impact of weather and climate on users' activities (for example farming) along with the elements that contextualize this information for decision making (Vaughan and Dessai, 2014), Prokopy et al. (2015) provide insights into the question of team communication in interdisciplinary projects aimed at producing decision-support, while Ditzler et al. (2018) advocate the use of the concept of *affordances* to enhance design of system analysis tools, thus focusing on what the tool provides in an interaction with the users. These elements are leaning towards co-production. Finally, Barnhart et al. (2018) with their conceptual model for creating decision-support tools for watershed modeling, postulate incorporating co-production in DSS design systematically. How are these postulates implemented in practice?

## 3. Materials and methods

We review multi-stakeholder integrated decision-support frameworks in the field of climate-change adaptation in agriculture. The review is driven by a will to critically examine the use of co-production in research practice. Thus we searched for frameworks descriptions in the peer-reviewed literature and we focus on frameworks initiated and developed in agricultural research projects.

### 3.1. *Search protocol*

To identify papers presenting multi-stakeholder integrated decision-support frameworks for climate-change adaptation we conducted a search in the Web of Science search engine. We used different combinations of search terms and their variants (decision-support; decision; framework; multi-stakeholder; multi-disciplinary; pluri-disciplinary; inter-disciplinary; trans-disciplinary; co-construction; co-design; co-production; climate change; climate change adaptation; climate resilience; agriculture; farming). We first screened search results for relevance, looking at titles and sometimes abstracts, and then proceeded with the more detailed selection procedure based on the inclusion and exclusion criteria (Table 1).

There were certain assumptions behind these criteria. First, we assumed that in order for co-production to occur, there need to be non-academic stakeholders involved in the process. Thus we excluded papers describing frameworks where no non-academic actors were involved, for example those that reported on cases of coupling different models, without stakeholder engagement (Jeon et al., 2018; Prasad et al., 2016; Mehryar et al., 2019). Second, as we were interested in getting methodological insights, the papers needed to present the frameworks in a way that would allow it: outlining the steps, mentioning the participants, and naming methods that were used. This penalized papers that were focused more on outcomes and impact of the used decision-support frameworks than their detailed methodology (Loboguerrero et al., 2018). Third, we focused on frameworks which had a purpose to support decision-making, be it at the level of farmers or funders allocating financing. And thus we did not include papers describing serious games whose focus was on social learning (Salvini et al., 2016; Dolinska, 2017). Fourth, we decided to analyse only those frameworks who had at least a test implementation, discarding the solely conceptual propositions
adaptation measures, we set out to explore to what extent the co-production approaches are reflected in the design of decision-support frameworks developed over past 10 years (matching the concept's growing popularity) in the field of climate change adaptation for agriculture. Through a literature review, we identify examples of integrated decision-support frameworks that involve multiple-stakeholders, and we assess whether and in what way co-production is used. The aim is to explore which elements and characteristics of decision-support frameworks provide useful entry-points for applying in practice co-production principles and to find some guidance on how these applications can be better replicated elsewhere.

We first shortly describe how decision-support evolved towards co-production. Secondly, we present a review of decision-support frameworks and then we present and – in parallel – discuss the results of this review. We conclude by proposing some paths for further reflection.

## 2. Towards co-production of decision-support systems

Decision-support systems (DSS) are designed to help in both structuring and resolving problems (Walling and Vaneeckhaute, 2020), through improving the process, the outputs or the outcomes of decision-making (Moser, 2009). A DSS will typically make use of a data processing tool (model, spatial analysis tool, etc.) combined with an evaluation routine (multi-criteria analysis, cost-benefit analysis, etc.) and have an interactive interface (Giupponi, 2014). In agriculture, DSS are aimed at supporting actors in dealing with challenges of agricultural management of various kind such as crop, land, pest, natural resources, farm economy or livestock. Seen as tools of both analysis and communication (Smits, 2002; Barnhart et al., 2018), agricultural DSS are often meant to be used by technical advisors allowing them to make better site-specific recommendations to farmers (Rose et al., 2016).

Literature provides various explanations for the observed usability gap in the field of agricultural DSS: the tools do not sufficiently match the actual needs in the field (Allen et al., 2017), they do not address actual farmers' concerns (Derner et al., 2012), do not reflect the ways targeted farmers make decisions (McCown, 2002; Carberry et al., 2002; Jakku and Thorburn, 2010), or are designed based on erroneous assumptions by scientists about the future users (Lemos and Rood, 2010; Bruine de Bruin and Bostrom, 2013; Sofoulis, 2011; Porter and Dessai, 2017). An early response to the usability gap in the DSS field was to increase the participation of users and to improve both quantity and quality of interaction between users and developers (Lowrey et al., 2009; Barnhart et al., 2018). However, more dialogue has been initially incorporated mostly at the stage of using rather than producing DSS (Carberry et al., 2002), with more emphasis on joint learning (as opposed to the knowledge transfer) between the advisors and the farmers (Duru et al., 2012). Further, voices have been raised to turn to dialogue also at the stage of designing DSS (Cerf et al., 2012; Martin, 2015). Finally, the increasingly popular participatory design approaches, such as for example user-centered design of IT products (Ortiz-Crespo et al., 2021; Steinke et al., 2022) found its place in the DSS development and users are more often invited to provide input into the construction of tools with the hope of creating engagement that would result in better use of those tools (see for example Sultan et al., 2020).

Another step towards co-production in DSS was taken when the underlying assumption that decision making is primarily limited by the quantity and quality of available information was called into question. Several authors suggested broadening decision-support beyond the information processing tools by factoring in the broader decision environment (Lawrence et al., 2001). This addresses institutional, political and communication processes relative to decisions (Pyke et al., 2007), supplements technical data with local knowledge (Nay et al., 2014) and finally, supports models with additional frameworks that would bring stakeholders closer to the process and allow to include uncertainty and risk, that models alone fail to incorporate (Doukas and Nikas, 2020). In this perspective decision-support is regarded more as a process than a

tool. More integrated decision-support frameworks are proposed, that structure different elements needed to support decision-making and allow for exploring impacts of decisions and evaluating options against the objectives of those invited to participate (for example, those who make or are impacted by decisions) (Fürst et al., 2010). As a result, involving multiple stakeholders in building decision-support becomes a primary issue. Walling and Vaneeckhaute (2020) formulate recommendations for dealing with stakeholder-oriented challenges in creating environmental DSS, including involving stakeholders early in the process and giving them a real role to play in the development of the tool. Other authors focus on additional elements of decision-support provision: Guido et al. (2020) explore different dimensions of context in co-production of Climate Information Services (CIS) - services offering information about the impact of weather and climate on users' activities (for example farming) along with the elements that contextualize this information for decision making (Vaughan and Dessai, 2014), Prokopy et al. (2015) provide insights into the question of team communication in interdisciplinary projects aimed at producing decision-support, while Ditzler et al. (2018) advocate the use of the concept of *affordances* to enhance design of system analysis tools, thus focusing on what the tool provides in an interaction with the users. These elements are leaning towards co-production. Finally, Barnhart et al. (2018) with their conceptual model for creating decision-support tools for watershed modeling, postulate incorporating co-production in DSS design systematically. How are these postulates implemented in practice?

## 3. Materials and methods

We review multi-stakeholder integrated decision-support frameworks in the field of climate-change adaptation in agriculture. The review is driven by a will to critically examine the use of co-production in research practice. Thus we searched for frameworks descriptions in the peer-reviewed literature and we focus on frameworks initiated and developed in agricultural research projects.

### 3.1. *Search protocol*

To identify papers presenting multi-stakeholder integrated decision-support frameworks for climate-change adaptation we conducted a search in the Web of Science search engine. We used different combinations of search terms and their variants (decision-support; decision; framework; multi-stakeholder; multi-disciplinary; pluri-disciplinary; inter-disciplinary; trans-disciplinary; co-construction; co-design; co-production; climate change; climate change adaptation; climate resilience; agriculture; farming). We first screened search results for relevance, looking at titles and sometimes abstracts, and then proceeded with the more detailed selection procedure based on the inclusion and exclusion criteria (Table 1).

There were certain assumptions behind these criteria. First, we assumed that in order for co-production to occur, there need to be non-academic stakeholders involved in the process. Thus we excluded papers describing frameworks where no non-academic actors were involved, for example those that reported on cases of coupling different models, without stakeholder engagement (Jeon et al., 2018; Prasad et al., 2016; Mehryar et al., 2019). Second, as we were interested in getting methodological insights, the papers needed to present the frameworks in a way that would allow it: outlining the steps, mentioning the participants, and naming methods that were used. This penalized papers that were focused more on outcomes and impact of the used decision-support frameworks than their detailed methodology (Loboguerrero et al., 2018). Third, we focused on frameworks which had a purpose to support decision-making, be it at the level of farmers or funders allocating financing. And thus we did not include papers describing serious games whose focus was on social learning (Salvini et al., 2016; Dolinska, 2017). Fourth, we decided to analyse only those frameworks who had at least a test implementation, discarding the solely conceptual propositions
<table><caption>**Table 1**
Inclusion and exclusion criteria.</caption><thead><tr><td>Inclusion criteria</td><td>Exclusion criteria</td><td>Number of papers excluded based on a given criterium</td></tr></thead><tbody><tr><td>Applied at least once in a tropical or subtropical area</td><td>Applied outside of a tropical or subtropical area</td><td>20</td></tr><tr><td>Applied in the context of agriculture</td><td>Supporting non-agricultural decision-making</td><td>6</td></tr><tr><td>Directly supporting a decision-making process</td><td>Not directly supporting decision-making</td><td>7</td></tr><tr><td>Related to climate-change adaptation</td><td>Not related to climate-change adaptation</td><td>5</td></tr><tr><td>Involvement of non-academic stakeholders / Drawing of knowledge from more than one source</td><td>Not participatory</td><td>15</td></tr><tr><td>Framework structured and outlined in a way allowing to describe different steps and their participants</td><td>Insufficiently described in terms of steps and participants</td><td>1</td></tr></tbody></table>

(Thornton et al., 2018; van Zonneveld et al., 2020). Fifth, following the example of lessons learned from unsuccessful efforts to bridge the science-policy gap described by Dinesh et al. (2021), we kept papers describing unsuccessful attempts at co-production, where the authors point out the elements of framework where the failures occurred, and suggest ways of improvement. And finally, we did not want to limit the analysis to the frameworks that were designed with the stated intent of co-production, we decided that any work undertaking methodological challenge of integrating non-academic stakeholders or knowledge in the process of constructing decision-support is of interest. When the coproduction was not the central interest of a paper, the authors may have chosen not to include information or methodological details that would be relevant for our analysis. We are also aware there may be co-production processes that are not initiated nor led by research and thus overlooked by peer-reviewed literature and as a consequence by our analysis.

Applying the criteria, we discarded 53 papers out of 74 preliminarily retained. We identified and analyzed a total of eighteen frameworks, described in twenty one scientific papers (Table 2).

## 3.2. Typology of frameworks

In regards to decisions that they support, we identified five categories of frameworks, the first two appearing more often than the remaining three. All frameworks included in our study are presented in Table 2 and numbered. We will use these numbers (in square brackets) when a reference is made to a particular framework.

### 3.2.1. Climate Smart Agriculture (CSA) interventions [frameworks 1–6]

With its goal of assuring the sustainability of farming under climate change, CSA is an increasingly popular approach to agricultural development (Lipper et al., 2014). The concept of “climate-smart” is based on three pillars used to assess farming practices: increased agricultural productivity, climate change adaptation and mitigation through reduced GHG emissions. The costs and benefits of different CSA options need to be weighed based on stakeholders' objectives and according to local conditions – different actors in different situations may assign different values to these pillars and there are trade-offs between them. The ultimate beneficiaries of CSA are farmers, but decision-support is often needed at the stage of investing in research or promotion of the best CSA options for a given geographical area. All the CSA frameworks that we reviewed are directed at actors planning or financing research or development interventions, although one [framework 4] is directed also at the private-sector actors and for another one, the only sectoral

framework in this group [3], primary users are banana plantations managers. For the other frameworks, implementation level varies from local [4], through district [5], sub-regional [5] and regional [1] to the national [2] level, or they are suitable for different levels [6]. As context-sensitive, CSA prioritisation requires inputs from local stakeholders – where “local” is determined by the level at which the concerned decisions are made.

### 3.2.2. Climate Information Services (CIS) for agriculture [7–12]

CIS (or climate services, weather and climate services) is a concept that was developed as a response to the growing demand for climate information from decision-makers and the gap between what decision-makers needed and what and how was already supplied. In order to better understand the needs of the users, the producers of climate information invite them to contribute their expertise to the process of development and communication of climate information products. This is referred to as co-production of CIS. In our study we identified a diversity of approaches – from a national level CIS provision models run by the national meteorological agencies in India and Senegal [7,11], through farmer-focused very localized methodologies like Participatory Integrated Climate Services for Agriculture (PICSA) [8], to a model of a citizen-science based platform [12] – all those frameworks focus on linking climate information with decision-making. They are all directed at farmers who are the end users. One of the frameworks [7], a climate services provision model developed in Senegal, is implemented in parallel at the local level, where end users are farmers and at the national level, where end-users are sectoral decision-makers, creating a space where science, policy and practice meet.

### 3.2.3. Farming system design [13–14]

We named another category “farming system design” for the frameworks that provide support for farmers in planning their activities. The focus and objectives may be achieving a generally climate-smart system [13] or implementation of a particular CSA strategy/option (agroforestry) [14]. Frameworks in this category are targeted at farmers and applied at the local level.

### 3.2.4. Sustainable management of socio-ecological systems [15–17]

Two of the frameworks in this category link land-users decisions with the landscape level and involve an exploration of their socio-ecological systems. The ultimate objective is land conservation under changing climate – these frameworks support the choice of desertification mitigation strategies [15] and development of sustainable and climate-resilient land-use strategies [16] respectively. In the first case the land users are farmers, in the second case, where the framework covers diverse land uses, farmers are only one type of potential users. The third framework in this category focuses on the management of a particular ecosystem – it is directed at wetland system managers in Ecuador.

The last identified framework is designed to inform climate-smart food systems policy [18] at the national level with the national policymakers as intended users.

The retained frameworks were applied mostly in West and East African countries, followed by countries from South Africa, South and Central America, South Asia and North Africa (Table 3).

The level of implementation of the frameworks differs – of eighteen frameworks, eight were either implemented in several countries [5,6,9,14,17,20] or have multiple implementations in one country [8,18]. One framework was implemented on virtual farms in two countries [15].

## 3.3. Data analysis

Reviewing decision-support frameworks from a co-production perspective required guiding principles against which to assess them. We use the four co-production principles proposed by Norström et al. (2020). According to the authors' definition, a co-production process
<table><caption>**Table 1**
Inclusion and exclusion criteria.</caption><thead><tr><td>Inclusion criteria</td><td>Exclusion criteria</td><td>Number of papers excluded based on a given criterium</td></tr></thead><tbody><tr><td>Applied at least once in a tropical or subtropical area</td><td>Applied outside of a tropical or subtropical area</td><td>20</td></tr><tr><td>Applied in the context of agriculture</td><td>Supporting non-agricultural decision-making</td><td>6</td></tr><tr><td>Directly supporting a decision-making process</td><td>Not directly supporting decision-making</td><td>7</td></tr><tr><td>Related to climate-change adaptation</td><td>Not related to climate-change adaptation</td><td>5</td></tr><tr><td>Involvement of non-academic stakeholders / Drawing of knowledge from more than one source</td><td>Not participatory</td><td>15</td></tr><tr><td>Framework structured and outlined in a way allowing to describe different steps and their participants</td><td>Insufficiently described in terms of steps and participants</td><td>1</td></tr></tbody></table>

(Thornton et al., 2018; van Zonneveld et al., 2020). Fifth, following the example of lessons learned from unsuccessful efforts to bridge the science-policy gap described by Dinesh et al. (2021), we kept papers describing unsuccessful attempts at co-production, where the authors point out the elements of framework where the failures occurred, and suggest ways of improvement. And finally, we did not want to limit the analysis to the frameworks that were designed with the stated intent of co-production, we decided that any work undertaking methodological challenge of integrating non-academic stakeholders or knowledge in the process of constructing decision-support is of interest. When the coproduction was not the central interest of a paper, the authors may have chosen not to include information or methodological details that would be relevant for our analysis. We are also aware there may be co-production processes that are not initiated nor led by research and thus overlooked by peer-reviewed literature and as a consequence by our analysis.

Applying the criteria, we discarded 53 papers out of 74 preliminarily retained. We identified and analyzed a total of eighteen frameworks, described in twenty one scientific papers (Table 2).

## 3.2. Typology of frameworks

In regards to decisions that they support, we identified five categories of frameworks, the first two appearing more often than the remaining three. All frameworks included in our study are presented in Table 2 and numbered. We will use these numbers (in square brackets) when a reference is made to a particular framework.

### 3.2.1. Climate Smart Agriculture (CSA) interventions [frameworks 1–6]

With its goal of assuring the sustainability of farming under climate change, CSA is an increasingly popular approach to agricultural development (Lipper et al., 2014). The concept of “climate-smart” is based on three pillars used to assess farming practices: increased agricultural productivity, climate change adaptation and mitigation through reduced GHG emissions. The costs and benefits of different CSA options need to be weighed based on stakeholders' objectives and according to local conditions – different actors in different situations may assign different values to these pillars and there are trade-offs between them. The ultimate beneficiaries of CSA are farmers, but decision-support is often needed at the stage of investing in research or promotion of the best CSA options for a given geographical area. All the CSA frameworks that we reviewed are directed at actors planning or financing research or development interventions, although one [framework 4] is directed also at the private-sector actors and for another one, the only sectoral

framework in this group [3], primary users are banana plantations managers. For the other frameworks, implementation level varies from local [4], through district [5], sub-regional [5] and regional [1] to the national [2] level, or they are suitable for different levels [6]. As context-sensitive, CSA prioritisation requires inputs from local stakeholders – where “local” is determined by the level at which the concerned decisions are made.

### 3.2.2. Climate Information Services (CIS) for agriculture [7–12]

CIS (or climate services, weather and climate services) is a concept that was developed as a response to the growing demand for climate information from decision-makers and the gap between what decision-makers needed and what and how was already supplied. In order to better understand the needs of the users, the producers of climate information invite them to contribute their expertise to the process of development and communication of climate information products. This is referred to as co-production of CIS. In our study we identified a diversity of approaches – from a national level CIS provision models run by the national meteorological agencies in India and Senegal [7,11], through farmer-focused very localized methodologies like Participatory Integrated Climate Services for Agriculture (PICSA) [8], to a model of a citizen-science based platform [12] – all those frameworks focus on linking climate information with decision-making. They are all directed at farmers who are the end users. One of the frameworks [7], a climate services provision model developed in Senegal, is implemented in parallel at the local level, where end users are farmers and at the national level, where end-users are sectoral decision-makers, creating a space where science, policy and practice meet.

### 3.2.3. Farming system design [13–14]

We named another category “farming system design” for the frameworks that provide support for farmers in planning their activities. The focus and objectives may be achieving a generally climate-smart system [13] or implementation of a particular CSA strategy/option (agroforestry) [14]. Frameworks in this category are targeted at farmers and applied at the local level.

### 3.2.4. Sustainable management of socio-ecological systems [15–17]

Two of the frameworks in this category link land-users decisions with the landscape level and involve an exploration of their socio-ecological systems. The ultimate objective is land conservation under changing climate – these frameworks support the choice of desertification mitigation strategies [15] and development of sustainable and climate-resilient land-use strategies [16] respectively. In the first case the land users are farmers, in the second case, where the framework covers diverse land uses, farmers are only one type of potential users. The third framework in this category focuses on the management of a particular ecosystem – it is directed at wetland system managers in Ecuador.

The last identified framework is designed to inform climate-smart food systems policy [18] at the national level with the national policymakers as intended users.

The retained frameworks were applied mostly in West and East African countries, followed by countries from South Africa, South and Central America, South Asia and North Africa (Table 3).

The level of implementation of the frameworks differs – of eighteen frameworks, eight were either implemented in several countries [5,6,9,14,17,20] or have multiple implementations in one country [8,18]. One framework was implemented on virtual farms in two countries [15].

## 3.3. Data analysis

Reviewing decision-support frameworks from a co-production perspective required guiding principles against which to assess them. We use the four co-production principles proposed by Norström et al. (2020). According to the authors' definition, a co-production process
<table><caption>**Table 2**
Overview of studied frameworks.</caption><thead><tr><th>Decision support framework/process/ platform</th><th>Reference</th><th>Category</th><th>What decision it supports and how?</th><th>End-users/level</th><th>Applied in</th></tr></thead><tbody><tr><td>1. CSA-PF (Andrieu et al. 2017)</td><td>Andrieu, N., Sogoba, B., Zougmore, R., Howland, F., Samake, O., Bonilla-Findji, O., Lizarazo, M., Nowak, A., Dembele, C., Corner-Dolloff, C., 2017. Prioritizing investments for climate-smart agriculture: Lessons learned from Mali. Agricultural Systems 154, 13–24.</td><td>CSA prioritisation</td><td>Allocation of resources to CSA interventions. Allows to identify CSA investment portfolios that maximize desired impacts for agriculture development in the face of climate change.</td><td>Development research planners and regional donors</td><td>Piloted in Mali (1-year pilot)</td></tr><tr><td>2. Target-CSA (Brandt et al. 2017)</td><td>Brandt, P., Kvakić, M., Butterbach-Bahl, K., Rufino, M. C., 2017. How to target climate-smart agriculture? Concept and application of the consensus-driven decision support framework “targetCSA.” Agricultural Systems 151, 234–245.</td><td>CSA prioritisation</td><td>Selecting and geographically targeting CSA interventions to reduce agricultural vulnerability to climate change at the national level</td><td>Planners and policy-makers at the national level</td><td>Tested in Kenya (national scale)</td></tr><tr><td>3. Decision analysis approach (Fernandez et al., 2022)</td><td>Fernandez, E., Do, H., Luedeling, E., Luu, T.T.G., Whitney, C., 2022. Prioritizing farm management interventions to improve climate change adaptation and mitigation outcomes—a case study for banana plantations. Agron. Sustain. Dev. 42, 76.</td><td>CSA prioritisation</td><td>Prioritizing farm management interventions to improve climate change adaptation and mitigation outcomes on banana plantations</td><td>Banana farm managers and banana farming researchers</td><td>Developed for Latin America</td></tr><tr><td>4. A framework to prioritize locally suitable CSA interventions (Khatri-Chhetri et al. 2019)</td><td>Khatri-Chhetri, A., Pant, A., Aggarwal, P.K., Vasireddy, V.V., Yadav, A., 2019. Stakeholders prioritisation of climate-smart agriculture interventions: Evaluation of a framework. Agricultural Systems 174, 23–31.</td><td>CSA prioritisation</td><td>Allocation of resources to CSA interventions. Allows to construct a portfolio of CSA interventions suitable for local context</td><td>Policy-makers at the local level, development organizations and the private sector for investment decision-making</td><td>India (Maharashtra).</td></tr><tr><td>5. CSA-RA (Mwongera et al. 2017)</td><td>Mwongera, C., Shikuku, K.M., Twyman, J., Läderach, P., Ampaire, E., Van Asten, P., Twomlow, S., Winowiecki, L.A., 2017. Climate smart agriculture rapid appraisal (CSA-RA): A tool for prioritizing context-specific climate smart agriculture technologies. Agricultural Systems 151, 192–203.</td><td>CSA prioritisation</td><td>Choice of CSA practices to implement. Helps to identify the key challenges in local smallholder agricultural systems, and to prioritize a set of CSA practices that can respond to such challenges.</td><td>Planners of CSA interventions and investor at the district level and across the sub-region</td><td>Tanzania and Uganda</td></tr><tr><td>6. A generic framework for targeting, out-scaling and prioritizing CSA interventions in agricultural systems (Notenbaert et al., 2017)</td><td>Notenbaert, A., Pfeifer, C., Silvestri, S., Herrero, M., 2017. Targeting, out-scaling and prioritizing climate-smart interventions in agricultural systems: Lessons from applying a generic framework to the livestock sector in sub-Saharan Africa. Agricultural Systems 151, 153–162.</td><td>CSA prioritisation</td><td>Allocation of resources to CSA interventions. Helps to determine which interventions are likely to reach the greatest possible positive impact across the different objectives of food and nutrition security, CC mitigation and CC adaptation.</td><td>Planners of CSA interventions</td><td>Applied in easter Africa region and in Northern Tanzania</td></tr><tr><td>7. MWG model (Ouedraogo et al. 2018, Chiputwa et al., 2020)</td><td>Chiputwa, B., Wainaina, P., Nakelse, T., Makui, P., Zougmoré, R.B., Ndiaye, O., Minang, P.A., 2020. Transforming climate science into usable services: The effectiveness of co-production in promoting uptake of climate information by smallholder farmers in Senegal. Climate Services 20, 100,203.</td><td>CIS co-production</td><td>Risk management decisions (decision-makers), Choice of farming practices (farmers) during rain season</td><td>National sectoral decision-makers / farmers</td><td>Senegal</td></tr></tbody></table>

(continued on next page)
<table><caption>**Table 2**
Overview of studied frameworks.</caption><thead><tr><th>Decision support framework/process/ platform</th><th>Reference</th><th>Category</th><th>What decision it supports and how?</th><th>End-users/level</th><th>Applied in</th></tr></thead><tbody><tr><td>1. CSA-PF (Andrieu et al. 2017)</td><td>Andrieu, N., Sogoba, B., Zougmore, R., Howland, F., Samake, O., Bonilla-Findji, O., Lizarazo, M., Nowak, A., Dembele, C., Corner-Dolloff, C., 2017. Prioritizing investments for climate-smart agriculture: Lessons learned from Mali. Agricultural Systems 154, 13–24.</td><td>CSA prioritisation</td><td>Allocation of resources to CSA interventions. Allows to identify CSA investment portfolios that maximize desired impacts for agriculture development in the face of climate change.</td><td>Development research planners and regional donors</td><td>Piloted in Mali (1-year pilot)</td></tr><tr><td>2. Target-CSA (Brandt et al. 2017)</td><td>Brandt, P., Kvakić, M., Butterbach-Bahl, K., Rufino, M. C., 2017. How to target climate-smart agriculture? Concept and application of the consensus-driven decision support framework “targetCSA.” Agricultural Systems 151, 234–245.</td><td>CSA prioritisation</td><td>Selecting and geographically targeting CSA interventions to reduce agricultural vulnerability to climate change at the national level</td><td>Planners and policy-makers at the national level</td><td>Tested in Kenya (national scale)</td></tr><tr><td>3. Decision analysis approach (Fernandez et al., 2022)</td><td>Fernandez, E., Do, H., Luedeling, E., Luu, T.T.G., Whitney, C., 2022. Prioritizing farm management interventions to improve climate change adaptation and mitigation outcomes—a case study for banana plantations. Agron. Sustain. Dev. 42, 76.</td><td>CSA prioritisation</td><td>Prioritizing farm management interventions to improve climate change adaptation and mitigation outcomes on banana plantations</td><td>Banana farm managers and banana farming researchers</td><td>Developed for Latin America</td></tr><tr><td>4. A framework to prioritize locally suitable CSA interventions (Khatri-Chhetri et al. 2019)</td><td>Khatri-Chhetri, A., Pant, A., Aggarwal, P.K., Vasireddy, V.V., Yadav, A., 2019. Stakeholders prioritisation of climate-smart agriculture interventions: Evaluation of a framework. Agricultural Systems 174, 23–31.</td><td>CSA prioritisation</td><td>Allocation of resources to CSA interventions. Allows to construct a portfolio of CSA interventions suitable for local context</td><td>Policy-makers at the local level, development organizations and the private sector for investment decision-making</td><td>India (Maharashtra).</td></tr><tr><td>5. CSA-RA (Mwongera et al. 2017)</td><td>Mwongera, C., Shikuku, K.M., Twyman, J., Läderach, P., Ampaire, E., Van Asten, P., Twomlow, S., Winowiecki, L.A., 2017. Climate smart agriculture rapid appraisal (CSA-RA): A tool for prioritizing context-specific climate smart agriculture technologies. Agricultural Systems 151, 192–203.</td><td>CSA prioritisation</td><td>Choice of CSA practices to implement. Helps to identify the key challenges in local smallholder agricultural systems, and to prioritize a set of CSA practices that can respond to such challenges.</td><td>Planners of CSA interventions and investor at the district level and across the sub-region</td><td>Tanzania and Uganda</td></tr><tr><td>6. A generic framework for targeting, out-scaling and prioritizing CSA interventions in agricultural systems (Notenbaert et al., 2017)</td><td>Notenbaert, A., Pfeifer, C., Silvestri, S., Herrero, M., 2017. Targeting, out-scaling and prioritizing climate-smart interventions in agricultural systems: Lessons from applying a generic framework to the livestock sector in sub-Saharan Africa. Agricultural Systems 151, 153–162.</td><td>CSA prioritisation</td><td>Allocation of resources to CSA interventions. Helps to determine which interventions are likely to reach the greatest possible positive impact across the different objectives of food and nutrition security, CC mitigation and CC adaptation.</td><td>Planners of CSA interventions</td><td>Applied in easter Africa region and in Northern Tanzania</td></tr><tr><td>7. MWG model (Ouedraogo et al. 2018, Chiputwa et al., 2020)</td><td>Chiputwa, B., Wainaina, P., Nakelse, T., Makui, P., Zougmoré, R.B., Ndiaye, O., Minang, P.A., 2020. Transforming climate science into usable services: The effectiveness of co-production in promoting uptake of climate information by smallholder farmers in Senegal. Climate Services 20, 100, 203.</td><td>CIS co-production</td><td>Risk management decisions (decision-makers), Choice of farming practices (farmers) during rain season</td><td>National sectoral decision-makers / farmers</td><td>Senegal</td></tr></tbody></table>

(continued on next page)
<table><caption>**Table 2 (continued)**</caption><thead><tr><th></th><th>Decision support framework/process/ platform</th><th>Reference</th><th>Category</th><th>What decision it supports and how?</th><th>End-users/level</th><th>Applied in</th></tr></thead><tbody><tr><td>8.</td><td>PICSA (Dayamba et al. 2018, Clarkson et al. 2019, Staub and Clarkson 2021)</td><td>Dayamba, D.S., Ky-Dembele, C., Bayala, J., Dorward, P., Clarkson, G., Sanogo, D., Diop Mamadou, L., Traoré, I., Diakité, A., Nenkam, A., Binam, J.N., Ouedraogo, M., Zougmore, R., 2018. Assessment of the use of Participatory Integrated Climate Services for Agriculture (PICSA) approach by farmers to manage climate risk in Mali and Senegal. Climate Services 12, 27–35. Clarkson, G., Dorward, P., Osbahr, H., Torgbor, F., Kankam-Boadu, I., 2019. An investigation of the effects of PICSA on smallholder farmers' decision-making and livelihoods when implemented at large scale – The case of Northern Ghana. Climate Services 14, 1–14. Staub, C.G., Clarkson, G., 2021. Farmer-led participatory extension leads Haitian farmers to anticipate climate-related risks and adjust livelihood strategies. Journal of Rural Studies 81, 235–245.</td><td>CIS co-production</td><td>Choice of farming practices by farmers: the timing of activities such as sowing dates, soil and water management, selection of crop varieties, fertilizer management and overall planning. Adaptation of livelihood options to local climate features and farmers' circumstances.</td><td>Farmers</td><td>Applied in 20 countries including: Mali, Senegal, Ghana, Haiti</td></tr><tr><td>9.</td><td>WCIS co-production process (Gbangou et al., 2020)</td><td>Gbangou, T., Sarku, R., Slobbe, E. V., Ludwig, F., Kranjac-Berisavljevic, G., Paparrizos, S., 2020. Coproducing Weather Forecast Information with and for Smallholder Farmers in Ghana: Evaluation and Design Principles. Atmosphere 11, 902.</td><td>CIS co-production</td><td>Choice of farming practices during rain season</td><td>Farmers</td><td>Piloted in Ghana</td></tr><tr><td>10.</td><td>Farmer field school approach to deliver CIS (Kumar et al. 2021)</td><td>Kumar, U., Werners, S.E., Paparrizos, S., Datta, D.K., Ludwig, F., 2021. Co-producing climate information services with smallholder farmers in the Lower Bengal Delta: How forecast visualization and communication support farmers' decision-making. Climate Risk Management 33, 100–346.</td><td>CIS co-production</td><td>Choice of farming practices</td><td>Farmers</td><td>Bangladesh</td></tr><tr><td>11.</td><td>IMD-AAS (AGRIMET platform) as applied in Maharashtra (Vedeld et al., 2020)</td><td>Vedeld, T., Hofstad, H., Mathur, M., Büker, P., Stordal, F., 2020. Reaching out? Governing weather and climate services (WCS) for farmers. Environmental Science & Policy 104, 208–216.</td><td>CIS co-production</td><td>Choice of farming practices</td><td>Farmers</td><td>India, Maharashtra</td></tr><tr><td>12.</td><td>Hydro-climatic EVO (Nyadzi et al., 2018)</td><td>Nyadzi, E., Nyamekye, A.B., Werners, S.E., Biesbroek, R.G., Dewulf, A., Slobbe, E.V., Long, H. P., Termeer, C.J.A.M., Ludwig, F., 2018. Diagnosing the potential of hydro-climatic information services to support rice farming in northern Ghana. NJAS: Wageningen Journal of Life Sciences 86–87, 51–63.</td><td>CIS co-production</td><td>Choice of farming practices</td><td>Farmers</td><td>Projected in Northern Ghana.</td></tr><tr><td>13.</td><td>A framework to co-design climate-smart farming systems with local stakeholders (Andrieu et al., 2019)</td><td>Andrieu, N., Howland, F., Acosta-Alba, I., Le Coq, J.-F., Osorio-Garcia, A.M., Martinez-Baron, D., Gamba-Trimiño, C., Loboguerrero, A.M., Chia, E., 2019. Co-designing Climate-Smart Farming Systems With Local Stakeholders: A Methodological Framework for Achieving Large-Scale Change. Front. Sustain. Food Syst. 3, 37</td><td>Farming system design</td><td>Choice of farming practices by farmers</td><td>Farmers</td><td>Applied in Honduras and Columbia on the farm-scale</td></tr></tbody></table>

(continued on next page)
<table><caption>**Table 2 (continued)**</caption><thead><tr><th></th><th>Decision support framework/process/ platform</th><th>Reference</th><th>Category</th><th>What decision it supports and how?</th><th>End-users/level</th><th>Applied in</th></tr></thead><tbody><tr><td>8.</td><td>PICSA (Dayamba et al. 2018, Clarkson et al. 2019, Staub and Clarkson 2021)</td><td>Dayamba, D.S., Ky-Dembele, C., Bayala, J., Dorward, P., Clarkson, G., Sanogo, D., Diop Mamadou, L., Traoré, I., Diakité, A., Nenkam, A., Binam, J.N., Ouedraogo, M., Zougmore, R., 2018. Assessment of the use of Participatory Integrated Climate Services for Agriculture (PICSA) approach by farmers to manage climate risk in Mali and Senegal. Climate Services 12, 27–35. Clarkson, G., Dorward, P., Osbahr, H., Torgbor, F., Kankam-Boadu, I., 2019. An investigation of the effects of PICSA on smallholder farmers' decision-making and livelihoods when implemented at large scale – The case of Northern Ghana. Climate Services 14, 1–14. Staub, C.G., Clarkson, G., 2021. Farmer-led participatory extension leads Haitian farmers to anticipate climate-related risks and adjust livelihood strategies. Journal of Rural Studies 81, 235–245.</td><td>CIS co-production</td><td>Choice of farming practices by farmers: the timing of activities such as sowing dates, soil and water management, selection of crop varieties, fertilizer management and overall planning. Adaptation of livelihood options to local climate features and farmers' circumstances.</td><td>Farmers</td><td>Applied in 20 countries including: Mali, Senegal, Ghana, Haiti</td></tr><tr><td>9.</td><td>WCIS co-production process (Gbangou et al., 2020)</td><td>Gbangou, T., Sarku, R., Slobbe, E. V., Ludwig, F., Kranjac-Berisavljevic, G., Paparrizos, S., 2020. Coproducing Weather Forecast Information with and for Smallholder Farmers in Ghana: Evaluation and Design Principles. Atmosphere 11, 902.</td><td>CIS co-production</td><td>Choice of farming practices during rain season</td><td>Farmers</td><td>Piloted in Ghana</td></tr><tr><td>10.</td><td>Farmer field school approach to deliver CIS (Kumar et al. 2021)</td><td>Kumar, U., Werners, S.E., Paparrizos, S., Datta, D.K., Ludwig, F., 2021. Co-producing climate information services with smallholder farmers in the Lower Bengal Delta: How forecast visualization and communication support farmers' decision-making. Climate Risk Management 33, 100–346.</td><td>CIS co-production</td><td>Choice of farming practices</td><td>Farmers</td><td>Bangladesh</td></tr><tr><td>11.</td><td>IMD-AAS (AGRIMET platform) as applied in Maharashtra (Vedeld et al., 2020)</td><td>Vedeld, T., Hofstad, H., Mathur, M., Büker, P., Stordal, F., 2020. Reaching out? Governing weather and climate services (WCS) for farmers. Environmental Science & Policy 104, 208–216.</td><td>CIS co-production</td><td>Choice of farming practices</td><td>Farmers</td><td>India, Maharashtra</td></tr><tr><td>12.</td><td>Hydro-climatic EVO (Nyadzi et al., 2018)</td><td>Nyadzi, E., Nyamekye, A.B., Werners, S.E., Biesbroek, R.G., Dewulf, A., Slobbe, E.V., Long, H. P., Termeer, C.J.A.M., Ludwig, F., 2018. Diagnosing the potential of hydro-climatic information services to support rice farming in northern Ghana. NJAS: Wageningen Journal of Life Sciences 86–87, 51–63.</td><td>CIS co-production</td><td>Choice of farming practices</td><td>Farmers</td><td>Projected in Northern Ghana.</td></tr><tr><td>13.</td><td>A framework to co-design climate-smart farming systems with local stakeholders (Andrieu et al., 2019)</td><td>Andrieu, N., Howland, F., Acosta-Alba, I., Le Coq, J.-F., Osorio-Garcia, A.M., Martinez-Baron, D., Gamba-Trimiño, C., Loboguerrero, A.M., Chia, E., 2019. Co-designing Climate-Smart Farming Systems With Local Stakeholders: A Methodological Framework for Achieving Large-Scale Change. Front. Sustain. Food Syst. 3, 37</td><td>Farming system design</td><td>Choice of farming practices by farmers</td><td>Farmers</td><td>Applied in Honduras and Columbia on the farm-scale</td></tr></tbody></table>

(continued on next page)
<table><caption>**Table 2 (continued)**</caption><thead><tr><th>Decision support framework/process/ platform</th><th>Reference</th><th>Category</th><th>What decision it supports and how?</th><th>End-users/level</th><th>Applied in</th></tr></thead><tbody><tr><td>14. A tool for tree selection in smallholder agroforestry farms (Van der Wolf et al. 2019)</td><td>Van Der Wolf, J., Jassogne, L., Gram, G., Vaast, P., 2019. Turning local knowledge on agroforestry into an online decision-support tool for tree selection in smallholders' farms. Ex. Agric. 55, 50–66. doi:https://doi.org/10.1017/S001447971600017X</td><td>Farming system design</td><td>The optimal shade tree selection for coffee and cocoa agroforestry systems</td><td>Farmers. Next-users: advisory services</td><td>Developed for Uganda and Ghana. Implemented on the virtual farms.</td></tr><tr><td>15. DESIRE-DSS (Schwilch et al., 2012)</td><td>Schwilch, G., Bachmann, F., de Graaff, J., 2012. Decision support for selecting SLM technologies with stakeholders. Applied Geography 34, 86–98.</td><td>Sustainable management of SES</td><td>Selecting, testing, and upscaling sustainable land use strategies to mitigate desertification</td><td>Land users</td><td>Morocco, Tunisia, Botswana and Chile (among the others)</td></tr><tr><td>16. Interdisciplinary, multi-scaled, and integrated socio-ecological approach, including MARS DSS (Berger et al. 2019)</td><td>Berger, C., Bieri, M., Bradshaw, K., Brümmer, C., Clemen, T., Hickler, T., Kutsch, W.L., Lenfers, U.A., Martens, C., Midgley, G.F., Mukwashi, K., Odipo, V., Scheiter, S., Schmullius, C., Baade, J., du Toit, J.C.O., Scholes, R.J., Smit, I.P.J., Stevens, N., Twine, W., 2019. Linking scales and disciplines: an interdisciplinary cross-scale approach to supporting climate-relevant ecosystem management. Arias-Hidalgo, M., Villa-Cox, G., Griensven, A.V., Solórzano, G., Villa-Cox, R., Mynett, A.E., Debels, P., 2013. A decision framework for wetland management in a river basin context: The “Abras de Mantequilla” case study in the Guayas River Basin, Ecuador. Environmental Science & Policy 34, 103–114.</td><td>Sustainable management of SES</td><td>Development of sustainable and climate resilient land use strategies.</td><td>Local land-use decision-makers</td><td>South Africa</td></tr><tr><td>17. WET-WIN (Arias-Hidalgo et al., 2013)</td><td>Jennings, S.A., Challinor, A.J., Smith, P., Macdiarmid, J.I., Pope, E., Chapman, S., Bradshaw, C., Clark, H., Vetter, S., Fitton, N., King, R., Mwamakamba, S., Madzivhandila, T., Mashingaidze, I., Chomba, C., Nawiko, M., Nyhodo, B., Mazibuko, N., Yeki, P., Kuwali, P., Kambwiri, A., Kazi, V., Kiama, A., Songole, A., Coskeran, H., Quinn, C., Sallu, S., Dougill, A., Whitfield, S., Kunin, B., Meebelo, N., Jamali, A., Kantande, D., Makundi, P., Mbungu, W., Kayula, F., Walker, S., Zimba, S., Yamdeu, J.H.G., Kapulu, N., Galdos, M.V., Eze, S., Tripathi, H. G., Sait, S.M., Kepinski, S., Likoya, E., Greathead, H., Smith, H.E., Mahop, M.T., Harwatt, H., Muzammil, M., Horgan, G., Benton, T., 2022. A New Integrated Assessment Framework for Climate-Smart Nutrition Security in sub-Saharan Africa: The Integrated Future Estimator for Emissions and Diets (IFEED). Front. Sustain. Food Syst. 6, 868,189.</td><td>Sustainable management of SES</td><td>Wetland management in a river basin context under data scarce conditions</td><td>Wetland managers</td><td>Ecuador</td></tr><tr><td>18. IFEED (Jennings et al. 2022)</td><td>Jennings, S.A., Challinor, A.J., Smith, P., Macdiarmid, J.I., Pope, E., Chapman, S., Bradshaw, C., Clark, H., Vetter, S., Fitton, N., King, R., Mwamakamba, S., Madzivhandila, T., Mashingaidze, I., Chomba, C., Nawiko, M., Nyhodo, B., Mazibuko, N., Yeki, P., Kuwali, P., Kambwiri, A., Kazi, V., Kiama, A., Songole, A., Coskeran, H., Quinn, C., Sallu, S., Dougill, A., Whitfield, S., Kunin, B., Meebelo, N., Jamali, A., Kantande, D., Makundi, P., Mbungu, W., Kayula, F., Walker, S., Zimba, S., Yamdeu, J.H.G., Kapulu, N., Galdos, M.V., Eze, S., Tripathi, H. G., Sait, S.M., Kepinski, S., Likoya, E., Greathead, H., Smith, H.E., Mahop, M.T., Harwatt, H., Muzammil, M., Horgan, G., Benton, T., 2022. A New Integrated Assessment Framework for Climate-Smart Nutrition Security in sub-Saharan Africa: The Integrated Future Estimator for Emissions and Diets (IFEED). Front. Sustain. Food Syst. 6, 868,189.</td><td>Climate-smart food systems policy</td><td>Delivering evidence to inform national-level policies on climate-smart and nutrition-secure food systems through combining models with in-country knowledge and expert academic judgement</td><td>National-level policymakers</td><td>Applied in Malawi, South Africa, Tanzania and Zambia</td></tr></tbody></table>

must be: context-based, pluralistic, goal-oriented and interactive.

Context-based means that the process needs to be situated within, rather than parachuted into, a given social, economic and ecological context. In the case of decision-support, this implies that it will take into

account both the scale (local, regional, national, etc.) and the circumstances in which decisions are being taken as well as needs, interests and beliefs of different actors involved in decision-making or affected by it. A co-production process shouldn't overlook any enabling or constraining
<table><caption>**Table 2 (continued)**</caption><thead><tr><th>Decision support framework/process/ platform</th><th>Reference</th><th>Category</th><th>What decision it supports and how?</th><th>End-users/level</th><th>Applied in</th></tr></thead><tbody><tr><td>14. A tool for tree selection in smallholder agroforestry farms (Van der Wolf et al. 2019)</td><td>Van Der Wolf, J., Jassogne, L., Gram, G., Vaast, P., 2019. Turning local knowledge on agroforestry into an online decision-support tool for tree selection in smallholders' farms. Ex. Agric. 55, 50–66. doi: https://doi.org/10.1017/S001447971600017X</td><td>Farming system design</td><td>The optimal shade tree selection for coffee and cocoa agroforestry systems</td><td>Farmers. Next-users: advisory services</td><td>Developed for Uganda and Ghana. Implemented on the virtual farms.</td></tr><tr><td>15. DESIRE-DSS (Schwilch et al., 2012)</td><td>Schwilch, G., Bachmann, F., de Graaff, J., 2012. Decision support for selecting SLM technologies with stakeholders. Applied Geography 34, 86–98.</td><td>Sustainable management of SES</td><td>Selecting, testing, and upscaling sustainable land use strategies to mitigate desertification</td><td>Land users</td><td>Morocco, Tunisia, Botswana and Chile (among the others)</td></tr><tr><td>16. Interdisciplinary, multi-scaled, and integrated socio-ecological approach, including MARS DSS (Berger et al. 2019)</td><td>Berger, C., Bieri, M., Bradshaw, K., Brümmer, C., Clemen, T., Hickler, T., Kutsch, W.L., Lenfers, U.A., Martens, C., Midgley, G.F., Mukwashi, K., Odipo, V., Scheiter, S., Schmullius, C., Baade, J., du Toit, J.C.O., Scholes, R.J., Smit, I.P.J., Stevens, N., Twine, W., 2019. Linking scales and disciplines: an interdisciplinary cross-scale approach to supporting climate-relevant ecosystem management. Arias-Hidalgo, M., Villa-Cox, G., Griensven, A.V., Solórzano, G., Villa-Cox, R., Mynett, A.E., Debels, P., 2013. A decision framework for wetland management in a river basin context: The “Abras de Mantequilla” case study in the Guayas River Basin, Ecuador. Environmental Science & Policy 34, 103–114.</td><td>Sustainable management of SES</td><td>Development of sustainable and climate resilient land use strategies.</td><td>Local land-use decision-makers</td><td>South Africa</td></tr><tr><td>17. WET-WIN (Arias-Hidalgo et al., 2013)</td><td>Jennings, S.A., Challinor, A.J., Smith, P., Macdiarmid, J.I., Pope, E., Chapman, S., Bradshaw, C., Clark, H., Vetter, S., Fitton, N., King, R., Mwamakamba, S., Madzivhandila, T., Mashingaidze, I., Chomba, C., Nawiko, M., Nyhodo, B., Mazibuko, N., Yeki, P., Kuwali, P., Kambwiri, A., Kazi, V., Kiama, A., Songole, A., Coskeran, H., Quinn, C., Sallu, S., Dougill, A., Whitfield, S., Kunin, B., Meebelo, N., Jamali, A., Kantande, D., Makundi, P., Mbungu, W., Kayula, F., Walker, S., Zimba, S., Yamdeu, J.H.G., Kapulu, N., Galdos, M.V., Eze, S., Tripathi, H. G., Sait, S.M., Kepinski, S., Likoya, E., Greathead, H., Smith, H.E., Mahop, M.T., Harwatt, H., Muzammil, M., Horgan, G., Benton, T., 2022. A New Integrated Assessment Framework for Climate-Smart Nutrition Security in sub-Saharan Africa: The Integrated Future Estimator for Emissions and Diets (IFEED). Front. Sustain. Food Syst. 6, 868,189.</td><td>Sustainable management of SES</td><td>Wetland management in a river basin context under data scarce conditions</td><td>Wetland managers</td><td>Ecuador</td></tr><tr><td>18. IFEED (Jennings et al. 2022)</td><td>Jennings, S.A., Challinor, A.J., Smith, P., Macdiarmid, J.I., Pope, E., Chapman, S., Bradshaw, C., Clark, H., Vetter, S., Fitton, N., King, R., Mwamakamba, S., Madzivhandila, T., Mashingaidze, I., Chomba, C., Nawiko, M., Nyhodo, B., Mazibuko, N., Yeki, P., Kuwali, P., Kambwiri, A., Kazi, V., Kiama, A., Songole, A., Coskeran, H., Quinn, C., Sallu, S., Dougill, A., Whitfield, S., Kunin, B., Meebelo, N., Jamali, A., Kantande, D., Makundi, P., Mbungu, W., Kayula, F., Walker, S., Zimba, S., Yamdeu, J.H.G., Kapulu, N., Galdos, M.V., Eze, S., Tripathi, H. G., Sait, S.M., Kepinski, S., Likoya, E., Greathead, H., Smith, H.E., Mahop, M.T., Harwatt, H., Muzammil, M., Horgan, G., Benton, T., 2022. A New Integrated Assessment Framework for Climate-Smart Nutrition Security in sub-Saharan Africa: The Integrated Future Estimator for Emissions and Diets (IFEED). Front. Sustain. Food Syst. 6, 868,189.</td><td>Climate-smart food systems policy</td><td>Delivering evidence to inform national-level policies on climate-smart and nutrition-secure food systems through combining models with in-country knowledge and expert academic judgement</td><td>National-level policymakers</td><td>Applied in Malawi, South Africa, Tanzania and Zambia</td></tr></tbody></table>

must be: context-based, pluralistic, goal-oriented and interactive.

Context-based means that the process needs to be situated within, rather than parachuted into, a given social, economic and ecological context. In the case of decision-support, this implies that it will take into

account both the scale (local, regional, national, etc.) and the circumstances in which decisions are being taken as well as needs, interests and beliefs of different actors involved in decision-making or affected by it. A co-production process shouldn't overlook any enabling or constraining
<table><caption>**Table 3**
Geographical repartition of frameworks' applications.</caption><thead><tr><th>Africa</th><th></th><th></th><th>South and Central America</th><th></th><th>South Asia</th><th></th></tr></thead><tbody><tr><td rowspan="2">West</td><td>Ghana</td><td>4</td><td>Latin America</td><td>1</td><td>India</td><td>2</td></tr><tr><td>Mali</td><td>2</td><td>Chile</td><td>1</td><td>Bangladesh</td><td>1</td></tr><tr><td rowspan="5">East</td><td>Senegal</td><td>1</td><td>Columbia</td><td>1</td><td></td><td></td></tr><tr><td>East African region</td><td>1</td><td>Equador</td><td>1</td><td></td><td></td></tr><tr><td>Uganda</td><td>2</td><td>Honduras</td><td>1</td><td></td><td></td></tr><tr><td>Kenya</td><td>1</td><td>Haiti</td><td>1</td><td></td><td></td></tr><tr><td>Tanzania</td><td>3</td><td></td><td></td><td></td><td></td></tr><tr><td>South-East</td><td>Malawi</td><td>1</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">South</td><td>Zambia</td><td>1</td><td></td><td></td><td></td><td></td></tr><tr><td>Botswana</td><td>1</td><td></td><td></td><td></td><td></td></tr><tr><td>South Africa</td><td>2</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">North</td><td>Morocco</td><td>1</td><td></td><td></td><td></td><td></td></tr><tr><td>Tunisia</td><td>1</td><td></td><td></td><td></td><td></td></tr><tr><td>21</td><td>6</td><td></td><td>3</td><td></td><td></td></tr></tbody></table>

factors from its context. Pluralistic principle brings a multi-stakeholder process beyond “including” different actors and making them “participate”. It calls for a certain posture of those designing and leading the process (in our case the researchers), that explicitly recognizes and draws on the multiple ways of doing but also knowing. This is the place to ask questions about knowledge integration, understood as a “desire to assimilate heterogeneous knowledge (via data, analysis, or claims)” (Klenk and Meehan, 2015). The concept of knowledge integration is part of the co-production discourse and we aim to find out whether and how it is practiced in decision-support. The second element of this posture is the acknowledgement of the power asymmetries between different ways of doing and knowing. A pluralistic process includes tools that factor in power imbalance between its participants. In a goal-oriented process all the participants know and share the process goals, as they had previously agreed upon them. There is a collective understanding of how to recognize if the process is going in the right direction at its different stages and whether it is ultimately successful. It is also understood that goals may evolve and thus some kind of mechanism to review or renegotiate the goals and potentially re-orient the actions should be integrated in the process. And finally, the interactive principle emphasizes the importance of the nature, the quantity and the quality of interactions

between different participants of the process. The underlying assumption is that a quality interaction impacts all interacting sides – thus mutual learning and change of perspectives and not the transfer of knowledge will characterize a co-production process (Norström et al., 2020). Here we ask about the presence of non-academic stakeholders in the process – who, how, at what stage and with what role was involved?

The four co-production principles were used as a template to organize the extracted data. We decided to use these principles as a reference, not only because we found them relevant, but also because the authors provide clear suggestions on how to specifically assess them. We additionally drew on the work of Steger et al. (2021) on the best practices in environmental trans-disciplinary work, to complete our list of qualitative indicators for each of the principles (Table 4).

In this paper we apply the instrumental approach to co-production, that is also referred to as normative approach (Wyborn et al., 2019), seen as a practice to be applied to improve the outcomes of adaptation projects. As demonstrated in a literature review by Bremer and Meish (Bremer and Meisch, 2017), such an approach is predominant in climate science. Jagannathan et al. (2020) report that the majority of effects of using co-production in climate-change adaptation projects falls in the category of benefits from decision-relevant knowledge and services, rather than transformation of norms and structures guiding the science-society relationship.

# 4. Findings and discussion

## 4.1. How the frameworks are based in their context

Our investigation into the context-based principle in the revised frameworks brought results that fall under the category of assessing and using some elements of local context: incorporating existing spaces, tapping into on-going processes or giving a place to actors who are already recognized by their communities and whose social capital can potentially be used for the benefit of the project. One example is using already established interaction spaces from previous/on-going projects: existing innovation platforms [6,13] or existing farmer field schools [13]. Another example is exploring how, in a particular context, knowledge is produced, shared, and connected with decision-making,

<table><caption>**Table 4**
The four co-production principles and their indicators (based on Norström et al., 2020 and Steger et al., 2021).</caption><thead><tr><th>Co-production principles</th><th>context-based</th><th>pluralistic</th><th>goal-oriented</th><th>interactive</th></tr></thead><tbody><tr><td rowspan="5">Indicators</td><td>The context, history, or on-going initiatives surrounding the problem or the geographic area in which the project is implemented are assessed</td><td>Some form of stakeholder mapping or social network analysis is performed</td><td>A theory of Change (ToC) is collectively developed</td><td>There are interactions among participants at different stages of the process</td></tr><tr><td>It is verified whether there is a shared understanding of key concepts and terminology</td><td>The framework brings together researchers (from different disciplines) and people from other sectors (from for example, government, business, civil society, local and indigenous communities) representing different types of knowledge and expertise (experiential, local, traditional, academic, official) and skills (analysis, translation, synthesis, facilitation, evaluation)</td><td>Project goals are developed collectively</td><td>All participating groups of actors have an active role in the process (no token participation)</td></tr><tr><td>Individuals with experience working locally with participant groups are involved in the process</td><td>Other dimensions of diversity are included, such as gender, ethnicity, age and nationality</td><td>There is an agreed measure of success of the project</td><td>There is two-way communication between participants</td></tr><tr><td>Existing networks, communities of practice, innovation platforms, etc. are identified and connected with the process</td><td>The process engages with power imbalances</td><td>Data collection methods are collectively developed</td><td>Criteria of credibility (of different knowledge systems) are dealt with in a respectful way</td></tr><tr><td>The process contributes to the pre-existing goals and objectives of partners</td><td>The process is allowing the knowledge and perceptions of different participants to be mobilized and articulated into forms that can be shared with others</td><td>Results are interpreted collectively</td><td>Mutual learning</td></tr><tr><td></td><td></td><td>The process doesn't exclude conflicting perspectives and allows negotiations</td><td>The process is collectively evaluated</td><td></td></tr></tbody></table>
<table><caption>**Table 3**
Geographical repartition of frameworks' applications.</caption><thead><tr><th>Africa</th><th></th><th>South and Central America</th><th>South Asia</th><th></th></tr></thead><tbody><tr><td rowspan="2">West</td><td>Ghana</td><td>4 Latin America</td><td>1 India</td><td>2</td></tr><tr><td>Mali</td><td>2 Chile</td><td>1 Bangladesh</td><td>1</td></tr><tr><td rowspan="5">East</td><td>Senegal</td><td>1 Columbia</td><td>1</td><td></td></tr><tr><td>East African region</td><td>1 Equador</td><td>1</td><td></td></tr><tr><td>Uganda</td><td>2 Honduras</td><td>1</td><td></td></tr><tr><td>Kenya</td><td>1 Haiti</td><td>1</td><td></td></tr><tr><td>Tanzania</td><td>3</td><td></td><td></td></tr><tr><td>South-East</td><td>Malawi</td><td>1</td><td></td><td></td></tr><tr><td rowspan="3">South</td><td>Zambia</td><td>1</td><td></td><td></td></tr><tr><td>Botswana</td><td>1</td><td></td><td></td></tr><tr><td>South Africa</td><td>2</td><td></td><td></td></tr><tr><td rowspan="3">North</td><td>Morocco</td><td>1</td><td></td><td></td></tr><tr><td>Tunisia</td><td>1</td><td></td><td></td></tr><tr><td>21</td><td>6</td><td>3</td><td></td></tr></tbody></table>

factors from its context. Pluralistic principle brings a multi-stakeholder process beyond “including” different actors and making them “participate”. It calls for a certain posture of those designing and leading the process (in our case the researchers), that explicitly recognizes and draws on the multiple ways of doing but also knowing. This is the place to ask questions about knowledge integration, understood as a “desire to assimilate heterogeneous knowledge (via data, analysis, or claims)” (Klenk and Meehan, 2015). The concept of knowledge integration is part of the co-production discourse and we aim to find out whether and how it is practiced in decision-support. The second element of this posture is the acknowledgement of the power asymmetries between different ways of doing and knowing. A pluralistic process includes tools that factor in power imbalance between its participants. In a goal-oriented process all the participants know and share the process goals, as they had previously agreed upon them. There is a collective understanding of how to recognize if the process is going in the right direction at its different stages and whether it is ultimately successful. It is also understood that goals may evolve and thus some kind of mechanism to review or renegotiate the goals and potentially re-orient the actions should be integrated in the process. And finally, the interactive principle emphasizes the importance of the nature, the quantity and the quality of interactions

between different participants of the process. The underlying assumption is that a quality interaction impacts all interacting sides – thus mutual learning and change of perspectives and not the transfer of knowledge will characterize a co-production process (Norström et al., 2020). Here we ask about the presence of non-academic stakeholders in the process – who, how, at what stage and with what role was involved?

The four co-production principles were used as a template to organize the extracted data. We decided to use these principles as a reference, not only because we found them relevant, but also because the authors provide clear suggestions on how to specifically assess them. We additionally drew on the work of Steger et al. (2021) on the best practices in environmental trans-disciplinary work, to complete our list of qualitative indicators for each of the principles (Table 4).

In this paper we apply the instrumental approach to co-production, that is also referred to as normative approach (Wyborn et al., 2019), seen as a practice to be applied to improve the outcomes of adaptation projects. As demonstrated in a literature review by Bremer and Meish (Bremer and Meisch, 2017), such an approach is predominant in climate science. Jagannathan et al. (2020) report that the majority of effects of using co-production in climate-change adaptation projects falls in the category of benefits from decision-relevant knowledge and services, rather than transformation of norms and structures guiding the science-society relationship.

# 4. Findings and discussion

## 4.1. How the frameworks are based in their context

Our investigation into the context-based principle in the revised frameworks brought results that fall under the category of assessing and using some elements of local context: incorporating existing spaces, tapping into on-going processes or giving a place to actors who are already recognized by their communities and whose social capital can potentially be used for the benefit of the project. One example is using already established interaction spaces from previous/on-going projects: existing innovation platforms [6,13] or existing farmer field schools [13]. Another example is exploring how, in a particular context, knowledge is produced, shared, and connected with decision-making,

<table><caption>**Table 4**
The four co-production principles and their indicators (based on Norström et al., 2020 and Steger et al., 2021).</caption><thead><tr><th>Co-production principles</th><th>context-based</th><th>pluralistic</th><th>goal-oriented</th><th>interactive</th></tr></thead><tbody><tr><td rowspan="5">Indicators</td><td>The context, history, or on-going initiatives surrounding the problem or the geographic area in which the project is implemented are assessed</td><td>Some form of stakeholder mapping or social network analysis is performed</td><td>A theory of Change (ToC) is collectively developed</td><td>There are interactions among participants at different stages of the process</td></tr><tr><td>It is verified whether there is a shared understanding of key concepts and terminology</td><td>The framework brings together researchers (from different disciplines) and people from other sectors (from for example, government, business, civil society, local and indigenous communities) representing different types of knowledge and expertise (experiential, local, traditional, academic, official) and skills (analysis, translation, synthesis, facilitation, evaluation)</td><td>Project goals are developed collectively</td><td>All participating groups of actors have an active role in the process (no token participation)</td></tr><tr><td>Individuals with experience working locally with participant groups are involved in the process</td><td>Other dimensions of diversity are included, such as gender, ethnicity, age and nationality</td><td>There is an agreed measure of success of the project</td><td>There is two-way communication between participants</td></tr><tr><td>Existing networks, communities of practice, innovation platforms, etc. are identified and connected with the process</td><td>The process engages with power imbalances</td><td>Data collection methods are collectively developed</td><td>Criteria of credibility (of different knowledge systems) are dealt with in a respectful way</td></tr><tr><td>The process contributes to the pre-existing goals and objectives of partners</td><td>The process is allowing the knowledge and perceptions of different participants to be mobilized and articulated into forms that can be shared with others</td><td>Results are interpreted collectively</td><td>Mutual learning</td></tr><tr><td></td><td></td><td>The process doesn't exclude conflicting perspectives and allows negotiations</td><td>The process is collectively evaluated</td><td></td></tr></tbody></table>
and trying to incorporate the existing dynamics into the framework. It
may be starting from screening current practices, innovations or
endogenous adaptation strategies [14,16] that can be then integrated in
the process, for example by including current practices among CSA
options to be assessed. It is important for decision-support to understand
the current practices of users, also as a way of anticipating and pre-
venting usability gap – some authors pointed out that users may delib-
erately ignore potentially useful information when it would disrupt their
current practices that they prefer to continue (Rice et al., 2009; Porter
and Dessai, 2017). Other examples include bringing into the decision-
support process actors such as farmers identified as leaders of opinion
in their communities [5,7,12], extension agents who have a good
working relationship with farmers [4,9], experts recognized in their
networks [3], or actors recognized as supportive to processes that may
help to reach framework's objectives [7,13], or for their outreach po-
tential [7,9]. The assumption seems to be that these actors feel enough
ownership of the process outputs to share them with the members of
their networks, mobilizing their own resources: social capital, trust,
legitimacy. Furthermore, capacity building of the participants, for
example training them in the use of equipment [9] or facilitation [8] is a
strategy that can help preserving at least some of the co-production
dynamics after the end of the project. These strategies of capitalising
on built networks, expertise and methods are also promoted in the Open
Innovation literature as a factor that supports sustainability transitions
(Rossing et al., 2021). It has to be noted, however, that next to the ad-
vantages listed above, working on the basis of existing networks may
have a disadvantage of reproducing existing power relations – potential
trade-offs between context-based and pluralistic principles should not be
overlooked.

Not all the reviewed frameworks fulfill the requirements of the context-based principle. We haven't found examples of the other points suggested by Norström et al. (2020): making sure that problem framing is locally shared and contributing to the pre-existing goals and objectives of partners. Following these suggestions, the most direct way of grounding a decision-support process in the local context would be to respond to a direct local demand. This was not the case in any of the reviewed frameworks. Most of the decision-support frameworks are developed in the context of research projects or programs constructed based on expert-assessment of research needs, and subjected to usual constraints of time and financing (projects need to be designed before extensive fieldwork can be financed). Whereas inside the projects the local context is explored with stakeholders and taken into account in decision-making process, the choice to work on a given topic or in a given area is rarely demand-driven. The MWG model from Senegal [7] stands out, as even if initiated by research, it is now carried out by the National Meteorological Agency (ANACIM), the location and composition of the local groups is locally decided, the local authorities are mandated to keep it running.

## 4.2. How the frameworks are made pluralistic

When exploring the pluralistic principle in the analyzed frameworks we looked at: the process of selecting the participants (both academic and non-academic), what types of knowledge (traditional, local, tacit) were integrated and in what way, was there space for conflicting/ competing views, if and how power imbalances were dealt with.

Selection of participants seems to be a key moment for applying the pluralistic principle, where “plurality” can be – and is – explicitly used as a selection criterion. In the studied frameworks, plurality is most often understood as a diversity of stakeholders (which alone is not sufficient to cover the pluralistic principle). One framework postulates including stakeholders representing the diversity of farming situations in regards to farming type, scale and land-ownership situation [15], another – the diversity of food system expertise [18]. When it comes to more political criteria, one framework requires selecting participants with the diversity of political, scientific, and technical views [1], another one demands a

conscious effort to include disadvantaged groups [8], such as non- and semi-literate farmers. Six frameworks mention gender as a selection criterion [5,7,8,9,10,12] - it is given more attention than any other demographic criterion, which is likely related to gender equity being one of the sustainable development goals. Interestingly, gender criterion is applied to participants such as farmers, and not considered in selecting experts or research team. Reasons to include gender criterion may be very different: in one case it is assuring the representativity of local population [5] while in two other cases it is an attempt to act on existing power relations: assuring gender balance [10] and inclusion of women as a disadvantaged group [8], which is more in line with the pluralistic principle. The reasons may be as well research-related, as found in analyzed frameworks – to increase understanding of differences between men and women in: division of labor, perception of climate change, sharing information, access to decision-making, needs for information. The results of analysis are gender-disaggregated in four of the frameworks, whereas two others reflect upon and take into account gender roles.

Another question that may need revisiting from a co-production perspective is that of representation (or representativeness) of the process participants, criterion emphasized by Wyborn et al. (2019) as crucial. However, it seems insufficient to think of participants in the co-production process solely as representatives of a certain category of actors. According to some authors, the participants in co-production process should be expected to « engage as individuals, with their own particular knowledge, skills, and expertise, rather than as representatives » (Landström et al., 2011). This could offer them a more equal role in the process. Meanwhile, farmers are rarely mobilized because of their knowledge – we noted only two cases: one framework seeks to mobilize farmers' knowledge on tree species [14] and another one requiring experience with local weather forecast [9]. In eight cases non-academic stakeholders other than farmers are mobilized on the basis of their expertise [1,2,3,4,5,15,17,18]. “Local” expertise is explicitly mentioned in over a half of the frameworks reviewed [4,5,7-13,15,17,18], including all the CIS co-production frameworks which are drawing on local forecast. Two of the frameworks mention involving “local-level agricultural experts” [5,7], one talks of “local research team” [15] (although the involvement seems to be due to the project set-up rather than a need for a specific local expertise), and one of “country experts” [18].

Looking at knowledge integration, “traditional” knowledge does not appear in the majority of reviewed frameworks. Two CIS frameworks directly incorporate traditional forecasting along scientific forecasting [9,12] to be used in on-line tools. Another example [14] is a decision-support tool for choosing adapted tree species for coffee agroforestry which directly mobilizes local traditional knowledge on tree species and their potential for eco-system services provision (Van Der Wolf et al., 2019). These three frameworks are the only ones that draw simultaneously on scientific and traditional knowledge.

The types of knowledge that are not easily verbalized or recorded (like tacit knowledge) are not easily incorporated into participatory processes. Harnessing non-explicit knowledge is not mentioned as an objective in any of the reviewed frameworks. However, some of the analyzed frameworks make use of methods that potentially allow it, if facilitated with this objective in mind. For example, space is made for practice in several frameworks, for example through incorporating farmer field schools [10,13], demonstration plots [7,11] or the use of rain gauges by farmers [9].

Another method that appears in two frameworks [5,15] that has a potential of mobilizing non-explicit types of knowledge are *transect walks* – a participatory method that uses controlled “walks” in the community to collect spatial information. In another framework [6] participatory mapping replaces expert GIS – it can also be facilitated in a way that allows drawing on implicit knowledge.

Stakeholder perceptions (including those of researchers) of climate and climate change are also not often explicitly addressed in the process
and trying to incorporate the existing dynamics into the framework. It
may be starting from screening current practices, innovations or
endogenous adaptation strategies [14,16] that can be then integrated in
the process, for example by including current practices among CSA
options to be assessed. It is important for decision-support to understand
the current practices of users, also as a way of anticipating and pre-
venting usability gap – some authors pointed out that users may delib-
erately ignore potentially useful information when it would disrupt their
current practices that they prefer to continue (Rice et al., 2009; Porter
and Dessai, 2017). Other examples include bringing into the decision-
support process actors such as farmers identified as leaders of opinion
in their communities [5,7,12], extension agents who have a good
working relationship with farmers [4,9], experts recognized in their
networks [3], or actors recognized as supportive to processes that may
help to reach framework's objectives [7,13], or for their outreach po-
tential [7,9]. The assumption seems to be that these actors feel enough
ownership of the process outputs to share them with the members of
their networks, mobilizing their own resources: social capital, trust,
legitimacy. Furthermore, capacity building of the participants, for
example training them in the use of equipment [9] or facilitation [8] is a
strategy that can help preserving at least some of the co-production
dynamics after the end of the project. These strategies of capitalising
on built networks, expertise and methods are also promoted in the Open
Innovation literature as a factor that supports sustainability transitions
(Rossing et al., 2021). It has to be noted, however, that next to the ad-
vantages listed above, working on the basis of existing networks may
have a disadvantage of reproducing existing power relations – potential
trade-offs between context-based and pluralistic principles should not be
overlooked.

Not all the reviewed frameworks fulfill the requirements of the context-based principle. We haven't found examples of the other points suggested by Norström et al. (2020): making sure that problem framing is locally shared and contributing to the pre-existing goals and objectives of partners. Following these suggestions, the most direct way of grounding a decision-support process in the local context would be to respond to a direct local demand. This was not the case in any of the reviewed frameworks. Most of the decision-support frameworks are developed in the context of research projects or programs constructed based on expert-assessment of research needs, and subjected to usual constraints of time and financing (projects need to be designed before extensive fieldwork can be financed). Whereas inside the projects the local context is explored with stakeholders and taken into account in decision-making process, the choice to work on a given topic or in a given area is rarely demand-driven. The MWG model from Senegal [7] stands out, as even if initiated by research, it is now carried out by the National Meteorological Agency (ANACIM), the location and composition of the local groups is locally decided, the local authorities are mandated to keep it running.

## 4.2. How the frameworks are made pluralistic

When exploring the pluralistic principle in the analyzed frameworks we looked at: the process of selecting the participants (both academic and non-academic), what types of knowledge (traditional, local, tacit) were integrated and in what way, was there space for conflicting/ competing views, if and how power imbalances were dealt with.

Selection of participants seems to be a key moment for applying the pluralistic principle, where “plurality” can be – and is – explicitly used as a selection criterion. In the studied frameworks, plurality is most often understood as a diversity of stakeholders (which alone is not sufficient to cover the pluralistic principle). One framework postulates including stakeholders representing the diversity of farming situations in regards to farming type, scale and land-ownership situation [15], another – the diversity of food system expertise [18]. When it comes to more political criteria, one framework requires selecting participants with the diversity of political, scientific, and technical views [1], another one demands a

conscious effort to include disadvantaged groups [8], such as non- and semi-literate farmers. Six frameworks mention gender as a selection criterion [5,7,8,9,10,12] - it is given more attention than any other demographic criterion, which is likely related to gender equity being one of the sustainable development goals. Interestingly, gender criterion is applied to participants such as farmers, and not considered in selecting experts or research team. Reasons to include gender criterion may be very different: in one case it is assuring the representativity of local population [5] while in two other cases it is an attempt to act on existing power relations: assuring gender balance [10] and inclusion of women as a disadvantaged group [8], which is more in line with the pluralistic principle. The reasons may be as well research-related, as found in analyzed frameworks – to increase understanding of differences between men and women in: division of labor, perception of climate change, sharing information, access to decision-making, needs for information. The results of analysis are gender-disaggregated in four of the frameworks, whereas two others reflect upon and take into account gender roles.

Another question that may need revisiting from a co-production perspective is that of representation (or representativeness) of the process participants, criterion emphasized by Wyborn et al. (2019) as crucial. However, it seems insufficient to think of participants in the co-production process solely as representatives of a certain category of actors. According to some authors, the participants in co-production process should be expected to « engage as individuals, with their own particular knowledge, skills, and expertise, rather than as representatives » (Landström et al., 2011). This could offer them a more equal role in the process. Meanwhile, farmers are rarely mobilized because of their knowledge – we noted only two cases: one framework seeks to mobilize farmers' knowledge on tree species [14] and another one requiring experience with local weather forecast [9]. In eight cases non-academic stakeholders other than farmers are mobilized on the basis of their expertise [1,2,3,4,5,15,17,18]. “Local” expertise is explicitly mentioned in over a half of the frameworks reviewed [4,5,7-13,15,17,18], including all the CIS co-production frameworks which are drawing on local forecast. Two of the frameworks mention involving “local-level agricultural experts” [5,7], one talks of “local research team” [15] (although the involvement seems to be due to the project set-up rather than a need for a specific local expertise), and one of “country experts” [18].

Looking at knowledge integration, “traditional” knowledge does not appear in the majority of reviewed frameworks. Two CIS frameworks directly incorporate traditional forecasting along scientific forecasting [9,12] to be used in on-line tools. Another example [14] is a decision-support tool for choosing adapted tree species for coffee agroforestry which directly mobilizes local traditional knowledge on tree species and their potential for eco-system services provision (Van Der Wolf et al., 2019). These three frameworks are the only ones that draw simultaneously on scientific and traditional knowledge.

The types of knowledge that are not easily verbalized or recorded (like tacit knowledge) are not easily incorporated into participatory processes. Harnessing non-explicit knowledge is not mentioned as an objective in any of the reviewed frameworks. However, some of the analyzed frameworks make use of methods that potentially allow it, if facilitated with this objective in mind. For example, space is made for practice in several frameworks, for example through incorporating farmer field schools [10,13], demonstration plots [7,11] or the use of rain gauges by farmers [9].

Another method that appears in two frameworks [5,15] that has a potential of mobilizing non-explicit types of knowledge are *transect walks* – a participatory method that uses controlled “walks” in the community to collect spatial information. In another framework [6] participatory mapping replaces expert GIS – it can also be facilitated in a way that allows drawing on implicit knowledge.

Stakeholder perceptions (including those of researchers) of climate and climate change are also not often explicitly addressed in the process
– only four frameworks allow space for the participants to express their perceptions [5,8,12,13].

When it comes to the members of research teams, they seem to be selected solely on the basis of their scientific expertise or field. However, in as many as seven cases [2,4,5,6,9,10,14], it is not specified what the disciplines or fields of expertise are. Only few frameworks explicitly mention the composition of a research team [13]: “an anthropologist, an economist, an agronomist, and policy and environmental scientists, in this last case with modeling skills”, (Andrieu et al., 2019) or provide details on participating experts [3: expertise in banana production, certification, food safety and compliance, sustainable agriculture, and import and export, (Fernandez et al., 2022)].

Other characteristics of researchers are almost not referred to in the reviewed papers. This is in line with previous findings on CIS co-construction (Guido et al., 2020) - the focus is on the users' characteristics whereas those of actors driving the process are not reflected upon. The kind of “reflexivity” that would be required here, is conceptualized in the responsible innovation (RI) literature (Stilgoe et al., 2013; Eastwood et al., 2019; Jakku et al., 2021) where it is presented among the key institutional capacities along anticipation, inclusion, and responsiveness. These capacities need to be present if the institutional environment of research (understood broadly, including funders and regulators) is to be enabling for RI. IT can be said that reflexivity, described by Stilgoe (Stilgoe et al., 2013: 1571) as “holding a mirror up to one's own actions, commitments and assumptions”, and, further, “being aware of the limits of knowledge and being mindful that a particular framing of an issue may not be universally held” would also constitute an enabling factor for co-production. Yet it is absent from reviewed frameworks, with maybe two exceptions, albeit in both of them it is more of a reflection than reflexivity. Notenbaert et al. (2017) [6], drawing on the work of Neef and Neubert (2011) reflect upon the experience, attitudes, and commitment of researchers participating in the framework application. In this case, it is a part of assessing the benefits of using a participatory stakeholder driven approach, which is one of the research topics of the project in which the framework was developed. In the other case, described by Schwilch et al. (2012) [15], we have an ex-post reflection on the attitude of researchers who act as process moderators – it is found to have a negative impact when they are not neutral enough or try to dominate the process. Decision-support literature addresses this issue. In the work of Walling and Vaneeckhaute (2020) having someone who can bridge gaps between participants of different backgrounds is listed as a recommendation for successful decision-support. Also Moser (2009) saw building trust and bridging divides as a desired quality of actors constructing decision-support. As researchers are not necessarily inclined to facilitate multi-actor processes, two possible solutions come to mind – making the assessment of researcher's attitude towards participation of non-academic actors and different systems of knowledge and their experience in trans-disciplinary research a part of the researchers selection procedure or, as suggested for one of the reviewed frameworks [15], leave facilitation with professional moderators and treating the research team as one of the stakeholders (Schwilch et al., 2012). This idea, to make researchers participate in the process of constructing decision-support on equal (or at least similar) terms with the other participants is not present in the reviewed frameworks, except for the one we mentioned. Researchers either clearly lead the process – assign the roles to others for example, or, much more rarely, take up a role of external observers, providing guidance only when necessary [1,8]. However, following the thought of some authors writing about trans-disciplinarity (Klenk and Meehan, 2015; Turnhout et al., 2020), to achieve “genuine” co-production, it is necessary that stakeholders are equal in the process.

While the participants in most of the frameworks are invited to share their local knowledge (we'll explore their other tasks when talking about the interactive principle), the frameworks fall short on integrating traditional or indigenous knowledge. Besides the few examples presented before, in the case of the CIS co-construction framework from

Senegal (Chiputwa et al., 2020), traditional forecast is used instrumentally as a point of reference to introduce scientific forecast [7]. This is in line with the findings of Roncoli et al. (2002), who see the comparison of the two knowledge systems with local participants as a way of introducing scientific forecast in appropriate and locally relevant ways (see also Guido et al., 2020). However, in one of our cases, the two forecast systems are put in competition [10]. The move from the traditional to the science-based forecast by participating farmers is a desired outcome of the process (Vedeld et al., 2020). This is in direct opposition to the pluralistic principle of co-production. As a counter-example, in the case of the framework to construct a hydro-climatic EVO platform (Nyadzi et al., 2018) both indigenous and scientific knowledge systems are adopted, their inherent value is explicitly recognized. This is in-line with pluralistic principle.

The presence of competing or conflicting views in the decision-support framework does not have to be related with different knowledge systems. From the pluralistic principle perspective, a framework should provide space for conflicting views of any kind to be adequately expressed. As Elinor Ostrom demonstrated in her early work on co-production of public services (Ostrom et al., 1979) and Miller and Wyborn (2020) usefully pointed out forty years later, the process of co-production as such does not exclude negotiations or even resistance. In the context of climate change adaptation, some authors warn against excluding frictions and antagonisms from the process, as it may lead to missing some innovative or experimental ways of adapting (Klenk and Meehan, 2015 after Castree et al., 2014). Among reviewed frameworks, one explicitly selects participants with “different political views” [1], and two openly recognize differences in views [2,15] while making efforts at minimizing dissent [2] or attempts at consensus through negotiations [15]. This is a small number. It can be partly explained by the fact that working away of the opposing views is inherent in many methods used in the frameworks, such as aggregation or majority vote [18]. Also, facilitation methods often aiming at reaching consensus, may lead to less popular views or opinions (or those held by less powerful participants) to be lost from final results. Accounting for power asymmetries is one of the indicators of pluralistic principle. Two of our frameworks openly recognize differences in power between the participants [5,6] while one other framework [17] unintentionally applies a strategy to deal with power asymmetries similar to the one suggested by Munaretto et al. (2014) - it devotes attention to more powerful (the local management body) and less powerful (other local stakeholders) groups separately [17: Arias-Hidalgo et al., 2013]. As these groups worked in separate workshops producing separate results, the fact that their priorities were not compatible was registered and addressed by the research team. In the case of this framework there was a trade-off between pluralistic and interactive principles – the two groups did not have a space to interact. Munaretto's suggestion (Munaretto et al., 2014) to alternate individual and subgroup discussion with plenary debates may permit to avoid such a trade-off.

### 4.3. How the frameworks are goal-oriented

To assess to what extent the frameworks were goal-oriented we looked for: common definition of goals, collectively developed monitoring and evaluation mechanisms, jointly implemented monitoring and evaluation activities. Only one framework [13] includes collectively developed Theory of Change (ToC). It consists of formalizing a collective vision, objectives, project governance and operating rules (Andrieu et al., 2019). One other framework used a ToC approach to develop impact pathways [3] inside the decision process (for different options evaluated within the framework) and not for the process as such. In one framework [15] the objectives are co-defined and agreed upon without using the ToC approach, but there is no information about the monitoring and evaluation of the set objectives. One process [6] recognizes that the exact objectives and goals of the CSA interventions will depend on the local context but doesn't account for how these goals will be
– only four frameworks allow space for the participants to express their perceptions [5,8,12,13].

When it comes to the members of research teams, they seem to be selected solely on the basis of their scientific expertise or field. However, in as many as seven cases [2,4,5,6,9,10,14], it is not specified what the disciplines or fields of expertise are. Only few frameworks explicitly mention the composition of a research team [13]: “an anthropologist, an economist, an agronomist, and policy and environmental scientists, in this last case with modeling skills”, (Andrieu et al., 2019) or provide details on participating experts [3: expertise in banana production, certification, food safety and compliance, sustainable agriculture, and import and export, (Fernandez et al., 2022)].

Other characteristics of researchers are almost not referred to in the reviewed papers. This is in line with previous findings on CIS co-construction (Guido et al., 2020) - the focus is on the users' characteristics whereas those of actors driving the process are not reflected upon. The kind of “reflexivity” that would be required here, is conceptualized in the responsible innovation (RI) literature (Stilgoe et al., 2013; Eastwood et al., 2019; Jakku et al., 2021) where it is presented among the key institutional capacities along anticipation, inclusion, and responsiveness. These capacities need to be present if the institutional environment of research (understood broadly, including funders and regulators) is to be enabling for RI. IT can be said that reflexivity, described by Stilgoe (Stilgoe et al., 2013: 1571) as “holding a mirror up to one's own actions, commitments and assumptions”, and, further, “being aware of the limits of knowledge and being mindful that a particular framing of an issue may not be universally held” would also constitute an enabling factor for co-production. Yet it is absent from reviewed frameworks, with maybe two exceptions, albeit in both of them it is more of a reflection than reflexivity. Notenbaert et al. (2017) [6], drawing on the work of Neef and Neubert (2011) reflect upon the experience, attitudes, and commitment of researchers participating in the framework application. In this case, it is a part of assessing the benefits of using a participatory stakeholder driven approach, which is one of the research topics of the project in which the framework was developed. In the other case, described by Schwilch et al. (2012) [15], we have an ex-post reflection on the attitude of researchers who act as process moderators – it is found to have a negative impact when they are not neutral enough or try to dominate the process. Decision-support literature addresses this issue. In the work of Walling and Vaneeckhaute (2020) having someone who can bridge gaps between participants of different backgrounds is listed as a recommendation for successful decision-support. Also Moser (2009) saw building trust and bridging divides as a desired quality of actors constructing decision-support. As researchers are not necessarily inclined to facilitate multi-actor processes, two possible solutions come to mind – making the assessment of researcher's attitude towards participation of non-academic actors and different systems of knowledge and their experience in trans-disciplinary research a part of the researchers selection procedure or, as suggested for one of the reviewed frameworks [15], leave facilitation with professional moderators and treating the research team as one of the stakeholders (Schwilch et al., 2012). This idea, to make researchers participate in the process of constructing decision-support on equal (or at least similar) terms with the other participants is not present in the reviewed frameworks, except for the one we mentioned. Researchers either clearly lead the process – assign the roles to others for example, or, much more rarely, take up a role of external observers, providing guidance only when necessary [1,8]. However, following the thought of some authors writing about trans-disciplinarity (Klenk and Meehan, 2015; Turnhout et al., 2020), to achieve “genuine” co-production, it is necessary that stakeholders are equal in the process.

While the participants in most of the frameworks are invited to share their local knowledge (we'll explore their other tasks when talking about the interactive principle), the frameworks fall short on integrating traditional or indigenous knowledge. Besides the few examples presented before, in the case of the CIS co-construction framework from

Senegal (Chiputwa et al., 2020), traditional forecast is used instrumentally as a point of reference to introduce scientific forecast [7]. This is in line with the findings of Roncoli et al. (2002), who see the comparison of the two knowledge systems with local participants as a way of introducing scientific forecast in appropriate and locally relevant ways (see also Guido et al., 2020). However, in one of our cases, the two forecast systems are put in competition [10]. The move from the traditional to the science-based forecast by participating farmers is a desired outcome of the process (Vedeld et al., 2020). This is in direct opposition to the pluralistic principle of co-production. As a counter-example, in the case of the framework to construct a hydro-climatic EVO platform (Nyadzi et al., 2018) both indigenous and scientific knowledge systems are adopted, their inherent value is explicitly recognized. This is in-line with pluralistic principle.

The presence of competing or conflicting views in the decision-support framework does not have to be related with different knowledge systems. From the pluralistic principle perspective, a framework should provide space for conflicting views of any kind to be adequately expressed. As Elinor Ostrom demonstrated in her early work on co-production of public services (Ostrom et al., 1979) and Miller and Wyborn (2020) usefully pointed out forty years later, the process of co-production as such does not exclude negotiations or even resistance. In the context of climate change adaptation, some authors warn against excluding frictions and antagonisms from the process, as it may lead to missing some innovative or experimental ways of adapting (Klenk and Meehan, 2015 after Castree et al., 2014). Among reviewed frameworks, one explicitly selects participants with “different political views” [1], and two openly recognize differences in views [2,15] while making efforts at minimizing dissent [2] or attempts at consensus through negotiations [15]. This is a small number. It can be partly explained by the fact that working away of the opposing views is inherent in many methods used in the frameworks, such as aggregation or majority vote [18]. Also, facilitation methods often aiming at reaching consensus, may lead to less popular views or opinions (or those held by less powerful participants) to be lost from final results. Accounting for power asymmetries is one of the indicators of pluralistic principle. Two of our frameworks openly recognize differences in power between the participants [5,6] while one other framework [17] unintentionally applies a strategy to deal with power asymmetries similar to the one suggested by Munaretto et al. (2014) - it devotes attention to more powerful (the local management body) and less powerful (other local stakeholders) groups separately [17: Arias-Hidalgo et al., 2013]. As these groups worked in separate workshops producing separate results, the fact that their priorities were not compatible was registered and addressed by the research team. In the case of this framework there was a trade-off between pluralistic and interactive principles – the two groups did not have a space to interact. Munaretto's suggestion (Munaretto et al., 2014) to alternate individual and subgroup discussion with plenary debates may permit to avoid such a trade-off.

### 4.3. How the frameworks are goal-oriented

To assess to what extent the frameworks were goal-oriented we looked for: common definition of goals, collectively developed monitoring and evaluation mechanisms, jointly implemented monitoring and evaluation activities. Only one framework [13] includes collectively developed Theory of Change (ToC). It consists of formalizing a collective vision, objectives, project governance and operating rules (Andrieu et al., 2019). One other framework used a ToC approach to develop impact pathways [3] inside the decision process (for different options evaluated within the framework) and not for the process as such. In one framework [15] the objectives are co-defined and agreed upon without using the ToC approach, but there is no information about the monitoring and evaluation of the set objectives. One process [6] recognizes that the exact objectives and goals of the CSA interventions will depend on the local context but doesn't account for how these goals will be
expressed in the process. One CSA prioritisation framework [1] openly states what is an underlying assumption of all of them – that CSA practices are assessed against various end-users' goals. However, there is no account of these goals being made explicit during the process. Four frameworks include some kind of monitoring and evaluation mechanism [8,9,13,18], while further three [7,10,12] end the process with final evaluation or assessment. In two cases evaluation is conducted in a participatory workshop setting [7,8]. These results suggest that co-defining the process goals, outcomes and outputs with participants as well as collectively establishing criteria for their assessment, doesn't seem to be a systematic practice, or at least is not systematically reported, in the decision-support.

A relatively rare use of ToC or similar approaches in decision-support may be surprising, given that its implementation in DSS is recommended as a way of bridging the usability gap by assuring that decision-support can contribute to the participants' larger and long-term goals (Allen et al., 2017). In this way – by explicitly connecting the process with local goals – the goal-oriented principle could strengthen the context-based principle. Defining goals collectively and discussing with stakeholders what “desirable outcomes” are would be a good moment to mobilize reflexivity and thus to integrate stakeholders' values and their framings of issues in the process – the elements which are largely overlooked in the analyzed frameworks, as none of them except one [16], provides space or method to make these elements explicit. According to some authors, co-producing knowledge cannot be dissociated from values (Vecchione and Chalabi, 2021). As O'Brien and Wolf (2010) point out, “what is considered legitimate and successful adaptation depends on what people perceive to be worth preserving and achieving”. Even the negative effects of climate change that are supposed to be avoided through better decision-making may be differently valued by different actors or communities.

### 4.4. How the frameworks are made interactive

From the point of view of interactive principle we assessed: the quantity and the quality of interaction in the frameworks, whether mutual learning took place, how were different actors affected by the interaction/how ready were they to adapt and who determined the rules guiding it. The first thing we looked was the quantity of interactions – how often and when in the process occur the moments of interaction with non-academic stakeholders. We found several models in the studied frameworks. In the most dominating, different stakeholders are present at all stages. This concerns almost all CIS co-production frameworks and seven frameworks all together [2,5,7,8, 9,10,13]. Three others [1,12,14] involve stakeholders throughout the process but the moments of interactions are interposed with stages where researchers operate on their own (analysis or modeling) - the results of previous stakeholder phases fuel the analysis or models and in the following phase stakeholders get a chance to react to the analysis or modeling results. In one more framework [4] stakeholders are engaged in all but the very first stage of preliminary analysis. Thus, interactions are frequent in the majority of frameworks.

Another observed model is to involve stakeholders at the first and the last stage of the process [15,17,18] - stakeholders set the starting conditions and provide input for the expert analysis or modeling and then further work on the results. Whatever the exact model is, all these fifteen frameworks involve stakeholders from the early stages of the process. Early involvement of stakeholders concerned by the decision (and not only the end users) is recognized as one of the most important success factors of decision-support (Giupponi, 2014; Gordon et al., 2014). Still, in two of the studied frameworks, stakeholders are involved late in the process [6,16]. In these cases, expert teams focus on producing knowledge before reaching out to non-academic actors for their feedback on the results.

Since the early reflections on improving DSS it was pointed out that not only the quantity but also the quality of interaction between the

producers and the users counted. The co-production perspective supposes that users are co-producers. That calls for a more egalitarian interaction, not simply extractive, where participants provide information and researchers analyse them. Overview of the roles – or tasks – assigned to non-academic stakeholders in different frameworks is presented in Table 5.

One of the frameworks, the CIS co-production model from Senegal, which involves frequent and regular meetings in multi-stakeholders groups throughout the farming season (and has been in place since several years), provides opportunity for continuous interaction between stakeholders, during which the core stakeholders are involved directly in the production of the CIS bulletin, each of them responsible to bring in specific knowledge, that is then collectively analyzed and transformed into CIS – it is the closest to the co-production of decision-support that we have found.

Continuous interaction and knowledge exchange between different actors provides space for one more important element of co-production – mutual learning. It is surprising that mutual learning between participants is mentioned in only three frameworks [9,10,15], of which one openly states that researchers learned from farmers [9] and one, developed as part of DESIRE project, presents mutual learning among the most important objectives and evaluates it as part of the process (Schwilch et al., 2009). Mutual learning is not systematically assessed in any other of the studied frameworks. One more framework mentions mutual questioning that leads to reflexivity [12].

As decision-support users are expected to adapt their decision-making (and thus practice) following their interaction with researchers, an egalitarian interaction would suggest scientists need to be ready to adapt their own practice, following their interaction with other participants of the process. This may go as far as making adaptations to their method (Landström et al., 2011) or allowing DSS co-production stakeholders to challenge the development of the model (Walling and Vanecekhaute, 2020). It is similar to the open innovation approach which advocates opening design and innovation techniques and tools to better account for reflexivity (Berthet et al., 2018). We looked at whether and how the participants are given a chance to change or influence a method used to co-produce decision-support. We identified several ways in which it is done. Participants may be asked to co-produce different stages of the method by: ranking priorities collectively to allow for additional analysis in order to identify best-bet CSA practices [1], participating in the selection of vulnerability indicators [2], assigning priority weights to the four indicators of CSA at the start of the process [4], co-defining the objectives of the platform and how it would operate [13], identifying driving forces for scenario narratives [18], agreeing on evaluation criteria [15]. In the PICSA approach the participating farmers literally produce the tools (resource allocation map, seasonal calendar, various matrixes) throughout the process. All these examples concern different elements of the process but not the process itself – stakeholders do not take part in determining what different stages are or what methods are used. While more than a half of the analyzed frameworks integrate models [1,2,3,6,13–18], the modeling work is almost exclusively expert-run, except for a DSS used in South Africa [16], designed collaboratively by the software developers, the interdisciplinary team of researchers, and the local stakeholders (Berger et al., 2019). No other example is found of researchers adapting their practice, or changing their method following interaction with other actors in the process, in the sense described by Landström et al. (2011). In RI literature, the capacity to change direction in response to what interaction with stakeholders brings in terms of feedback or emerging perspectives, is called “responsiveness” (Stilgoe et al., 2013). Some ways of creating an environment that would enable responsiveness are proposed, for example introducing stage-gate reviews (Macnaghten and Owen, 2011) or mid-project reviews (Eastwood et al., 2019), i.e. decision points where directions and deliverables could be adapted. Such points could be collectively agreed upon in the monitoring and evaluation system – the goal-oriented principle could potentially strengthen
expressed in the process. One CSA prioritisation framework [1] openly states what is an underlying assumption of all of them – that CSA practices are assessed against various end-users' goals. However, there is no account of these goals being made explicit during the process. Four frameworks include some kind of monitoring and evaluation mechanism [8,9,13,18], while further three [7,10,12] end the process with final evaluation or assessment. In two cases evaluation is conducted in a participatory workshop setting [7,8]. These results suggest that co-defining the process goals, outcomes and outputs with participants as well as collectively establishing criteria for their assessment, doesn't seem to be a systematic practice, or at least is not systematically reported, in the decision-support.

A relatively rare use of ToC or similar approaches in decision-support may be surprising, given that its implementation in DSS is recommended as a way of bridging the usability gap by assuring that decision-support can contribute to the participants' larger and long-term goals (Allen et al., 2017). In this way – by explicitly connecting the process with local goals – the goal-oriented principle could strengthen the context-based principle. Defining goals collectively and discussing with stakeholders what “desirable outcomes” are would be a good moment to mobilize reflexivity and thus to integrate stakeholders' values and their framings of issues in the process – the elements which are largely overlooked in the analyzed frameworks, as none of them except one [16], provides space or method to make these elements explicit. According to some authors, co-producing knowledge cannot be dissociated from values (Vecchione and Chalabi, 2021). As O'Brien and Wolf (2010) point out, “what is considered legitimate and successful adaptation depends on what people perceive to be worth preserving and achieving”. Even the negative effects of climate change that are supposed to be avoided through better decision-making may be differently valued by different actors or communities.

### 4.4. How the frameworks are made interactive

From the point of view of interactive principle we assessed: the quantity and the quality of interaction in the frameworks, whether mutual learning took place, how were different actors affected by the interaction/how ready were they to adapt and who determined the rules guiding it. The first thing we looked was the quantity of interactions – how often and when in the process occur the moments of interaction with non-academic stakeholders. We found several models in the studied frameworks. In the most dominating, different stakeholders are present at all stages. This concerns almost all CIS co-production frameworks and seven frameworks all together [2,5,7,8, 9,10,13]. Three others [1,12,14] involve stakeholders throughout the process but the moments of interactions are interposed with stages where researchers operate on their own (analysis or modeling) - the results of previous stakeholder phases fuel the analysis or models and in the following phase stakeholders get a chance to react to the analysis or modeling results. In one more framework [4] stakeholders are engaged in all but the very first stage of preliminary analysis. Thus, interactions are frequent in the majority of frameworks.

Another observed model is to involve stakeholders at the first and the last stage of the process [15,17,18] - stakeholders set the starting conditions and provide input for the expert analysis or modeling and then further work on the results. Whatever the exact model is, all these fifteen frameworks involve stakeholders from the early stages of the process. Early involvement of stakeholders concerned by the decision (and not only the end users) is recognized as one of the most important success factors of decision-support (Giupponi, 2014; Gordon et al., 2014). Still, in two of the studied frameworks, stakeholders are involved late in the process [6,16]. In these cases, expert teams focus on producing knowledge before reaching out to non-academic actors for their feedback on the results.

Since the early reflections on improving DSS it was pointed out that not only the quantity but also the quality of interaction between the

producers and the users counted. The co-production perspective supposes that users are co-producers. That calls for a more egalitarian interaction, not simply extractive, where participants provide information and researchers analyse them. Overview of the roles – or tasks – assigned to non-academic stakeholders in different frameworks is presented in Table 5.

One of the frameworks, the CIS co-production model from Senegal, which involves frequent and regular meetings in multi-stakeholders groups throughout the farming season (and has been in place since several years), provides opportunity for continuous interaction between stakeholders, during which the core stakeholders are involved directly in the production of the CIS bulletin, each of them responsible to bring in specific knowledge, that is then collectively analyzed and transformed into CIS – it is the closest to the co-production of decision-support that we have found.

Continuous interaction and knowledge exchange between different actors provides space for one more important element of co-production – mutual learning. It is surprising that mutual learning between participants is mentioned in only three frameworks [9,10,15], of which one openly states that researchers learned from farmers [9] and one, developed as part of DESIRE project, presents mutual learning among the most important objectives and evaluates it as part of the process (Schwilch et al., 2009). Mutual learning is not systematically assessed in any other of the studied frameworks. One more framework mentions mutual questioning that leads to reflexivity [12].

As decision-support users are expected to adapt their decision-making (and thus practice) following their interaction with researchers, an egalitarian interaction would suggest scientists need to be ready to adapt their own practice, following their interaction with other participants of the process. This may go as far as making adaptations to their method (Landström et al., 2011) or allowing DSS co-production stakeholders to challenge the development of the model (Walling and Vanecekhaute, 2020). It is similar to the open innovation approach which advocates opening design and innovation techniques and tools to better account for reflexivity (Berthet et al., 2018). We looked at whether and how the participants are given a chance to change or influence a method used to co-produce decision-support. We identified several ways in which it is done. Participants may be asked to co-produce different stages of the method by: ranking priorities collectively to allow for additional analysis in order to identify best-bet CSA practices [1], participating in the selection of vulnerability indicators [2], assigning priority weights to the four indicators of CSA at the start of the process [4], co-defining the objectives of the platform and how it would operate [13], identifying driving forces for scenario narratives [18], agreeing on evaluation criteria [15]. In the PICSA approach the participating farmers literally produce the tools (resource allocation map, seasonal calendar, various matrixes) throughout the process. All these examples concern different elements of the process but not the process itself – stakeholders do not take part in determining what different stages are or what methods are used. While more than a half of the analyzed frameworks integrate models [1,2,3,6,13–18], the modeling work is almost exclusively expert-run, except for a DSS used in South Africa [16], designed collaboratively by the software developers, the interdisciplinary team of researchers, and the local stakeholders (Berger et al., 2019). No other example is found of researchers adapting their practice, or changing their method following interaction with other actors in the process, in the sense described by Landström et al. (2011). In RI literature, the capacity to change direction in response to what interaction with stakeholders brings in terms of feedback or emerging perspectives, is called “responsiveness” (Stilgoe et al., 2013). Some ways of creating an environment that would enable responsiveness are proposed, for example introducing stage-gate reviews (Macnaghten and Owen, 2011) or mid-project reviews (Eastwood et al., 2019), i.e. decision points where directions and deliverables could be adapted. Such points could be collectively agreed upon in the monitoring and evaluation system – the goal-oriented principle could potentially strengthen
<table><caption>**Table 5**
Overview of the roles of non-academic stakeholders in the studied frameworks.</caption><thead><tr><th></th><th>Decision support framework</th><th>Non-academic stakeholders involved</th><th>Criteria for including non-academic stakeholders</th><th>When in the process interaction takes place</th><th>Role of non-academic stakeholders</th><th>Who decides this role</th></tr></thead><tbody><tr><td>1.</td><td>CSA-PF<br>Andrieu et al. 2017</td><td>1. NGO 2. A development organization<br>3. two donors, the national directorate of agriculture, two national NGOs<br>4. relevant actors at national, sub-national, and community levels (farmers and government stakeholders are mentioned)</td><td>1. its previous role as facilitator of a platform, expertise on climate change and agriculture in Mali 2. rural development expertise<br>3. potential end-users 4. diversity of political, scientific, and technical views, knowledge and experience on climate change and agriculture or experience implementing projects or programs in relation to the topic</td><td>Throughout, interposed with expert analysis</td><td>1. to facilitate the process 2-3. to steer the process 4. to discuss, debate, and rank priorities to allow for additional analysis; to review the results from expert analysis; to fill in the gaps in the data; to evaluate; to analyse</td><td>1. Project team 2. Facilitator with the support of the project team 3. Steering committee 4. Facilitator with the support of the project team</td></tr><tr><td>2.</td><td>Target-CSA<br>Brandt et al. 2017</td><td>multi-sectoral stakeholder groups (government, civil society, private sector are mentioned)<br>professionals, researchers, and practitioners working in banana production, certification, food safety and compliance, sustainable agriculture, and import and export</td><td>expert knowledge</td><td>Throughout</td><td>to select indicators, to analyse, to express preferences</td><td>Project team</td></tr><tr><td>3.</td><td>Decision analysis approach<br>Fernandez et al., 2022</td><td>1. officers from the state and district agriculture departments (25%), 2. extension offices, 3. development organizations (NGOs and donor agencies) (25%), 4. private sector (10%), 5. farmers (40%), 6. local resource persons<br>1. farmers, local leaders, 2. local-level agricultural experts (agriculture and livestock officials; district-level, ward-level, and village-level authorities; rural organizations; and farmer organizations), district-level extension agents, private sector, donor organizations, policy implementers</td><td>membership in the banana expert network</td><td>Replaced due to covid-19 restriction</td><td>to provide information/share knowledge; to clarify relationships between interventions and intended outcomes; to improve the project team's understanding of the system</td><td>Project team</td></tr><tr><td>4.</td><td>A framework to prioritize locally suitable CSA interventions<br>Khatri-Chhetri et al. 2019</td><td></td><td>1, 2, 3, 4: areas of work, knowledge on climate change adaptation in agriculture, working experience with farming communities in the state. 5: randomly selected from the five project districts</td><td>Throughout (but the first stage)</td><td>to assign priority weights to indicators; to evaluate based on criteria prepared by the project team; to rank</td><td>Project team</td></tr><tr><td>5.</td><td>CSA-RA<br>Mwongera et al. 2017</td><td></td><td>1. to be representative of the local population (i.e. with respect to gender, age, income, agriculture enterprise, ethnic groups and agro-ecological zones). 2. to be knowledgeable about agriculture and/or climate in the region</td><td>Throughout</td><td>to provide information</td><td>Project team</td></tr><tr><td>6.</td><td>A generic framework for targeting, out-scaling and prioritizing CSA interventions in agricultural systems<br>Notenbaert et al., 2017</td><td>(In Tanzania) farmers, traders, food processors, government officials</td><td>participation in local innovation platform</td><td>Late</td><td>to analyse, to plan to formulate recommendations based on own estimation of impacts of recommended solutions, to discuss the characterisation of intervention scenarios, to map</td><td>Project team</td></tr><tr><td>7.</td><td>MWG model Ouedraogo et al. 2018, Chiputwa et al., 2020</td><td>leader farmers, pastoralists, fishermen, technicians from relevant ministries, experts from the meteorological office and from relevant non-governmental organizations, technical staff from community radios, televisions and phone companies, services in charge of water resources, agriculture, forestry, the media, the producers' organizations, women-based organizations</td><td>for farmers and media: ability to support transfer of CIS; women farmers deliberately included</td><td>Throughout</td><td>to co-produce, translate and transfer CIS</td><td>Project team</td></tr><tr><td>8.</td><td>PICSA<br>Dayamba et al. 2018, Clarkson et al. 2019, Staub and Clarkson 2021</td><td>1. smallholder farmers 2. extension workers, NGO field staff, community volunteers acting as facilitators.</td><td>conscious effort to include disadvantaged participants, allows participation of non- and semi-literate farmers, gender</td><td>Throughout</td><td>1. to provide information, to analyse (their own resource allocation, historical climate data), to share ideas about the causes of identified problems, to co-construct different tools, to construct budget, to learn, to</td><td>Project team. Farmers may volunteer to become the method facilitators</td></tr></tbody></table>

(continued on next page)
<table><caption>Table 5 (continued)</caption><thead><tr><th></th><th>Decision support framework</th><th>Non-academic stakeholders involved</th><th>Criteria for including non-academic stakeholders</th><th>When in the process interaction takes place</th><th>Role of non-academic stakeholders</th><th>Who decides this role</th></tr></thead><tbody><tr><td>9.</td><td>WCIS co-production process<br>Gbangou et al., 2020</td><td>1. farmers, 2. agricultural extension agents, 3. meteorological extension agent</td><td>1. experience with local forecasts, availability, gender, willingness to participate 2. outreach potential 3. experience with rain gauges</td><td>Throughout</td><td>decide, to suggest improvements to the process 2. to facilitate and monitor the process<br>1. to provide information; to express needs; to chose and refine images for local forecast indicators; to learn to use software; install rain gauges and read and record rainfall data; to collect and share local forecast indicators and rainfall observation data; to evaluate the process<br>2. to see, share, and interact with the forecasts and data collected by the farmers and scientists; to give their opinion in evaluating the coproduction experiment 3. to train farmers to express their needs, to express their preferences, to learn about forecast, to provide feedback, to discuss and interpret data, to co-develop advisory service, to evaluate the process</td><td>Project team</td></tr><tr><td>10.</td><td>Farmer field school approach to deliver CIS Kumar et al. 2021</td><td>local meteorological officers, local agricultural extension officers, smallholder farmers</td><td>Gender balance</td><td>Throughout</td><td>to provide feedback<br>*recommended in the future: to share practice-based knowledge</td><td>Project team. Farmers may decide to give feedback through provided channels.</td></tr><tr><td>11.</td><td>IMD-AAS (AGRIMET platform) as applied in Maharashtra<br>Vedeld et al., 2020</td><td>farmers (limited)</td><td>Not precised. In practice, having a mobile phone and being aware of the service are necessary to provide feedback.</td><td>Limited, feedback only</td><td>1. to share their perception of the climate-water-food production problems; to provide information on current actions of farmers and organizations to manage these problems, farmers' hydro-climatic informational needs and use, the value of seasonal climate forecast, to attempt ex-ante avaluation 2. to provide information 3. to share traditional knowledge; to upload data on the platform</td><td>Project team</td></tr><tr><td>12.</td><td>Hydro-climatic EVO<br>Nyadzi et al., 2018</td><td>1.Applied Meteorological Unit - Meteorological Agency, Irrigation Development Authority, Crop Division of Ministry of Food & Agriculture, Irrigation Water Manager, Hydrological Services, Rice Farmers Association, Agriculture and Development Non-Governmental Organization 2. Rice Farmers Community Leaders, Water Managers, Leadership of the irrigation scheme, women in the diagnostic phase 3. Farmers in the application phase</td><td>1. their principal role (civil society representatives, policy and decision makers, farmer representatives) and expertise in climate, water and farming 2. involvement in rice farming, gender</td><td>Throughout, interposed with expert analysis</td><td>1. to provide information, to identify stakeholders who are source of information 2. to co-define the objectives of the platform and how it would operate, to conduct analysis, perform diagnosis, co-define monitoring system of expected changes, to monitor the process, define technical and organizational options to test, to test chosen options to share local agroforestry knowledge, to share their perception of benefits of shade trees, to rank tree species (that they have the biggest knowledge of) for ecosystem services<br>1. to learn from each other, to provide information, to prioritize local solutions for further assessment 1&2: to confirm or reformulate the main objectives, to agree on</td><td>Participants chose they own role in the project from the roles defined by researchers.</td></tr><tr><td>13.</td><td>A framework to co-design climate-smart farming systems with local stakeholders<br>Andrieu et al., 2019</td><td>1. farmers, 2. farmers/local actors</td><td>2. Recognition by farmers as support for the adoption of CSA practices</td><td>Throughout</td><td></td><td></td></tr><tr><td>14.</td><td>A tool for tree selection in smallholder agroforestry farms<br>Van der Wolf et al. 2019</td><td>farmers</td><td>Involvement in coffee agroforestry</td><td>Throughout, interposed with expert analysis</td><td></td><td>Project team</td></tr><tr><td>15.</td><td>DESIRE-DSS<br>Schwilch et al., 2012</td><td>1. local land users, representatives of local authorities and community-based organizations living and working in the specific local</td><td>1. Diversity: large-scale, small-scale, land owners, tenants, pastoralists, etc.</td><td>First&amp;last</td><td></td><td>Project team</td></tr></tbody></table>

(continued on next page)
<table><caption>Table 5 (continued)</caption><thead><tr><th></th><th>Decision support framework</th><th>Non-academic stakeholders involved</th><th>Criteria for including non-academic stakeholders</th><th>When in the process interaction takes place</th><th>Role of non-academic stakeholders</th><th>Who decides this role</th></tr></thead><tbody><tr><td>9.</td><td>WCIS co-production process<br>Gbangou et al., 2020</td><td>1. farmers, 2. agricultural extension agents, 3. meteorological extension agent</td><td>1. experience with local forecasts, availability, gender, willingness to participate 2. outreach potential 3. experience with rain gauges</td><td>Throughout</td><td>decide, to suggest improvements to the process 2. to facilitate and monitor the process<br>1. to provide information; to express needs; to chose and refine images for local forecast indicators; to learn to use software; install rain gauges and read and record rainfall data; to collect and share local forecast indicators and rainfall observation data; to evaluate the process<br>2. to see, share, and interact with the forecasts and data collected by the farmers and scientists; to give their opinion in evaluating the coproduction experiment 3. to train farmers to express their needs, to express their preferences, to learn about forecast, to provide feedback, to discuss and interpret data, to co-develop advisory service, to evaluate the process</td><td>Project team</td></tr><tr><td>10.</td><td>Farmer field school approach to deliver CIS Kumar et al. 2021</td><td>local meteorological officers, local agricultural extension officers, smallholder farmers</td><td>Gender balance</td><td>Throughout</td><td>to provide feedback<br>*recommended in the future: to share practice-based knowledge</td><td>Project team. Farmers may decide to give feedback through provided channels.</td></tr><tr><td>11.</td><td>IMD-AAS (AGRIMET platform) as applied in Maharashtra<br>Vedeld et al., 2020</td><td>farmers (limited)</td><td>Not precised. In practice, having a mobile phone and being aware of the service are necessary to provide feedback.</td><td>Limited, feedback only</td><td>1. to share their perception of the climate-water-food production problems; to provide information on current actions of farmers and organizations to manage these problems, farmers' hydro-climatic informational needs and use, the value of seasonal climate forecast, to attempt ex-ante avaluation 2. to provide information 3. to share traditional knowledge; to upload data on the platform</td><td>Project team</td></tr><tr><td>12.</td><td>Hydro-climatic EVO<br>Nyadzi et al., 2018</td><td>1.Applied Meteorological Unit - Meteorological Agency, Irrigation Development Authority, Crop Division of Ministry of Food & Agriculture, Irrigation Water Manager, Hydrological Services, Rice Farmers Association, Agriculture and Development Non-Governmental Organization 2. Rice Farmers Community Leaders, Water Managers, Leadership of the irrigation scheme, women in the diagnostic phase 3. Farmers in the application phase</td><td>1. their principal role (civil society representatives, policy and decision makers, farmer representatives) and expertise in climate, water and farming 2. involvement in rice farming, gender</td><td>Throughout, interposed with expert analysis</td><td>1. to provide information, to identify stakeholders who are source of information 2. to co-define the objectives of the platform and how it would operate, to conduct analysis, perform diagnosis, co-define monitoring system of expected changes, to monitor the process, define technical and organizational options to test, to test chosen options to share local agroforestry knowledge, to share their perception of benefits of shade trees, to rank tree species (that they have the biggest knowledge of) for ecosystem services</td><td>Participants chose they own role in the project from the roles defined by researchers.</td></tr><tr><td>13.</td><td>A framework to co-design climate-smart farming systems with local stakeholders<br>Andrieu et al., 2019</td><td>1. farmers, 2. farmers/local actors</td><td>2. Recognition by farmers as support for the adoption of CSA practices</td><td>Throughout</td><td>1. to learn from each other, to provide information, to prioritize local solutions for further assessment 1&amp;2: to confirm or reformulate the main objectives, to agree on</td><td>Project team</td></tr><tr><td>14.</td><td>A tool for tree selection in smallholder agroforestry farms<br>Van der Wolf et al. 2019</td><td>farmers</td><td>Involvement in coffee agroforestry</td><td>Throughout, interposed with expert analysis</td><td>1. to learn from each other, to provide information, to prioritize local solutions for further assessment 1&amp;2: to confirm or reformulate the main objectives, to agree on</td><td>Project team</td></tr><tr><td>15.</td><td>DESIRE-DSS<br>Schwilch et al., 2012</td><td>1. local land users, representatives of local authorities and community-based organizations living and working in the specific local</td><td>1. Diversity: large-scale, small-scale, land owners, tenants, pastoralists, etc.</td><td>First&amp;last</td><td>1. to provide information, to identify stakeholders who are source of information 2. to co-define the objectives of the platform and how it would operate, to conduct analysis, perform diagnosis, co-define monitoring system of expected changes, to monitor the process, define technical and organizational options to test, to test chosen options to share local agroforestry knowledge, to share their perception of benefits of shade trees, to rank tree species (that they have the biggest knowledge of) for ecosystem services</td><td>Project team</td></tr></tbody></table>

(continued on next page)
<table><caption>**Table 5 (continued)**</caption><thead><tr><th></th><th>Decision support framework</th><th>Non-academic stakeholders involved</th><th>Criteria for including non-academic stakeholders</th><th>When in the process interaction takes place</th><th>Role of non-academic stakeholders</th><th>Who decides this role</th></tr></thead><tbody><tr><td></td><td></td><td>context 2. external experts from NGO and GO</td><td></td><td></td><td>evaluation criteria, to select, compare, score, rank options, to negotiate the best option for implementation and to decide upon 1-2 strategies for implementation</td><td></td></tr><tr><td>16.</td><td>Interdisciplinary, multi-scaled, and integrated socio-ecological approach, including MARS DSS<br>Bergeret et al. 2019</td><td>local land-use decision-makers</td><td>Additional criteria not communicated</td><td>Late</td><td>to participate in the design of the information contained in the land-use planning tool as well as the format of the tool</td><td>Project team</td></tr><tr><td>17.</td><td>WET-WIN<br>Arias-Hidalgo et al., 2013</td><td>1. Technical Secretariat of the local management structure 2. experts from the Ministries of Environment and Agriculture, local municipalities, UNDP<br>1. representatives of government, civil society and the agriculture sector in each country<br>2. individuals with expertise in climate- smart agriculture, nutrition security and agricultural and food systems policy</td><td>Additional criteria not communicated<br>1. To be representative of as broad a range of food system expertise as possible 2. Level and scope of expertise</td><td>First (only for one stakeholder) &amp;last</td><td>1. to devise a set of management options 2. to evaluate qualitative indicators 1&amp;2. to express their preferences<br>1. to identify two driving forces for scenario narratives (to analyse) 2. to share their perspectives and provide input on questions identified by researchers</td><td>Project team</td></tr><tr><td>18.</td><td>iFEED<br>Jennings et al. 2022</td><td></td><td></td><td>First&amp;last</td><td></td><td>Project team</td></tr></tbody></table>

the interactive principle.

Some frameworks make an attempt at turning non-academic stakeholders from data providers into data co-producers by using elements of citizen science. This is the case of two frameworks from the CIS co-production category, that both require farmers to upload their traditional forecast on phone app or on-line platform. In the first case [9] farmers co-construct traditional indicators and select visual material that will be used in the app to represent them. In the second case [12] farmers, through uploading information on the platform, provide traditional seasonal forecast of rainfall onset and cessation date and rainfall amount and degree of temperature as well as twenty-four hours weather forecast of low, medium or high rain. Another framework [14] directly incorporates traditional local knowledge of tree species that is used to build an on-line DSS. Here farmers are acting as experts on the species that they identified as their area of expertise thus participating in the project through a similar mechanism as researchers. These examples suggest that the high-tech solutions such as mobile applications or on-line platforms are well suited to provide space to integrate traditional knowledge at similar level as expert knowledge.

One more identified way of making the process more interactive is to let the participants determine their own role in the process, which was applied in one of the frameworks [13] where participants chose their own level of implication from the roles defined by the research team. To a certain extent, establishing a stakeholders' steering committee which was part of one framework design [1] may also be seen as a strategy to apply both interactive and pluralistic principle – the committee members may suggest to include new participants of whom the research team may not have thought.

### 4.5. Co-production principles across frameworks

The various ways of applying co-production principles identified through our study are summarized in Table 6. We found more examples of how the pluralistic and the interactive principles can be applied in

building decision-support frameworks than of the context-based and the goal-oriented principles. Yet, in the context of decision-support for climate change adaptation, these last two principles seem to be particularly important: making sure that the participants share the conception of the problem and agree on adaptation goals may be prerequisite for the process' success. After a decision-support project is initiated, there are still ways of picking up on the context-based principle by trying to connect to the existing dynamics, as observed in many of the reviewed frameworks. When it comes to the goal-oriented principle, if a Theory of Change has not been developed and a monitoring and evaluation protocol agreed upon, there are not many options left to respond to this principle – hence it could be recommended to develop it systematically.

The overview of the principles traced in different frameworks (presented in Table 7) reveals that the CIS co-production frameworks are the most successful in including examples of application of all the principles – unsurprisingly, given that they openly aim at co-production. It appears that the type of framework (or rather the type of decision it supports) may determine the attention payed to different principles and the details of how they are implemented. For example, for pluralistic principle, CSA prioritisation frameworks, where decision is to be made between solutions representing different visions of development, may give more consideration to the inclusion of actors representing various political and technical views, whereas the implementers of CIS co-production frameworks will be more often faced with the issue of traditional knowledge integration, as traditional ways of predicting the weather may be in competition with the proposed decision-support. When it comes to the level at which a framework is implemented, or more precisely, the type of stakeholders it mobilizes, it may affect the application of the interactive principle – the institutional stakeholders may be less available for continuous interaction.

Unlike the pluralistic principle, which can be planned for at the stage of selecting the participants and/or selecting participatory methodologies, the context-based, the goal-oriented and the interactive principles seem to be facing some structural barriers to their full implementation.
<table><caption>**Table 5 (continued)**</caption><thead><tr><th></th><th>Decision support framework</th><th>Non-academic stakeholders involved</th><th>Criteria for including non-academic stakeholders</th><th>When in the process interaction takes place</th><th>Role of non-academic stakeholders</th><th>Who decides this role</th></tr></thead><tbody><tr><td></td><td></td><td>context 2. external experts from NGO and GO</td><td></td><td></td><td>evaluation criteria, to select, compare, score, rank options, to negotiate the best option for implementation and to decide upon 1-2 strategies for implementation</td><td></td></tr><tr><td>16.</td><td>Interdisciplinary, multi-scaled, and integrated socio-ecological approach, including MARS DSS<br>Bergeret et al. 2019</td><td>local land-use decision-makers</td><td>Additional criteria not communicated</td><td>Late</td><td>to participate in the design of the information contained in the land-use planning tool as well as the format of the tool</td><td>Project team</td></tr><tr><td>17.</td><td>WET-WIN<br>Arias-Hidalgo et al., 2013</td><td>1. Technical Secretariat of the local management structure 2. experts from the Ministries of Environment and Agriculture, local municipalities, UNDP<br>1. representatives of government, civil society and the agriculture sector in each country<br>2. individuals with expertise in climate- smart agriculture, nutrition security and agricultural and food systems policy</td><td>Additional criteria not communicated<br>1. To be representative of as broad a range of food system expertise as possible 2. Level and scope of expertise</td><td>First (only for one stakeholder) &amp;last</td><td>1. to devise a set of management options 2. to evaluate qualitative indicators 1&amp;2. to express their preferences<br>1. to identify two driving forces for scenario narratives (to analyse) 2. to share their perspectives and provide input on questions identified by researchers</td><td>Project team</td></tr><tr><td>18.</td><td>iFEED<br>Jennings et al. 2022</td><td></td><td></td><td>First&amp;last</td><td></td><td>Project team</td></tr></tbody></table>

the interactive principle.

Some frameworks make an attempt at turning non-academic stakeholders from data providers into data co-producers by using elements of citizen science. This is the case of two frameworks from the CIS co-production category, that both require farmers to upload their traditional forecast on phone app or on-line platform. In the first case [9] farmers co-construct traditional indicators and select visual material that will be used in the app to represent them. In the second case [12] farmers, through uploading information on the platform, provide traditional seasonal forecast of rainfall onset and cessation date and rainfall amount and degree of temperature as well as twenty-four hours weather forecast of low, medium or high rain. Another framework [14] directly incorporates traditional local knowledge of tree species that is used to build an on-line DSS. Here farmers are acting as experts on the species that they identified as their area of expertise thus participating in the project through a similar mechanism as researchers. These examples suggest that the high-tech solutions such as mobile applications or on-line platforms are well suited to provide space to integrate traditional knowledge at similar level as expert knowledge.

One more identified way of making the process more interactive is to let the participants determine their own role in the process, which was applied in one of the frameworks [13] where participants chose their own level of implication from the roles defined by the research team. To a certain extent, establishing a stakeholders' steering committee which was part of one framework design [1] may also be seen as a strategy to apply both interactive and pluralistic principle – the committee members may suggest to include new participants of whom the research team may not have thought.

### 4.5. Co-production principles across frameworks

The various ways of applying co-production principles identified through our study are summarized in Table 6. We found more examples of how the pluralistic and the interactive principles can be applied in

building decision-support frameworks than of the context-based and the goal-oriented principles. Yet, in the context of decision-support for climate change adaptation, these last two principles seem to be particularly important: making sure that the participants share the conception of the problem and agree on adaptation goals may be prerequisite for the process' success. After a decision-support project is initiated, there are still ways of picking up on the context-based principle by trying to connect to the existing dynamics, as observed in many of the reviewed frameworks. When it comes to the goal-oriented principle, if a Theory of Change has not been developed and a monitoring and evaluation protocol agreed upon, there are not many options left to respond to this principle – hence it could be recommended to develop it systematically.

The overview of the principles traced in different frameworks (presented in Table 7) reveals that the CIS co-production frameworks are the most successful in including examples of application of all the principles – unsurprisingly, given that they openly aim at co-production. It appears that the type of framework (or rather the type of decision it supports) may determine the attention payed to different principles and the details of how they are implemented. For example, for pluralistic principle, CSA prioritisation frameworks, where decision is to be made between solutions representing different visions of development, may give more consideration to the inclusion of actors representing various political and technical views, whereas the implementers of CIS co-production frameworks will be more often faced with the issue of traditional knowledge integration, as traditional ways of predicting the weather may be in competition with the proposed decision-support. When it comes to the level at which a framework is implemented, or more precisely, the type of stakeholders it mobilizes, it may affect the application of the interactive principle – the institutional stakeholders may be less available for continuous interaction.

Unlike the pluralistic principle, which can be planned for at the stage of selecting the participants and/or selecting participatory methodologies, the context-based, the goal-oriented and the interactive principles seem to be facing some structural barriers to their full implementation.
<table><caption>**Table 6**

Incorporating co-production principles in decision-support frameworks.</caption><thead><tr><th>Co-production principles (Nörstrom et al., 2020)</th><th>Ways to apply in a decision-support framework, as identified in our study</th></tr></thead><tbody><tr><td>Context-specific</td><td>Using already established interaction spaces from previous/on-going projects, for example: innovation platforms<br>Farmer Field Schools.<br>Incorporating actors already active in knowledge production, sharing and use, for example: local leaders of opinion, extension agents who have a good working relationship with farmers, experts recognized by their peers, actors who have large networks.<br>Screening for current practices, innovations or endogenous adaptation strategies and integrating them in the process.</td></tr><tr><td>Pluralistic</td><td>Having a selection strategy, for example selecting participants from a diversity of locally present situations (technical, social, economic), with a broad range of expertise, with a diversity of political, scientific, and technical views; making a conscious effort to include disadvantaged groups.<br>Assuring gender balance and disaggregate results according to gender, if relevant.<br>When building a research team, using criteria such as: experience with transdisciplinary research and participatory methods, posture, readiness to commit, non-dominating attitude, etc.<br>Providing space for conflicting views and negotiation.<br>Creating separate interaction spaces for groups with competing views/in case of strong power asymmetry.<br>Taking into account farmers' specific knowledge when selecting farmers.<br>Not putting different kinds of knowledge in competition, recognising their intrinsic value.</td></tr><tr><td>Goal-oriented</td><td>Formalizing with the participants project objectives and agreeing on when they will be achieved, establishing a monitoring/evaluation strategy, for example using Theory of Change.<br>Establishing together with stakeholders what desirable outcomes of the process are and using this opportunity to discuss their underlying values.</td></tr><tr><td>Interactive</td><td>Involving stakeholders throughout the process.<br>Creating opportunities for mutual learning and monitoring/assessing mutual learning.<br>Inviting participants to contribute to different elements of the method, for example: rank priorities collectively, participate in the selection of indicators, assigning priority weights to indicators, participating in the definition of the problem and the way in which research may be able to contribute to finding a solution, co-defining the objectives of a platform (or a different set-up) and how it would operate, identifying driving forces for scenario narratives, agreeing on evaluation criteria.<br>Inviting participants to produce their own tools of analysis, for example maps, calendars, matrixes.<br>If the project involves citizen science, putting traditional/indigenous knowledge at the same level as scientific knowledge, for example by creating space to upload it on the on-line tools.<br>If the project involves citizen science, involving participants in the design of the method/tool or its elements.<br>Letting the participants determine their own role/ level of implication in the process<br>Establishing a stakeholders' steering committee.</td></tr></tbody></table>

Responding to local demand or adapting the project trajectory (if results of monitoring and evaluation or of interaction with stakeholders demand it) will not be possible if the project design as well as the budget and reporting mechanisms don't allow it, regardless of the readiness or skills of research teams.

<table><caption>**Table 7**

Traces of co-production principles across the frameworks.</caption><thead><tr><th>Decision support framework</th><th>Context-based principle</th><th>Pluralistic principle</th><th>Goal-oriented principle</th><th>Interactive principle</th></tr></thead><tbody><tr><td>CSA-PF</td><td></td><td></td><td></td><td></td></tr><tr><td>1. Andrieu et al. 2017</td><td></td><td>X</td><td>X</td><td>X</td></tr><tr><td>2. Target-CSA</td><td></td><td>X</td><td></td><td>X</td></tr><tr><td>3. Brandt et al. 2017<br>Decision analysis approach</td><td>X</td><td>X</td><td>X</td><td></td></tr><tr><td>3. Fernandez et al., 2022<br>A framework to prioritize locally suitable CSA interventions</td><td>X</td><td>X</td><td></td><td>X</td></tr><tr><td>4. Khatri-Chhetri et al. 2019<br>CSA-RA</td><td></td><td>X</td><td>X</td><td>X</td></tr><tr><td>5. Mwongera et al. 2017<br>A generic framework for targeting, out-scaling and prioritizing CSA interventions in agricultural systems</td><td>X</td><td>X</td><td>X</td><td></td></tr><tr><td>6. Notenbaert et al., 2017<br>MWG model</td><td></td><td></td><td></td><td></td></tr><tr><td>7. Ouedraogo et al. 2018, Chiputwa et al., 2020<br>PICSA</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>8. 2018, Clarkson et al. 2019, Staub and Clarkson 2021<br>WCIS co-production process</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>9. Gbangou et al., 2020<br>Farmer field school approach to deliver CIS</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>10. Kumar et al. 2021<br>IMD-AAS (AGRIMET platform) as applied in Maharashtra Vedeld et al., 2020</td><td></td><td>X</td><td></td><td></td></tr><tr><td>11. Hydro-climatic EVO<br>Nyadzi et al., 2018<br>A framework to co-design climate-smart farming systems with local stakeholders</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>12. Andrieu et al., 2017<br>A tool for tree selection in</td><td>X</td><td>X</td><td></td><td>X</td></tr></tbody></table>

(continued on next page)
<table><caption>**Table 6**

Incorporating co-production principles in decision-support frameworks.</caption><thead><tr><th>Co-production principles (Nörstrom et al., 2020)</th><th>Ways to apply in a decision-support framework, as identified in our study</th></tr></thead><tbody><tr><td>Context-specific</td><td>Using already established interaction spaces from previous/on-going projects, for example: innovation platforms<br>Farmer Field Schools.<br>Incorporating actors already active in knowledge production, sharing and use, for example: local leaders of opinion, extension agents who have a good working relationship with farmers, experts recognized by their peers, actors who have large networks.<br>Screening for current practices, innovations or endogenous adaptation strategies and integrating them in the process.</td></tr><tr><td>Pluralistic</td><td>Having a selection strategy, for example selecting participants from a diversity of locally present situations (technical, social, economic), with a broad range of expertise, with a diversity of political, scientific, and technical views; making a conscious effort to include disadvantaged groups.<br>Assuring gender balance and disaggregate results according to gender, if relevant.<br>When building a research team, using criteria such as: experience with transdisciplinary research and participatory methods, posture, readiness to commit, non-dominating attitude, etc.<br>Providing space for conflicting views and negotiation.<br>Creating separate interaction spaces for groups with competing views/in case of strong power asymmetry.<br>Taking into account farmers' specific knowledge when selecting farmers.<br>Not putting different kinds of knowledge in competition, recognising their intrinsic value.</td></tr><tr><td>Goal-oriented</td><td>Formalizing with the participants project objectives and agreeing on when they will be achieved, establishing a monitoring/evaluation strategy, for example using Theory of Change.<br>Establishing together with stakeholders what desirable outcomes of the process are and using this opportunity to discuss their underlying values.</td></tr><tr><td>Interactive</td><td>Involving stakeholders throughout the process.<br>Creating opportunities for mutual learning and monitoring/assessing mutual learning.<br>Inviting participants to contribute to different elements of the method, for example: rank priorities collectively, participate in the selection of indicators, assigning priority weights to indicators, participating in the definition of the problem and the way in which research may be able to contribute to finding a solution, co-defining the objectives of a platform (or a different set-up) and how it would operate, identifying driving forces for scenario narratives, agreeing on evaluation criteria.<br>Inviting participants to produce their own tools of analysis, for example maps, calendars, matrixes.<br>If the project involves citizen science, putting traditional/indigenous knowledge at the same level as scientific knowledge, for example by creating space to upload it on the on-line tools.<br>If the project involves citizen science, involving participants in the design of the method/tool or its elements.<br>Letting the participants determine their own role/ level of implication in the process<br>Establishing a stakeholders' steering committee.</td></tr></tbody></table>

Responding to local demand or adapting the project trajectory (if results of monitoring and evaluation or of interaction with stakeholders demand it) will not be possible if the project design as well as the budget and reporting mechanisms don't allow it, regardless of the readiness or skills of research teams.

<table><caption>**Table 7**

Traces of co-production principles across the frameworks.</caption><thead><tr><th>Decision support framework</th><th>Context-based principle</th><th>Pluralistic principle</th><th>Goal-oriented principle</th><th>Interactive principle</th></tr></thead><tbody><tr><td>CSA-PF</td><td></td><td></td><td></td><td></td></tr><tr><td>1. Andrieu et al. 2017</td><td></td><td>X</td><td>X</td><td>X</td></tr><tr><td>2. Target-CSA</td><td></td><td>X</td><td></td><td>X</td></tr><tr><td>3. Brandt et al. 2017<br>Decision analysis approach</td><td>X</td><td>X</td><td>X</td><td></td></tr><tr><td>3. Fernandez et al., 2022<br>A framework to prioritize locally suitable CSA interventions</td><td>X</td><td>X</td><td></td><td>X</td></tr><tr><td>4. Khatri-Chhetri et al. 2019<br>CSA-RA</td><td></td><td>X</td><td>X</td><td>X</td></tr><tr><td>5. Mwongera et al. 2017<br>A generic framework for targeting, out-scaling and prioritizing CSA interventions in agricultural systems</td><td>X</td><td>X</td><td>X</td><td></td></tr><tr><td>6. Notenbaert et al., 2017<br>MWG model</td><td></td><td></td><td></td><td></td></tr><tr><td>7. Ouedraogo et al. 2018, Chiputwa et al., 2020<br>PICSA</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>8. Dayamba et al. 2018, Clarkson et al. 2019, Staub and Clarkson 2021</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>9. WCIS co-production process</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>10. Gbangou et al., 2020<br>Farmer field school approach to deliver CIS</td><td></td><td>X</td><td>X</td><td>X</td></tr><tr><td>11. IMD-AAS (AGRIMET platform) as applied in Maharashtra Vedeld et al., 2020</td><td></td><td>X</td><td></td><td></td></tr><tr><td>12. Hydro-climatic EVO<br>Nyadzi et al., 2018<br>A framework to co-design climate-smart farming systems with local stakeholders</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>13. Andrieu et al., 2019<br>A tool for tree selection in</td><td></td><td>X</td><td>X</td><td>X</td></tr></tbody></table>

(continued on next page)