# Culture of impact in agricultural research organisations: What for and how?

Marie Ferre, Genowefa Blundo Canto, Geraldo Stachetti Rodrigues, María-Margarita Ramírez-Gómez, Graciela Luzia Vedovoto, Beatriz-Elena Agudelo-Chocontá, Daniela Vieira Marques, Roberto Manolio Valladão Flores, Gonzalo-Alfredo Rodríguez-Borray, Mirian Oliveira de Souza, et al.

<figure><img src="image_1.png" /></figure>

## To cite this version:

Marie Ferre, Genowefa Blundo Canto, Geraldo Stachetti Rodrigues, María-Margarita Ramírez-Gómez, Graciela Luzia Vedovoto, et al. Culture of impact in agricultural research organisations: What for and how?. Research Policy, 2025, 54 (1), pp.105140. 10.1016/j.respol.2024.105140. hal-05105416

HAL Id: hal-05105416

https://hal.science/hal-05105416v1

Submitted on 10 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
research practices (Watts et al., 2008), advance impact evaluations<fnref n="1" /> (Douthwaite and Hoffecker, 2017), and make research and innovations more transformative to society (Schot and Steinmueller, 2018). For some research organisations, this trend resulted in the establishment of a “culture of impact” (Leeuwis et al., 2018; Hainzelin et al., 2017) whereby there is a general recognition within the organisation that research should also be designed and practiced according to the types of impact it aims to generate. A culture of impact (also sometimes referred to as “evaluative culture”<fnref n="2" />) encourages the research community at large to better understand and reflect upon their role in contributing to long-term societal change, equip and support researchers to fruitfully carry out this reflection, and infuse this learning process across the various levels of an organisation (Blundo-Canto et al., 2019). Such a culture is instrumental for supporting organisational learning in impact evaluation (Cooper, 2014; Rodrigues et al., 2010) and can be used as a mechanism to re-think innovation systems and their link to policies, and implement transformative changes in institutional settings (Weber and Rohracher, 2012; Kok et al., 2019).

Building an organisational culture is driven by a multiplicity of factors (e.g., organisation's history, routines, values, modes of governance, organisation's capacity to provide a common frame), which contribute to determine its nature, vision, and how it operates (Blundo-Canto et al., 2019). Klerkx et al. (2017) study how researchers engage in co-innovation and argue that the institutional context, including individual and organisational dimensions such as historical practices, highly influences the way new organisational norms, routines or shared expectations are set, governing actors' behaviour. Thus, the context conditions the type of responses to an organisational culture's ambition. In the agricultural field in particular, a number of studies have questioned management models in agricultural research organisations at the light of contextual factors. Schmid et al. (2016) and Maru et al. (2018a) contribute to the reflection on “new models” of agricultural research that can facilitate innovation and address complex problems of agricultural development by analysing its impact pathways. Others have studied how the association between the type of research (e.g., level of application) and its degree of perceived “responsibility” to generate impact on society influence the funding environment (Calvert, 2006). Other authors have studied the suitability of the CGIAR organisational models (including funding mechanisms) to its development mandate (Leeuwis et al., 2018). Conti et al. (2024) recently examined future organisational scenarios or prototypes of how agricultural research organisations might adapt to address complex systemic challenges. Finally, Rijswijk et al. (2019) use the concept of ‘organisational identity’<fnref n="3" /> to describe the responses to newly introduced practices (along digital agriculture) among agricultural research organisations by examining organisational nature and context as well as capabilities, existing practices, partnerships, and values.

The literature on agricultural research organisation models that integrate a culture of impact shows that it can take various operational forms. “Culture of impact” can be referred to as “impact plan”, “impact

strategy”, or “impact ambition”, involving different types of configurations for implementing and “infusing” organisational practices associated with such a culture (see Reed et al. (2022) for an overview of various configurations across a set of organisations). Among studies focusing on organisational issues pertaining to culture of impact, a few have investigated models that integrate monitoring and evaluation of research activities (Stone-Jovicich et al., 2019; Blundo-Canto et al., 2019), foster interactions and relationships with new social actors, and encourage the use of research products (e.g., Bin et al., 2013). Joly et al. (2016) analysed the research impact assessment systems established in five agricultural research organisations and their objectives. Turner et al. (2022) discussed the tension triggering impact evaluation practices (summative versus formative purposes<fnref n="4" />), and how the need for building evaluative capacity emerged in four agricultural research organisations and what it translated into. Preskill and Boyle (2008) investigated what “evaluative capacity building” (ECB) means in practice among fifteen organisations, focusing on participants’ motivations for engaging in ECB, learning strategies, and perceived outcomes. Finally, Mayne and Johnson (2015) and Maru et al. (2018b) investigated the understanding and use of Theory of Change concept in the agricultural and development field.<fnref n="5" />

Yet, and despite the recent rise of debates over culture of impact in agricultural research, in-depth understanding of what drives, internally and externally, agricultural research organisations to develop a culture of impact, as well as of the changes that a culture of impact in agricultural research organisations translates into for the individuals and for the organisation more generally, remain scarce. Addressing this issue is important for understanding, in a comprehensive manner, “what it takes” and “what it means” to introduce a culture of impact in research organisations, and thus for reflecting on suitable organisational strategies. Our study contributes to fill this gap by carrying out a cross-analysis of three organisations that explicitly promote this ambition. Thus, we conduct a comprehensive cross-case reflection on organisational drivers, structuring, and practices targeted to reinforcing the evaluation of societal impacts associated with agricultural research and associated learnings, and on how these in turn influence its strategies, values, and trajectories. Our study aims to address the following research questions: Which are the main motivations for an agricultural research organisation to build a culture of impact? How does a culture of impact roll out? Which changes does it entail? We take stock of three experiences and examine i) the organisational trajectories for developing a culture of impact, including the reasons to its emergence, and the internal and external factors that hindered or facilitated its development; ii) how the culture of impact materialises in terms of discourses, visions, strategies, methods, tools, spaces of exchanges, resources, and support; and iii) the various cognitive and behavioural changes this culture generates among the staff, e.g. in terms mainly of perceptions, knowledge, capacities, interactions, and practices.

## 2. State-of-the art – conceptualizing the notion of “culture of impact”

A comprehensive analysis of the concept of “culture of impact” calls for investigating different strands of the literature. Indeed, we first need

<footnote n="1">¹ Agricultural innovation systems are increasingly recognised as complex systems in which interventions are not fully predictable and do not generate linear impacts. Douthwaite and Hoffecker (2017) prone for distancing from standard logic models (like the “convention conventional model the “adoption impact pathway”) that tend to ignore the complexity dynamics and miss important alternate impact pathways, and for investing on and exploring more dynamic, flexible, systemic and “complexity-aware” approaches to monitoring and evaluation approaches and tools.</footnote>

<footnote n="2">² An evaluative culture is usually embedded into a culture of impact in the sense that it seeks out information on organisational performance in order to feed a learning process, for better managing and delivering its programmes and services (Mayne, 2008).</footnote>

<footnote n="3">³ Organisational identity refers to “what is central, enduring, and distinctive about an organisation’s character” and includes tangible and intangible attributes (Rijswijk et al., 2019).</footnote>

<footnote n="4">⁴ Reed et al. (2021) define these notions as follows: an evaluation designs with summative focus is oriented towards “achieving, evidencing and claiming impacts and being accountable”; a more formative focus is rather about “ongoing monitoring, learning, adaptation and taking epistemic responsibility for the generation of impact”.</footnote>

<footnote n="5">⁵ Maru et al. (2018b) use the notion of “mainstreaming” practices, in the sense of “embedding a new concept, principles or an approach into a routine practice of individuals and organisations of relevant domains, while recognising that there is no guarantee that the new approach will be institutionalised as originally intended”.</footnote>
to understand how and under which form a “new” organisational culture emerges and sustains itself. Second, culture of impact pertains, in part, to practicing impact evaluation and applying impact evaluation approaches and methods on a variety of objects, with the purpose, among others, to learn and integrate lessons learned into research practices, for improving how research contribute to societal impact in targeted areas. For this reason, organisational learning and its underlying mechanisms are key to the development of such a culture. Third, the introduction of a culture and new associated manners of thinking on research and interventions implies changes of various natures across different levels of the organisation: hence a focus on literature concerning how change occurs in an organisation.

An organisational culture is influential to development trajectory of an organisation and its practices (Alsabbagh and Khalil, 2017). It is defined by shared assumptions, values, beliefs, experiences, knowledge, and traditions that shape the organisation's identity and mission (Blundo-Canto et al., 2019; Schein, 1990). It builds and evolves over time, and is usually associated with structural and strategic reconfigurations, redesign of systems and procedures, as well as changes in operations and job profiles (Mintzberg and Westley, 1992). For investigating change in an organisational culture, Muscalu (2014) suggests to explore on beliefs, attitudes, and behaviour, in order to assess the degree of fitness of uptake of a new culture, and the deepness of changes associated with it. Furthermore, an organisational culture shapes people's behaviour through learning (Blundo-Canto et al., 2019; Akhtar et al., 2011), and is also the result of that learning (Seyedyousefi et al., 2016). A variety of cultures often co-exist within an organisation (Akhavan et al., 2014), among which Seyedyousefi et al. (2016) identify a learning culture as one that promotes the development of others.

Organisational learning is of growing interest to researchers and practitioners (Jyothibabu et al., 2012; Templeton et al., 2002). It is defined as the ability of an organisation to process knowledge (i.e., create, acquire, transfer, and integrate knowledge in an objective of improvement, or adaptation to a context and new opportunities (Jerez-Gómez et al., 2005)), and refers to the process by which the staff acquires capabilities, learn, and innovate (Chiva & Alegre 2007; Cayla, 2007). These processes tend to be gradual, cumulative, and systemic, involving contributions of both individual and collective learning experiences to achieve organisational goals (Rebelo and Gomes, 2007), as well as moves of knowledge between different levels of action, from the individual to the group level, and to the organisational level and back again (Alsabbagh and Khalil, 2017; Liao and Wu, 2010). Along this line, the concept of organisational learning capability relates to organisational and managerial characteristics that facilitate the organisational learning process (Chiva and Alegre, 2007) and help shape a culture within an organisation (Abdi and Senin, 2015). Various criteria are identified to assess such capabilities: knowledge management (Akhavan et al., 2014; Omotayo, 2015), management commitment, systems perspective, openness and experimentation (Jerez-Gómez et al., 2005), risk-taking, interaction with the environment, dialogue, and participatory decision making (Alegre and Chiva, 2008).

Importantly, the development of any organisational culture involves a set of changes at individual and organisational levels for the culture to “permeate” and “infuse” an organisation. With the latter term transpires the idea that the realisation of changes requires both a formal implementation plan with specific structures or units in charge, and a diffusionist approach made of lesser coordinated avenues and intermediaries. These changes involve implementing specific “spreading” strategies across the organisation, adapting the organisation's visions, adopting or adapting rules and actions that foster specific reflections, and supporting learning processes to develop values and attitudes and orient research practices and interactions. Yet, fostering learning and change is not an easy process (e.g., it implies fluidity of knowledge between different levels of action: individual, group, and organisation (Alsabbagh and Khalil, 2017; Liao and Wu, 2010)). Turner et al. (2023) illustrate this difficulty by examining how a new multi-actor innovation logic

integrates with the different institutional logics that co-exist within an agricultural research institute. They found that organisations and their members need specific time for learning and reflecting on the values and practices associated with the newly introduced logic. Furthermore, McKay et al. (2013) and Choi and Ruona (2011) argue that individual readiness to change is key to enable organisational change, and that the readiness level tends to be positively influenced by an environment that promotes a learning culture.

Establishing a culture of impact in a research organisation, in particular, goes along with the development of a posture at organisational and individual levels that questions the impacts to which research contributes, their articulation, and their evaluation (Blundo-Canto et al., 2019). It may also involve a shift in the way impact evaluation is being thought through, i.e., switching from an objective of impact accountability and a summative purpose to a learning and formative purpose, stimulated by the engagement of more diversified stakeholder groups along research design, monitoring, and evaluation (Spaapen, 2015; Joly and Matt, 2022; Matt et al., 2017). For instance, Saari and Kallio (2010) discuss the value of “developmental impact evaluation” that combines frameworks of organisational learning and impact evaluation, facilitating continuous strategic thinking in research processes, through both learning within innovation networks and promoting impact-oriented reflections. Blundo-Canto et al. (2019) formulate broad principles in regards to what a culture of impact in an agricultural research organisation would translate into: (i) transdisciplinary dialogue on the multiple roles and functions played by researchers; (ii) adaptability to different needs, lexicon and functions to foster appropriation; (iii) support by high-level management and fitting in a broader strategical frame; (iv) capacity reinforcement of staff in understanding and characterising the impacts of research; (v) resources dedicated to develop and sustain the culture of impact; and (vi) adequate internal and external communication towards promoting this culture. The necessary changes associated with the uptake of a culture of impact are very much fostered by organisational learning, implying a degree of awareness and criticism on own research practices (Watts et al., 2007), the capacity of the organisation to generate, analyse, transfer, and integrate knowledge on research processes and their impacts (Jerez-Gómez et al., 2005), and a capacity to modify behaviour and actions following reflection on this new knowledge (Zhou et al., 2015; Chiva et al., 2007), in order to improve practices and performances, and to adapt to new opportunities.

The review of the above literature emphasises that organisational change towards the integration of a culture of impact in agricultural research organisation is enabled by a supportive environment (Wolf et al., 2015; Blundo-Canto et al., 2019; Joly et al., 2016), including the development of suitable strategies, visions, skills, and tools, which can foster changes in organisational and individual practices. The uptake of such a culture is therefore very much revealed by individual and organisational level “pushes”. This justifies our research questions and focus on capturing i) historical path of development of a culture of impact, ii) organisational level attributes, strategies and actions supporting its development, and iii) individual level of changes that result from this culture.

## 3. Material and method

### 3.1. Study cases

We analyse three agricultural research organisations that have embarked on a similar ambition with regard to the establishment of a culture of impact. They are the French Agricultural Research Centre for International Development (Cirad), the Brazilian Agricultural Research Corporation (Embrapa), and the Colombian Agricultural Research Corporation (AGROSAVIA) - see Appendix 1 for maps with their organisational reaches. Table 1 below presents their main attributes. These organisations operate in the Global South; they slightly vary in their mandate and reach of action: Cirad has a broad mandate to support
<table><caption>**Table 1**
Descriptive information of the organisations.</caption><thead><tr><th></th><th>Cirad</th><th>Embrapa</th><th>AGROSAVIA</th></tr></thead><tbody><tr><td>Attached to</td><td>Ministry of Higher Education, Research &amp; Innovation, &amp; Ministry of Europe &amp; Foreign Affairs</td><td>Ministry of Agriculture and Livestock</td><td>Ministry of Agriculture and Rural Development</td></tr><tr><td>Date of creation</td><td>1984</td><td>1973</td><td>1993</td></tr><tr><td>Mission</td><td>Works in partnerships for the sustainable development of tropical and Mediterranean regions</td><td>Works to assure the sustainability of agriculture through solutions in research, development and innovation for the benefit of Brazilian society</td><td>Transform sustainably the Colombian agricultural sector with the power of knowledge to improve the lives of producers and consumers</td></tr><tr><td>Research units (Appendix 1)</td><td>Headquarter in France, 14 regional offices in Global South</td><td>43 decentralised research units and 24 central units, in Brazil; 4 international scoping laboratories (Labex)</td><td>1 central headquarter, 13 regional research centres, 8 local units, 2 experimental farms, in Colombia</td></tr><tr><td>Number Researchers</td><td>1140</td><td>2190</td><td>778</td></tr><tr><td>Number staff</td><td>1650</td><td>7790</td><td>1865</td></tr><tr><td>Approximate budget</td><td>215 million USD (2022): 65 % public funds, 35 % own resources (contract-based)</td><td>716 million USD (2022), from federal government</td><td>80 million USD (2022): 92 % public funds, 8 % own resources</td></tr></tbody></table>

Note: Situation of the three organisations in 2022. Inspiration for designing this table: Bin et al. (2013).

international development through agricultural research; Embrapa has a national mandate mainly focused on agricultural technology and R&D (it also has scoping offices, known as Labex in various continents); AGROSAVIA has a national mandate focusing on applied agricultural research and technical services. This diversity provides contrasting cases in terms of trajectories, characteristics, and external contexts, which may have influenced the construction and rolling out of the culture of impact, while common features likely emerge. The choice of these cases is further justified by the fact that they share an intentionally implemented process to institutionalising impact evaluation and reflexive approaches on research impact, which is not formalised up to this level in other agricultural research organisations.<fnref n="6" /> Moreover, each of these organisations is in demand for a reflective (cross)-learning process on this organisational priority,<fnref n="7" /> allowing conducive conditions for analysing the effects of a culture of impact. Next, we succinctly describe the three cases and their respective organisational activities centring impact evaluation approaches.

Cirad works in partnerships (public and private partners) to build

knowledge, alternatives, and interfaces supporting innovation and agricultural development in tropical and Mediterranean regions. In 2010, in line with a “research-for-development” mandate, Cirad initiated a process of institutionalising reflexivity on the contribution of research to societal impacts (Hainzelin et al., 2017, 2016). It translated into the development of approaches for supporting and facilitating these reflections. An *ex post* evaluation method (ImpresS *ex post*) was developed (Barret et al., 2017) based on reflexive and participatory retrospective analyses of innovation paths, collectively rebuilding impact pathways and investigating the cumulated “impacts” of sets of agricultural research interventions over long timeframes, thus enabling collective reflection on the contribution of research to impacts (Faure et al., 2018; Faure et al., 2020; Blundo-Canto et al., 2020a). Then, an adaptive *ex ante* (prospective) approach (“ImpresS *ex ante*”) (Blundo-Canto et al., 2020b) was designed, engaging collective thinking among actors involved in a future research intervention (research and non-research actors) on the changes and impacts that it would generate and the plausible pathways by which to achieve them.<fnref n="8" /> These tools have been institutionalised through Cirad’s organisational strategy (“Strategic Vision”, Cirad, 2017) and planning documents like the “Strategic science and partnership objectives” (Cirad, 2018b) where one of Cirad’s ambitions is stated to be the development of the culture of impact in the organisation.

Embrapa works on developing solutions to enhance the country’s production of food, fibres, and energy, through research, development and innovation to ensure the sustainability of agriculture for the benefit of Brazilian society. In the last 25 years, Embrapa developed and systematised structures and methodologies to account for comprehensive impact assessments of the technologies designed through its research programmes (Avila et al., 2015; Rodrigues et al., 2010; Vedovoto et al., 2022).<fnref n="9" /> This ambition is exercised in two ways: i) the organisation-wide application of an *ex post* Reference Methodology, composed of “Economic Surplus” (Avila et al., 2008) and “Ambitec-Agro” (Rodrigues et al., 2003, 2010) methods that, in a multidimensional approach, considers economic, social, environmental, and institutional development impacts of agricultural technology adoption; and ii) the use of these studies and results of impact assessments for the annual release of its “Social Balance Report” (26th edition this year, Embrapa, 2023) as well as for managerial documents (organisational performance evaluation, Embrapa’s Strategic Master Plan), accountability reports, and scientific and collaboration purposes (publications, conference papers, projects/research, partnerships). Embrapa directives also encourage research teams to carry out *ex ante* impact assessments, based on the same methods to determine ‘expected impacts’ from technology development research and transfer. This *ex ante* process not being formalised yet, the present study will not focus on this. In the development of a culture centred on impact, in its latest Strategic Master Plan, Embrapa established a set of goals to guide strategic, tactical, and operational research actions towards a vision of impact-based results (Embrapa, 2020).

AGROSAVIA works on producing knowledge and technologies that are used and adapted to sustainably transform the Colombian

<footnote n="6">⁶ Joly et al. (2016) report on the extent to which impact evaluation results are used by agricultural research institutes, indirectly indicating a growing interest in culture of impact among those organisations: the US Department of Agriculture (USDA), the French National Institute for Agricultural Research and Environment (INRAe), the Commonwealth Scientific and Industrial Research Organisation (CSIRO) in Australia, Embrapa, the Consultative Group on International Agricultural Research (CGIAR). This effort exists in New Zealand’s Agresearch and France’s INRAe but their mandates are not in the Global South.</footnote>

<footnote n="7">⁷ Regular discussions between AGROSAVIA, Cirad, and Embrapa on this subject started in 2018 in the frame of a one-week workshop organised by Embrapa and the International Centre for Tropical Agriculture (CIAT) from CGIAR, which aimed to present the evaluation approaches of the respective institutions (Embrapa, 2018a, 2018b).</footnote>

<footnote n="8">⁸ Support to the conception and implementation of Monitoring and Evaluation participatory and actor centred approaches in Cirad’s interventions has been ongoing. Since it was very new in 2022, it is not part of the study.</footnote>

<footnote n="9">⁹ Embrapa has put into place a systematic and wide routine for reporting out in thousands of farms and technology adopting units, on hundreds of technological innovations, contributing to a large database available to public consultation via its impact platform webpage (https://www.embrapa.br/balanco-social). Results feed directly into establishment and monitoring of quantitative strategic goals, with baselines monitored and checked as to their fulfilment. Hence, strategic planning with specific, measurable, traceable, and time-bound targets became a milestone at the institution, possible via the consolidation of the impact assessment process.</footnote>
agricultural sector by increasing sustainable management of natural resources and improve Colombian populations' livelihoods. AGROSAVIA has long been developing economic and environmental impact assessments of part of its projects, technologies. Since 2016, it has embarked on the institutionalisation of a comprehensive impact analysis strategy that includes components: i) "Social Balance" (AGROSAVIA, 2022) inspired by the methodology developed by Embrapa, which assesses the economic and socio-environmental impacts of a sample of technologies and other actions of the Corporation, and which the results then feed annual reporting; ii) *ex ante* impact analysis in the formulation phase of R&D and transfer projects, to estimate the adoption and potential impacts of the technologies generated and disseminated among targeted populations, which recently (2022) evolved towards identifying contextual problems and opportunities for projects' implementation and for designing more plausible anticipated impact pathways along the use of developed technologies; iii) the recent development of other *ex post* impact assessment methods; and iv) the development of a culture of impact in the organisation, as part of a corporate strategic planning called "cultural transformation" with horizon 2030, having generation and evaluation of impacts as central elements.

## 3.2. Methodology

### Analytical dimensions and data collection

As per our research questions, we focused on three main analytical dimensions: i) the historical path of development of a culture of impact and its *modus operandi*, looking at the evolution of the organisation and the historical emergence of strategies and tools to develop a culture of impact; ii) the organisational level attributes, strategies, actions, and motivations that constitute the supportive environment to the culture of impact, including drivers, constraints, and opportunities to its development; and iii) the cognitive and practical individual changes experienced by staff, as well as their perceived organisational change following the development of such a culture. To explore these dimensions, we relied on a combination of different methods, which were adapted across the organisations to fit their respective contexts. Table 2 describes the data collection process and samples of participating populations.

First, we reviewed the scientific and grey literature pertaining to the organisations, including reports, publications, and strategic and operational documents. This aimed to specify how the notion of "culture of impact" is defined and communicated within each organisation, how it is being "practiced" within the organisation according to their explicit operational strategy, as well as reconstitute the respective historical trajectories of development of a culture of impact. Second, using interviews and structured and open-questions surveys, we reached to agents who actively contributed to the historical path and reflection premises of the culture of impact (Table 3), in order to capture key drivers to the establishment to such a culture and potential obstacles, constraints, and opportunities to its development. In Embrapa, the survey was sent out to 103 managers linked to Embrapa Headquarters and 43 Decentralised Units spread throughout the country. The 18 respondents included managers across different research centres and top management at the organisation's Headquarters, who were, for most of them, involved with impact assessments, specifically in the preparation of impact reports, coordination of impact assessment teams, and selection of technologies to be evaluated in the decentralised units. In Cirad, respondents were selected as per their involvement in the trajectory of building the culture of impact, and their management responsibilities in the organisation. Out of the 23 persons invited for 1-hour interview, 13 accepted the invitation. In AGROSAVIA, the interviews were carried out with researchers who had a focus on impact issues, directors whose role is closely related to impact generation in the Corporation due to their capacity to make decisions. Out of 39 persons requested, 31 accepted the invitation.

<table><caption>**Table 2**
Data collection – overview.</caption><thead><tr><th rowspan="2">Activity</th><th rowspan="2">Target population</th><th colspan="3">Number of responses</th></tr><tr><th>Cirad</th><th>Embrapa</th><th>AGROSAVIA</th></tr></thead><tbody><tr><td>Literature review</td><td>Relevant literature pertaining to the organisations, providing information on the trajectory and <i>modus operandi</i> of culture of impact</td><td>NA</td><td>NA</td><td>NA</td></tr><tr><td>Interview or survey</td><td><b>Key informants, key players of the historical process of development.</b><br>This includes researchers and managers: Table 3<br><b>Participants or contributors to impact-related activities</b><br>- In CIRAD: contributors and participants to the development and application of <i>ex post</i> and <i>ex ante</i> evaluation approaches.<br>They include researchers and support staff. Target population was selected based on records of participants to these activities<br>- In EMBRAPA:</td><td>13</td><td>18</td><td>31</td></tr><tr><td>Engaged-population survey</td><td><ol><li>Appointed agents involved in annual impact assessment of technology innovations. They include researchers and analysts associated with four Research Units. Target population was selected based on focal points' recommendations</li><li>Agents in departments that conduct the "Social Balance Report", including researchers who have technologies adopted and referred to in this process<br>- In AGROSAVIA: contributors and participants to the development and application of <i>ex post</i> and <i>ex ante</i> evaluation approaches.<br>They include researchers and support staff</li></ol></td><td>65</td><td>16</td><td></td></tr></tbody></table>

Note: "NA": not applicable.

Third, we conducted a survey (hereafter referred to as "engaged-population survey") among the staff of each organisation who have been closely participating or contributing to the activities directly pertaining to the culture of impact (e.g., methodological development, support to project development, technological or project evaluation, strategic planning). Thus, we aimed to capture the effects of these activities, which can broadly be differentiated into *ex ante* and *ex post*-related activities, on individual changes. We asked survey participants whether they experienced changes in multiple domains as a result of their involvement in impact evaluation-related activities and the nature of change they have experienced. Those included cognitive changes (i.e., in terms of perception, knowledge, attitude, capacity) and behavioural changes (i.e., practice, interaction), and cover multiple areas, from the conception of research ideas to the building of projects, their monitoring, evaluation, and research interactions (Table 4 for an overview). The survey also included questions on opportunities that supported
<table><caption>**Table 3**
Key informants of the historical trajectory of culture of impact – participants’ profiles.</caption><thead><tr><th rowspan="2">Profile</th><th colspan="3">Number of participants</th></tr><tr><th>Cirad</th><th>Embrapa</th><th>AGROSAVIA</th></tr></thead><tbody><tr><td>Researcher involved in methodological development of impact evaluation organisational approaches, support materials</td><td>0</td><td>0</td><td>6</td></tr><tr><td>Former or retired researcher/staff involved in methodological development of impact evaluation organisational approaches, support materials</td><td>1</td><td>0</td><td>3</td></tr><tr><td>Current manager of the organisation (research unity, department, section)</td><td>10</td><td>18</td><td>21</td></tr><tr><td>Former or retired manager of the organisation (research unity, department, section)</td><td>2</td><td>0</td><td>1</td></tr></tbody></table>

<table><caption>**Table 4**
Grid of changes surveyed at individual level, as a result of an organisational culture of impact.</caption><thead><tr><th>Main change</th><th>Type of change</th><th>Description</th></tr></thead><tbody><tr><td rowspan="5">Cognitive change experienced</td><td rowspan="2">Knowledge</td><td>About notions related to impact and impact evaluation</td></tr><tr><td>About the types of impacts to which research can contribute</td></tr><tr><td rowspan="2">Capacity</td><td>About the innovation (process) under study</td></tr><tr><td>About the principles, specificities and conditions of application of impact evaluation methods</td></tr><tr><td rowspan="6">Practical changes experienced</td><td rowspan="2">Perception</td><td>To interact with actors potentially impacted by the research</td></tr><tr><td>To use impact reflection methods independently</td></tr><tr><td rowspan="2">Interaction</td><td>Of the role/mission of your organisation</td></tr><tr><td>Of your own role within your organisation and your team</td></tr><tr><td rowspan="2">Behaviour</td><td>Of the role of research in contributing to societal impacts</td></tr><tr><td>With partners in the South and actors in the field</td></tr><tr><td rowspan="2">Observed/perceived organisational change</td><td rowspan="2">Behaviour</td><td>Interactions with staff from other professions and/or disciplines in the organisation</td></tr><tr><td>In formulating/constructing research questions</td></tr><tr><td rowspan="2"></td><td rowspan="2"></td><td>In choosing themes and issues and research questions in the retrospective evaluation</td></tr><tr><td>In planning future interventions</td></tr><tr><td rowspan="2"></td><td rowspan="2"></td><td>Changes in the organisation at the operational or managerial level: e.g., changes in vision, communication, knowledge management</td></tr><tr><td></td></tr></tbody></table>

these changes. Note that in Embrapa, another survey was conducted, which aimed to capture postures and practices that align with a culture of impact, beyond those who interacted directly with *ex post* impact evaluation activities. The survey was conducted among staff (n = 123) involved with social balance reporting; it brings complementary findings.

The above steps were possible through building an inter-institutes group composed of researchers and support staff (authors of the paper) who were highly involved in the building of a culture of impact in their organisations. We thus undertook a mutual learning process translating into regular interactions between 2021 and 2023 for building and implementing the analytical framework, the methodology and analysing the results, including a 3-days workshop in Bogotá-Colombia in April 2023. This process was particularly useful to create a foundation to share experiences, aspirations, and be able to interrogate and adjust culture of impact's path in accordance with contexts and priorities.

# 4. Results

## 4.1. Trajectories of development of a culture of impact in each organisation

We trace and compare the trajectories of development of the culture of impact in the three organisations, highlighting major phases, task forces, key events, and supporting tools and approaches (see Appendix 2 for a graphical representation of organisations’ trajectories, and details). Table 5 captures the key comparative elements to the trajectories. We find that Cirad, AGROSAVIA, and Embrapa have followed relatively similar paths for building a culture of impact. They all share methodological development reflections, organisation of trainings and facilitation events, internal and external moments of exchanges, design of supporting tools to equip researchers for carrying out evaluations, setting up of specific human and financial resources, and an organisational structure that evolved step-by-step towards creating conducive conditions for a culture of impact to realize. Yet, a few points deserve attention, as follows.

First, the three organisations show similar reasons for developing a culture of impact, which is primarily to demonstrate impacts of research (i.e., accountability purpose): this contextual incentivizing factor to prove usefulness of research investments was strongly experienced by the organisations. This factor was then completed by an (internal) learning aspiration in the aim of contributing to strategic planning of the organisation, and which expresses differently across the cases. For Cirad, the learning process is oriented towards improving research practices, better reflecting on the plausible and multiple impacts of interventions, and (ultimately) contributing to inform strategic organisational decisions. In contrast, the learning process in Embrapa is more oriented towards gathering information for, on the one hand, enabling adjustment and adaptive management of proposed technologies in order to maximize adoption rates and technological performance, and on the other hand informing the directions of organisational investment in terms of choice of technologies to be developed. This finding very much echoes the contrasts along main purposes for promoting evaluation: it is rather a formative purpose in Cirad with a focus on learning to produce new knowledge on how research contributes, as part of long-term innovation processes, to societal impacts, while it is mainly used as a metric for organisational performance and assessment of economic outcomes in Embrapa (with effect on funding level allocated to research branches), with a focus on innovation transfer and adoption. In AGROSAVIA, the purpose is a combination of both in the sense that it is proposed as a guide to the corporative agenda that seeks both to improve research practices and metric to measure societal impact.

Methodological development and tools’ accessibility is another major component, common to the three trajectories. Cirad, Embrapa, and AGROSAVIA trajectories are all marked by strong investments in *ex post* evaluation methodological developments, which then fed *ex ante* methodological reflections, serving the design and planning of interventions. Methodological choices, however, vary. With regard to *ex post* method, Cirad developed its own methodology in order to suit to the nature of the agricultural objects studied (often involving long-term innovation process, complex agricultural systems, multitude interactions between research and other stakeholders), its desire to focus on disentangling the contribution of its research activities to impact among other contributions, and its willingness to account for multiple voices in the application of evaluation approaches, therefore relying on multi-stakeholders participatory processes and actor-centred methods. For building this method, Cirad largely exchanged with various experts from international agricultural research institutes and relied on internal research work; this is a major step of its culture of impact’s trajectory. AGROSAVIA and Embrapa on the other hand chose existing methodologies suitable to quantitatively measure the gain of adoption, i.e., the net economic benefits of their projects or technologies (through the ‘economic surplus’ method in particular), with a stronger reliance on the
<caption>**Table 5**</caption>

Key elements across trajectories of development and modus operandi of a culture of impact in the three organisations.

<table><caption>**Table 5 (continued)**</caption><thead><tr><th></th><th>Cirad</th><th>Embrapa</th><th>AGROSAVIA</th></tr></thead><tbody><tr><td>Trajectories of development</td><td></td><td></td><td></td></tr><tr><td>Contextual "pushing" factor</td><td>Ex post: alignment with research organisations questioning impact of research, societal/funders pressure</td><td>Ex post: desire to demonstrate return on investment of public research; communicate about societal impacts and organisational performance; orient decision-making on technology development and transfer processes</td><td>Ex post: change in direction of the organisation with a stronger focus on economic and social impacts and accountability on how resources are invested</td></tr><tr><td>Triggering element</td><td>Ex ante + MEL: Build project and research interventions differently (participatory and actor cantered).<br>2015: A dedicated organisational project to develop systematic ex post evaluation. Then, an institutional decision (2017) to build on ex post learnings to develop ex ante (with dedicated budget)<br>Ex post: learning on research contribution to societal impacts, improving research practices</td><td>Ex ante: maximize technology adequacy and adoption rates to favour agricultural economic returns<br>1997: An organisational decision based on the first Strategic Planning, to implement ex post impact assessments on agricultural research</td><td>Ex ante (corporate model): funders' demands<br>Ex ante (participatory model): governmental demand for more inclusive research 2016: a dedicated macro-project and team to establish corporate impact assessment strategy (ex ante, ex post, culture): partnership with Embrapa<br>2021: ex ante (participatory model): partnership with Cirad<br>Ex post: accountability, learning to improve research and transfer practices, institutionalise impact evaluation</td></tr><tr><td>Purpose</td><td>Ex ante and M&E: improving plausibility/groundness and change-oriented management of research for development projects</td><td>Ex ante: improvement of research project focus, prediction of technology adoption rates</td><td>Ex ante: prioritising institutional research agenda and better directing/orienting projects</td></tr><tr><td>Evolution of purpose</td><td>Ex post and ex ante: same purpose + strengthening culture of impact</td><td>Ex post: establishing goals in organisational strategic planning to guide strategic & operational research actions towards impact-based results; strengthening culture of impact</td><td>Ex post: accountability and learning to improve research and transfer practices, strengthening culture of impact<br>Ex ante: improving plausibility - groundness of research projects (participatory model)<br>2019-2030 corporate</td></tr><tr><td>Institutionalisation</td><td>2018-2028 strategic plan</td><td>2021-2030 strategic plan</td><td></td></tr></tbody></table>

<table><thead><tr><th></th><th>Cirad</th><th>Embrapa</th><th>AGROSAVIA</th></tr></thead><tbody><tr><td></td><td>includes impact as one of the four organisational "ambitions"<br>Institutionalised ex post and ex ante evaluation approaches</td><td>includes impact as a "strategic goal"<br>Institutionalised ex post evaluation approach</td><td>strategic plan includes impact as a key "higher purpose"<br>Institutionalised ex post evaluation (Social Balance); ex ante approach being institutionalised<br>An "impact team", a dedicated funding mechanism</td></tr><tr><td></td><td>An Impact team (ImpresS)<br>A dedicated funding mechanism</td><td>An "impact team"<br>A dedicated funding mechanism</td><td>An "impact team", a dedicated funding mechanism</td></tr><tr><td>Modus operandi</td><td>ImpresS team: 3 researchers,<br>4 impact mission agents, 1 department director (coordinator)</td><td>Impact assessment team located at SMAE for coordinating the system nationwide (1 supervisor, 2 researchers, 4 analysts), in SPAT (Social Balance teams) in decentralised units: 150 staff, researchers and analysts for carrying out field evaluations</td><td>Impact-Base Team: 4 researchers, 1 director of planning and institutional cooperation (coordinator)<br>Social balance team (ex post): 94 staff in research centres, local units, Communications Office<br>Culture team: 4 staff of planning, Human Management</td></tr><tr><td>Impact teams</td><td>Project development agents: trained by ImpresS teams for strategic planning of intervention using ImpresS ex ante</td><td>Superintendence of strategy and decentralised units</td><td>Research Centres, Planning and Institutional Cooperation Directorate</td></tr><tr><td>Organisational positioning</td><td>Research unit + impact and science marketing direction (project development team), direction research & strategy<br>Interventions/ project/ strategies planning and M&E building, with project development officers, accompanying Evaluative capacity building, Fostering reflexive learning on impact within the institution<br>Impact and outcome evaluation<br>Methodological development</td><td>Evaluative capacity building; methodological support<br>Fostering reflexive learning on impact within the institution<br>Impact evaluation<br>Methodological development</td><td>Project/strategies planning and accompanying<br>Evaluative capacity building</td></tr><tr><td>Key functions of impact-dedicated team</td><td>Scientific publication<br>Monitoring organisational goals on impact</td><td>Scientific publication<br>External communication<br>Monitoring organisational goals on impact</td><td>Fostering reflexive learning on impact within the institution (researchers, managers, support staff)<br>Impact evaluation<br>Methodological development</td></tr></tbody><tfoot><tr><td colspan="4">(continued on next page)</td></tr></tfoot></table>
<table><caption>Table 5 (continued)</caption><thead><tr><th></th><th>Cirad</th><th>Embrapa</th><th>AGROSAVIA</th></tr></thead><tbody><tr><td rowspan="2">Formalisation</td><td>For ex ante and ex post: Dedicated yearly budget (functioning, evaluations funding); Institutional annual meeting presentations</td><td>For ex post (social balance): Dedicated yearly budget; Institutional annual meeting presentations and board of directors' presentations for ex post</td><td>External communication</td></tr><tr><td>Representation in the direction research and strategy</td><td>Representation in the superintendence of strategy Performance evaluation of the decentralised units</td><td>For ex post (social balance): Dedicated yearly budget; Presentation at annual general meeting of active members</td></tr><tr><td rowspan="2">Developed evaluation approaches</td><td>Ex post: ImpresS ex post participatory impact pathway evaluation method applied on long-term innovation processes (projects' cluster of 15–30 yrs)</td><td>Ex post: “Reference Methodology”: economic surplus, Ambitec-Agro for socio-environmental and organisational development dimensions: Ex post impact assessments on adopted technologies (~160 yearly), in participative field studies with selected users (Social Balance report)</td><td>Coordination in Planning and Institutional Cooperation Directorate</td></tr><tr><td>Ex ante: ImpresS ex ante approach on interventions, projects, programmes, value chains, partnership networks (for strategic planning)</td><td>Ex ante: Project-oriented assessments for determining ‘expected impacts’ from technologies</td><td>Ex post (Social Balance): economic surplus and Ambitec-Agro (ref. Embrapa) applied on selected technologies</td></tr><tr><td rowspan="2">Tools</td><td>Monitoring, Evaluation &amp; Learning (MEL) and outcome evaluation: offer in development</td><td>Organisational Performance Assessment process</td><td>Ex ante (corporate model): quantitative adoption estimation method applied to the technologies being developed Ex ante (participatory model): “Weaving Impact” method applied in the participatory formulation of projects</td></tr><tr><td>Guidelines ex post method Guidelines ex ante approach Trainings</td><td>Guidelines ex post</td><td>Guidelines ex post (economic surplus and Ambitec-Agro methods, selection of technologies, definition of adoption criteria); Guidelines social balance</td></tr><tr><td></td><td>Videos</td><td>Trainings</td><td></td></tr><tr><td></td><td>Dedicated page on organisational website Dedicated website Dedicated knowledge</td><td>Dedicated page on organisational website</td><td>Guidelines ex ante</td></tr><tr><td></td><td></td><td></td><td>Trainings</td></tr></tbody></table>

<table><caption>Table 5 (continued)</caption><thead><tr><th></th><th>Cirad</th><th>Embrapa</th><th>AGROSAVIA</th></tr></thead><tbody><tr><td rowspan="2">Implementation modalities</td><td>management system Inter-organisational community of practice on change-oriented approaches</td><td>Dedicated website</td><td>Videos Dedicated page on organisational website</td></tr><tr><td>Ex post: voluntary-based, on responses to an organisational call and selected by a committee made of impact team and direction</td><td>Ex post: annually mandatory with selection of technologies by each decentralised unit's social balance team, with financial support from management branch</td><td>Ex post: annually mandatory for a sample of technologies selected by the corporative community</td></tr><tr><td rowspan="2">Spaces of exchange</td><td>Ex ante and MEL support: voluntary application by research teams, tailored to each intervention</td><td>Ex ante implemented occasionally, depending on interest from project, when there is an interaction with economists</td><td>Ex ante corporate model: mandatory for all projects</td></tr><tr><td>Inter-organisational community of practice Annual internal presentations</td><td>Inter-decentralised units community of practice</td><td>Ex ante Participatory model: voluntary applications</td></tr><tr><td rowspan="2">Appropriation dynamics</td><td>Young recruit's introduction</td><td>Annual presentations ex post</td><td>Annual training and workshop of lessons and learnings of the Social Balance teams</td></tr><tr><td>Adaptations of ex post Spontaneous requests for ex ante support</td><td>Adaptation of ex post methods</td><td>Spontaneous requests for ex ante support</td></tr><tr><td rowspan="2">Key documents</td><td>Autonomous application of ex post and ex ante</td><td>Autonomous application of ex post method</td><td>Autonomous application of ex ante (corporate model)</td></tr><tr><td>Ex post impact case study reports (standardised) Ex post outcome evaluation reports (ad hoc) Ex ante reports (ad hoc)</td><td>Social Balance yearly report (standardised)</td><td>Social Balance yearly report (standardised)</td></tr><tr><td></td><td>Feeds into: Strategic science and partnership objectives; Science and society note; Cirad annual activity report</td><td>Feeds into: Strategic Master Plan, Business Plan, Organisational Performance Evaluation</td><td>Ex ante reports (ad hoc)</td></tr><tr><td></td><td></td><td></td><td>Feeds into: Strategic frameworks of innovation networks</td></tr></tbody></table>

Note: In 2021 – time of the data collection for this survey, the MEL and outcome evaluations were not in place yet in Cirad; same for ex ante Participatory model at AGROSAVIA.
attribution principle.<fnref n="10" /> They then widened the scope using the 'Ambitec-Agro method' to also capture the economic, social, environmental, and organisational development impact dimensions of the technology transfer and adoption. Inter-institutes dialogue also took place during these trajectories. In fact, AGROSAVIA's *ex post* evaluation and annual reporting methods ('social balance') are directly inspired by Embrapa's experience. With regard to *ex ante* methodological development, principles also vary. For Cirad, the objective is an encouragement towards staff carrying reflective processes for better planning of interventions and the impacts they target. In AGROSAVIA and Embrapa, the sake of *ex ante* tends to be oriented towards improving project focus, and predictions of adoption rates and associated economic returns of the developed technologies. Yet, AGROSAVIA has recently evolved its approach towards higher level of contextualisation analysis prior to introduction of technologies in a specific territory (AGROSAVIA, 2023), and consideration of participatory processes for increasing plausibility of impact pathways. Last on this methodological pillar of the trajectories, we find that all organisations made considerable investments in developing procedures, accessible methodological guidelines, platforms, trainings, for encouraging organisational staff to question and undertake these approaches.

In sum, the three trajectories are marked by a methodological path (in the aim of stimulating ideas and improving evaluation approaches), an organisational path (i.e., translating into different levels of recognition and promotion of the culture), and a scientific path, that feeds methodological development, accountability of the contribution of research to societal impacts, and learning processes. The realisation of these paths is possible through significant and continuous dedicated human and financial resources, specific organisational tools and dedicated teams to support its development, and a level of systematisation in the application of the evaluation approaches (in terms of procedures (call) or choice of technologies/cases).

## 4.2. Modus operandi – the creation of a supporting environment to a culture of impact

We detail the current vision and operational system of the culture of impact in each organisation, highlighting the engines of the organisations that contribute to the development and uptake of such a culture. Appendix 3 presents the respective organisations' *modus operandi* (i.e., chosen way of proceeding) for promoting a culture of impact, and Table 5 underlines key comparative elements on the same. Looking at how each organisation expresses or defines “its” culture of impact provides preliminary insights into the chosen modalities of deployment. Culture of impact in Cirad is described as a desire “*at both the individual and collective levels to better understand the complex mechanisms at work within innovation processes that generate impacts. It is based on rigorous tools and on the desire to better interact and work with the actors involved in innovation processes. It takes the form of capitalizing on collective experiences (ex post), continuously improving research practices, and taking impact pathways into account ex ante in the programming of actions undertaken by Cirad*” (Cirad, 2018a - website). At Embrapa, the “culture of impact” can be defined as the promotion of an organisational mind-set and practice that seeks to measure, communicate, and maximize the positive impact of research, development, and innovation activities for the sustainability of Brazilian agriculture and society as a whole. In AGROSAVIA, it is defined as a corporate aspiration for which the staff, from their different roles, orient their individual and collective actions to generate well-being for producers and consumers and towards the sustainable

management of agroecosystems. Those formulations (often not framed as institutional statements) bring out contrasting attributes. Cirad for instance highlights the idea of fostering understanding mechanisms to the generation of impacts and working with actors involved in innovation processes; while from Embrapa's discourse emerge ideas of assessment, measure, and communication towards social accountability. AGROSAVIA on the other hand makes larger references to a global learning process in the institution, which shall foster the culture of impact.

Important as part of the *modus operandi* is the institutionalisation and formalisation of this culture, with the concept being embedded into key strategic documents: in the Strategical Scientific and Partnership Objectives (OSSP) and strategic vision for Cirad (Cirad, 2017, 2018b); in the institutional Strategic Master Plan for Embrapa by including operational impact assessment activities linked to short and long-term goals (Embrapa, 2020), and in corporate strategic plan for AGROSAVIA. All organisations have also invested in setting up an “impact team” with specific expertise and skills, and recognised at organisational level as main engine for deploying a culture of impact. We find that the composition of these teams varies but is commonly composed of researchers (often from an economic background), support staff, and directive representatives. This proximity to the direction level is important for facilitating communication and feeding into strategic choices. Another common aspect is that these teams share various missions including research, methodological development, support and accompaniment of staff in the application of evaluation methods, monitoring of these applications, and accompaniment of research teams in strategic planning and monitoring via customizing and adapting ImpresS *ex ante* approach to the specific intervention (for Cirad specifically).

We find that all organisations count on “brokering” or “bringing” people between the “impact team” and research units, to optimize diffusion of practices and reflections among research teams and partners, and thus enable a “multiplier effect” of the culture of impact. In Cirad, these bridging people are the “project development officers” (20 in total) who are being trained primarily to use ImpresS *ex ante* and M&E tools in their day-to-day activities, in order to enhance their routine and autonomous use in research teams support activities. In Embrapa, this diffusion process occurs in particular through the allocation of financial and human resources to the 43 Decentralised Units for both training of the evaluation teams (present in each Unit) involved in impact assessments (SPAT), and carrying out annual impact assessments of technologies, through liaising with the responsible researchers and investigating “technology-adopting” stakeholders. In AGROSAVIA, the “Base team” is linked up with 13 Social Balance Teams (in each research centre) that are continuously trained by the former, and jointly develop analyses. Moreover, the three organisations aim at passing on this culture of impact to their partners. In Cirad, this is part of the mandate of the ImpresS team. In Embrapa and AGROSAVIA, the impact teams are often called upon by organisational partners - research institutes, NGOs, Universities - for their impact assessment expertise. Last, accessibility to support materials, methodological guidelines, technical and organisational publications, and promotion of workshops and courses related to impact assessment is key to promote learning and capacity strengthening, and a priority in these organisations.

Yet, we find that the way culture of impact is implemented follows diverse modalities and assumptions. In Cirad for instance, all *ex post*, M&E, and *ex ante* approaches are internal processes carried out voluntarily by research teams and their partners, with methodological support from the ImpresS team and the project development team. This voluntaryness principle has been chosen in the idea that, at the difference with imposing staff involvement into these activities, it would facilitate appropriation of impact-related and participatory concepts and methods, as well as autonomous reflexive capacity on the contribution of research to impacts. We observe that Embrapa and AGROSAVIA in contrast, opted for more systematic and mandatory procedures,

<footnote n="10">¹⁰ Evaluation applications in Embrapa follow a proposed “reference methodology” based on econometrics and multi-criteria approaches. Investigation with specific objectives and deeper analyses (e.g., with experimental or quasi-experimental design, counterfactuals) usually involve research specialists, whether in economics, sociology, or ecology for instance.</footnote>
formalising dedicated research teams to coordinate and promote impact evaluation activities, including organisational recognition and compensations for the quality of their work.

A difference also exists along standardisation of procedures in link with the application of evaluations, and capitalisation of results. In Cirad, the conduct of evaluation used to follow an “opportunity/demand-based principle” along which ImpresS ex post evaluation or ImpresS ex ante accompaniments would be carried out on demand and in a flexible manner. Recently (2022), a funding mechanism has been put into place to incentivise for application to ex post evaluations every year (still on a voluntary basis). There is not yet clear mechanism in Cirad enabling systematic capitalisation of lessons from ex post or ex ante accompaniments to the top level. In Embrapa, however, standardised and systematic procedures are in place around ex post assessments and social balance: every year, the ‘Technology Prospection and Evaluation Sector’ (SPAT) teams in each decentralised unit submit technology impact reports to the central ‘Strategy Monitoring and Evaluation Supervision’ (SMAE) team, for compilation of the annual institutional research highlights, the annual bibliometric review, the file of awards and distinctions received, the accounting balance-sheet, and all information for the Social Balance Report. The latter provides strategic information on research and technology impacts to support the sustainability of the national agricultural sector. In AGROSAVIA, the Social Balance teams participate in the development of the strategy, and maintain a permanent relationship with the other Directorates in order to share information and discuss results. The Corporation allocates an annual budget to develop the impact strategy, sometimes complemented with external funding.

### 4.3. Insights into drivers, obstacles, and opportunities to the development of a culture of impact

The above findings were derived from documentary analyses and discussions with representatives of the impact teams of the organisations engaged in this study. We now report on the results from in-depth interviews with key organisational informants who consist, for the majority, of high-level managers having long-term experience in the organisations, and thus also a sharp and comprehensive view of their functioning. While part of the results confirms findings highlighted in Table 5, others are interesting insights into the drivers, obstacles, and opportunities to the development of a culture of impact (see Table 6). First and along the reasons for launching a culture of impact, beside commonly shared motivations around demonstrating usefulness of activities and generating learning to feed into agenda priorities, we note an ethical positioning in line with the R4D mandate of the institution calling for a responsibility to think and practice research along the socioeconomic impacts it (aims to) generate (Cirad), and thus a need to change paradigm and move from a “culture of promises” to a “culture of impact” (Hainzelin et al., 2016). Another interesting reason, also experienced by Cirad, was a strong willingness to stand out from other organisations in terms of impact approaches, and develop comprehensive and complexity-aware approaches. A desire for transparency towards society (public) on what research does and how it performs was also reported (Embrapa). It was also noted that the development of such a culture was very much enabled by a synergy of human resources willing to substantially push the reflection on this at that particular moment. With regard to common difficulties for introducing a culture of impact, part of key informants reported a relative resistance by staff to different approaches and new questionings, a fear of additional workload, and a fear of losing creativity of research and freedom in knowledge production.

Reported drivers to the development of this culture and the adhesion of staff to it are of two kinds. On the one hand, they are about internal conducive conditions, including i) supportive direction, ii) perception of an opportunity for the culture of impact to explore new questions and new paths in ones' profession, conducting inter-disciplinary works, and

fostering mutual learning across fields and types of actors, iii) internal recognition of socio-economic works, and iv) presence of a motivated, leading, and devoted teams able to play the role of intermediaries between disciplines and actors. For the case of Cirad, low level of formalisation, and tailored support for each specific intervention accompaniment request are two additional drivers. On the other hand, external drivers also matter. For instance, in Embrapa is reported the importance of external recognition, organisation's image and relationship with society through a comprehensible dialogue, hence the importance of a culture of impact. Then, in terms of constraints to its deployment, nature of research disciplines appears as potential brakes, together with insufficient resources (in terms of operational capacity to meet demands on evaluations), and lack of strategic vision by some leaders. Embrapa specifically also reports on lack of uniform tools to facilitate data collection with accurate and precise bases, and lack of capitalisation on learning to enable continuous improvement in practices, processes, and scientific productions. The latter is considered as a risk to culture of impact. This may suggest that fatigue among staff may arise if no organisational capitalisation process is in place to take count of learnings of evaluations and thus of individual and collective work investments.

A majority of informants report that this culture is generally accepted but there is a feeling that it did not yet lead to profound transformations on research practices, nor full appropriation at all organisational levels (the next section provides details on the infusion of this culture on research practices). Organisations coincide in their views along with the need for continuous investments, convinced leaders, and recruitment of staff trained or open to training on the topic (i.e., having “change-oriented” mind-sets). Other reported opportunities for fostering development of a culture of impact included creating more occasions for inter-disciplinary exchanges/works, relying on “champion” researchers to infuse the culture of impact in their unit, increasing communication on impact themes and research results, working with different communities and types of agriculture, and generating spaces for dialogue on the topic.

### 4.4. Changes generated in the organisations by engaging into the culture of impact

#### 4.4.1. Sample description and overview of analyses

In Table 7, we present the main characteristics of the “engaged-population” sample, and Table 8 presents how the analysis was conducted on the sample.

#### 4.4.2. Changes in perception vis-à-vis a culture of impact

We analysed the general perception and changes in the perceptions of the “engaged-population” in regard to the culture of impact in the organisation (Table 9 below). We find that the majority of participants do believe in an alignment between the role of the organisation and its ambition to establish a culture of impact; this perception not varying significantly across positions (research management, research support, researcher). Along this, participants, regardless of the organisation, describe the ambition of a culture of impact with the following statements: “justified”, “essential”, “relevant”, “necessary”, “crucial”, “expected”, and “strategic”, highlighting the relevance of this culture. In AGROSAVIA, participants report specific benefits of having a culture of impact as “improving corporate visibility”, “being an instrument of accountability”, encouraging “performance in activities, commitment, identity, and incentivising teams to achieve objectives”, or achieving “greater social recognition in contributing to the development of the country's agricultural sector”. In Cirad, respondents explain that this culture “enables the organisation's values to be put into practice”, including e.g., commitment to development and beneficiaries, co-construction of interventions with different actors, sharing of knowledge and skills, and trans-disciplinarity. Notably, something that could be referred to as a “turning point” is the fact that the deployment of the culture of impact has switched from a perceived constraint to an interest by staff. Yet, a set of
<table><caption>**Table 6**

Why a culture of impact and under which conditions?</caption><thead><tr><th></th><th></th><th>Cirad semi-directed interviews, n = 13</th><th>Embrapa closed-questions survey, n = 18</th><th>AGROSAVIA structured interview, n = 31</th></tr></thead><tbody><tr><td rowspan="10">Construction phase</td><td rowspan="8">Reasons-motivations<br/>for building a culture of impact</td><td>- Being better at demonstrating usefulness of Cirad's activities</td><td rowspan="8">- Transparency of information for society and control bodies<br/>- Accountability to society justifying the investments in agricultural RD&amp;I<br/>- Establishing clear metrics to perform holistic assessment of research results and impacts<br/>- Monitoring and communicating about the impact of agricultural technologies<br/>- Demonstrating the importance of technological solutions developed through research<br/>- Directing and prioritising future actions and research on topics of greater relevance<br/>- Strengthening organisational image and importance</td><td rowspan="8">- Accountability to government and society being a publicly funded institute that must fulfil its societal mandate<br/>- Excelling into delivering positive impacts to farmers, society, and the environment.<br/>- Demonstrating to society the benefits resulting from investing in research, development, and innovation<br/>- Fostering long-term interest of key researchers in reflexive learning on the impact of the organisation<br/>- Desire for an organisational appropriation of the impact reflection.</td></tr><tr><td>- Responding to societal demand on demonstrating public investments into research</td></tr><tr><td>- Desire to shift from scientific excellence to research in partnerships' relevance and usefulness</td></tr><tr><td>- Shifting from a culture of promises to a culture of impact</td></tr><tr><td>- Moving away from the dichotomy research vs. development; bringing research closer to societal needs</td></tr><tr><td>- Fostering an organisational reflexive capacity, to inform research strategic programming</td></tr><tr><td>- Fostering organisational learning and improving research practices</td></tr><tr><td>- Ethical positioning of Cirad</td></tr><tr><td>- Willingness to stand out from other organisations in terms of impact approaches, and develop comprehensive and complexity-aware approaches</td></tr><tr><td>- Building upon the strategic planning 2008–2011 stating request to produce a methodological reflection on the impact of its activities</td></tr><tr><td rowspan="10">Difficulties, constraints<br/>to the emergence of a culture of impact</td><td>- A synergy of conducive (human-related) factors that enabled the launch of such culture</td><td rowspan="10">- Scarcity of human and financial resources<br/>- Methodological and training difficulties: need to adapt the method to different types of technologies and maintain a continuous training schedule<br/>- Resistance to changes of mind-set, especially for those who have worked for many years at Embrapa<br/>- Logistical difficulties and lack of financial resources in surveying impact data in field studies</td><td rowspan="10">- Dependency on public sector funding, government policies, which can prevent the organisation from directing its own agenda<br/>- Resistance of researchers and administrative staff to conduct work related to economic and social impact</td></tr><tr><td>- Perception of “instrumentalisation” of researchers by the direction to carry out evaluations</td></tr><tr><td>- Fear of additional workload, anxiety vis-à-vis the level of effort demanded by new approaches</td></tr><tr><td>- Fear of losing freedom in knowledge production, creativity of research</td></tr><tr><td>- Resistance to open up to new questioning, uncertainties etc.</td></tr><tr><td>- Perception of no added value, and that research practices are already (implicitly) impact-driven</td></tr><tr><td>- Fear that this might lead to questioning of research postures, epistemics</td></tr><tr><td>- Perception of an opportunity to foster inter-disciplinary works, mutual learning across fields and types of actors, and about the position of own research</td></tr><tr><td>- No hierarchical request, low level of formalisation; a voluntary approach (change in research practices and critical thinking not imposed)</td></tr><tr><td>- A team of motivated, leading, and devoted people with communication skills</td></tr><tr><td rowspan="10">Development &amp; “infusion” phase</td><td rowspan="8">Drivers of the culture of impact (facilitators to its appropriation; conducive conditions)</td><td>- Brokering people able to play the role of intermediaries between disciplines and actors' profiles</td><td rowspan="8">- Internal competence to carry out impact assessments (well trained and up-to-date teams)<br/>- Awareness of teams on the internalised innovation process<br/>- Organisation's image and relationship with society through a comprehensible dialogue (annual Social Balance Report)<br/>- Influence of impact assessments on the organisational performance evaluation process<br/>- Communication and interaction with managers and teams about the importance of impact studies<br/>- An institutionalised and standardised methodological basis</td><td rowspan="8">- Strategic presence of the Corporation at the regional level through the Research Centres<br/>- Enables to showcase the results of actions in the field to partners and society more broadly (through the annual Social Balance report - impact analysis strategy)<br/>- Financial security because of the public status and projects/alliances with the private sector<br/>- Increased awareness among researchers, who are more open and alert to the impact that their projects can generate</td></tr><tr><td>- To be able to justify research actions and provides arguments to focus on some topics</td></tr><tr><td>- A way to valorise researcher's work in impact analysis (via scientific publications)</td></tr><tr><td>- Internal communication through website, oral presentations to staff</td></tr><tr><td>- A supportive direction, willing to provide human and financial resources</td></tr><tr><td>- Link of the ImpresS team with high-level management, visibility</td></tr><tr><td>- An institutionalised team of agents (DIMS/project development team,</td></tr></tbody></table>

(continued on next page)
<table><caption>Table 6 (continued)</caption><thead><tr><th></th><th>Cirad semi-directed interviews, n = 13</th><th>Embrapa closed-questions survey, n = 18</th><th>AGROSAVIA structured interview, n = 31</th></tr></thead><tbody><tr><td></td><td>ImpresS) able to accompany project or research development and infusing questioning on impact generation and corresponding mechanisms<ul><li>A diversity of agents in their profile and background being "touched" by the culture</li><li>"Word of mouth" among agents to share experiences and raise interest of others</li><li>Capacity to adapt to the demand of researchers, tailored support for each specific intervention and request, flexibility in the type of accompaniment proposed</li></ul></td><td></td><td></td></tr><tr><td>Hindering factors to its appropriation</td><td><ul><li>Type of discipline makes adhesion to these approaches and concepts more or less easy: agents already working on "finalised research", participatory action research and transdisciplinarity (e.g., on production systems) tend to consider that they already carry out such reflections, resulting sometimes in resistance; versus agents working on "more distant" objects (e.g., fundamental research like genomic characterisation, biological processes)</li><li>Level of exposure to other disciplines and approaches. Some disciplines already working with similar questioning and tools may be reluctant to new "similar" approaches</li></ul></td><td><ul><li>Insufficient financial resources, which restricts the set of technologies to be assessed</li><li>Lack of strategic vision by some leaders</li><li>Lack of uniform tools for all Decentralised Units to facilitate data collection with accurate and precise bases</li><li>Lack of capitalisation on learning to enable continuous improvement in practices, processes, and scientific productions</li><li>Lack of adequacy between operational capacity (number and trained people) and volume of demands with regard to evaluation work</li></ul></td><td><ul><li>Complexity of internal processes, hindering establishment of continuous interactions with farmers communities</li><li>Lack of articulation between research department and department in charge of bringing technological offers to producers</li><li>Very few researchers in economic, social and human disciplines</li><li>Lack of transdisciplinary work in the development of research and innovation proposals</li><li>Centralised decision-making process, giving little power to directors of the research centres in their regions</li></ul></td></tr><tr><td>Perceptions</td><td><ul><li>Majority find it essential given development mandate of Cirad</li><li>Consistency of the concepts with what is going on in the organisation</li><li>This culture generally accepted, included at management levels, but did not yet lead to a transformation on research practices</li><li>Staff took the culture of impact up with different levels of engagement, depending on their interest. Little resistance, natural to go along this way</li><li>Staff has a desire to learn about and test evaluation approaches</li><li>Enables increased awareness on strategic orientation and roles to play for developing agricultural innovations and contributing to long term innovation processes with multiple stakeholders</li><li>Enables researcher to formulate stronger and more informed messages to decision makers</li><li>Contributes to foster dialogue among disciplines and professions; offers opportunities to work together, and spaces of dialogue</li><li>Integrating modules on "research impact" in doctorate programmes</li><li>Integrating selection criteria in the recruitment processes (for "change-oriented" researchers)</li><li>Focusing on a larger spectrum of agents categories in the organisation beside researchers, such as technicians, who have a role to play for making change and impact happen</li><li>Expanding criteria for the evaluation and promotion of researchers (to include a specific effort to contribute to fostering change and impact as a criteria)</li><li>Offering more opportunities for inter-disciplinary exchanges/works to researchers, as part of increasing mind-openness</li></ul></td><td><ul><li>For most managers, the dissemination of a culture of impact is of fundamental importance for organisational survival, and must be advocated and recognised by all employees and leaders in the organisation</li><li>Development of a culture of impact has evolved as expected</li></ul></td><td><ul><li>Elements in the corporation that denote an emergence of a culture of impact. Yet, a need to generate actions that allow it to be fully appropriated at individual and organisational levels.</li></ul></td></tr><tr><td>Opportunities, suggestions to further development of culture of impact</td><td></td><td><ul><li>Methodological level: Improving evaluation methodologies, streamlining processes to allow assessment of more technologies, updating tools based on worldwide benchmarking organisations</li><li>Institutional/strategic level: continuous training of teams, reinforcing culture of impact in management, continuous investment in impact assessment, promoting interaction and partnerships with academic and non-academic communities, improving interaction between impact assessment and strategic/planning processes, recruiting staff with expertise in related areas</li></ul></td><td><ul><li>Generating incentives, especially for researchers, supporting to focus their projects on impact</li><li>Polices of the current (2022) government focusing on territories and small and medium-sized producers are in accordance with the corporate transformative superior purpose.</li><li>Increasing work with producers and generating participatory work methodologies</li><li>Working with different communities (indigenous, Afrocolombian-black communities, smallholders) and types of agriculture</li><li>Establishing a structure (group) transversal to the entire corporation</li></ul></td></tr></tbody><tfoot><tr><td colspan="4" style="text-align: right;">(continued on next page)</td></tr></tfoot></table>
<table><caption>**Table 6 (continued)**</caption><thead><tr><th></th><th>Cirad semi-directed interviews, n = 13</th><th>Embrapa closed-questions survey, n = 18</th><th>AGROSAVIA structured interview, n = 31</th></tr></thead><tbody><tr><td></td><td><ul><li>Cultivating an opening to what is happening outside, work of other organisations</li><li>Widening the Impact team to further meet with coming demands from staff</li><li>Presence of "champion" researchers that contribute to infuse culture of impact in their unit</li><li>Targeting institutes partners to adhere to culture of impact</li></ul></td><td><ul><li>Communication level: extending dissemination about impact theme and research results</li><li>Impact assessment coordination team: further integrating support and technical teams, together with productive sector; better defining a coordinating team at the national level, able to carry out field surveys and provide adequate resources</li><li>Strengthening culture of impact at all levels of organisation: units, directorate, heads, superintendence, management, supervision</li></ul></td><td>that is in charge of championing and implementing the culture of impact throughout the corporation.<br/><br/>- Generating spaces for internal dialogue, at all levels of organisation</td></tr></tbody></table>

Note: Related findings across the organisations are presented along a same row.

<table><caption>**Table 7**

Descriptive statistics of "engaged-population".</caption><thead><tr><th>Descriptive variables</th><th>Cirad (n = 65)</th><th>Embrapa (n = 16)</th><th>Embrapa (n = 123)</th><th>AGROSAVIA (n = 294)</th></tr></thead><tbody><tr><td>Gender (female)</td><td>42.3 %</td><td>25 %</td><td>80 %</td><td>41,5 %</td></tr><tr><td>Age</td><td>12.7 %</td><td></td><td></td><td>29,3 %</td></tr><tr><td>20–35 years old</td><td>46 %</td><td>63 %</td><td>34 %</td><td>48,6 %</td></tr><tr><td>36–50 years old</td><td>41.3 %</td><td>31 %</td><td>48 %</td><td>20,4 %</td></tr><tr><td>51–65 years old</td><td>0</td><td>6 %</td><td>18 %</td><td>1.7 %</td></tr><tr><td>Over 65 years old</td><td>69 %</td><td>44 %</td><td>52 %</td><td>74,1 %</td></tr><tr><td>Researcher</td><td>15 %</td><td>0</td><td></td><td>1,0 %</td></tr><tr><td>Research support (project development)</td><td></td><td></td><td></td><td></td></tr><tr><td>Direction</td><td>1,5 %</td><td>0</td><td></td><td>5,8 %</td></tr><tr><td>Research management (analyst)</td><td>0</td><td>56 %</td><td>46 %</td><td>0</td></tr><tr><td>Administrative support</td><td>0</td><td>0</td><td>2 %</td><td>19 %</td></tr><tr><td>Others (PhD students, engineer)</td><td>15 %</td><td>0</td><td></td><td>0</td></tr><tr><td>Number of years in organisation</td><td></td><td></td><td></td><td></td></tr><tr><td>1–10 years</td><td>31 %</td><td>37 %</td><td>4 %</td><td>61,9 %</td></tr><tr><td>11–20 years</td><td>37 %</td><td>50 %</td><td>50 %</td><td>20,4 %</td></tr><tr><td>Over 21 years</td><td>28 %</td><td>13 %</td><td>46 %</td><td>17,7 %</td></tr><tr><td>Engagement Intensity (EI) score<sup>a</sup></td><td>2.5 (1.2) (scale: 1 to 5)</td><td>3.1 (1.7) (scale: 1 to 6)</td><td>Not applicable</td><td>3.08 (2.07) (scale: 1 to 7)</td></tr></tbody></table>

Note: The higher the EI score, the higher his/her engagement has been. See Appendix 4 for an overview of the nature of such engagement and Appendix 5 for details on the calculation of EI. In Cirad, researchers include Cirad's researchers and partners' researchers.

<table><caption>**Table 8**

Sampling and data analysis.</caption><thead><tr><th></th><th>Cirad</th><th>Embrapa</th><th>AGROSAVIA</th><th>Type of analyses</th></tr></thead><tbody><tr><td>1. All participants of "engaged-population"</td><td>65</td><td>16 and 123</td><td>294</td><td>General perception, role of research, institution, opportunities</td></tr><tr><td>2. Participants who took part to ex ante activities exclusively</td><td>50</td><td>NA</td><td>202</td><td>Type of change experienced</td></tr><tr><td>3. Participants who took part to ex post activities exclusively</td><td>8</td><td>16</td><td>26</td><td>Comparison across EI: low EI, high EI</td></tr></tbody></table>

Note: For a matter of better comparability of the results, we studied the changes experienced by the targeted population (outside general perceptions) by looking on the one hand at staff who was exposed to ex ante evaluation related activities, and on the other hand at staff exposed specifically to ex post evaluation related activities.

the population mentioned insufficient resources and concrete actions in link with this culture. Next, a majority reported that this culture has influenced the way they perceive the role of research in society. In Cirad, for instance, participants report a "more optimistic vision of the role of research" or "a better understanding of the various roles research can play and the synergies with other stakeholders". In AGROSAVIA, this finding is

<table><caption>**Table 9**

Proportion of respondents that report a change in perception as a result of culture of impact.</caption><thead><tr><th></th><th>Perception of an alignment between culture of impact and role of organisation</th><th>Change in one's perception of one's role within organisation and team</th><th>Change in one's perception of the role of research in contributing to societal impacts</th><th>Changes observed in the organisation (management, process) as a result of the organisational impact ambition</th></tr></thead><tbody><tr><td>Cirad (n = 65)</td><td>86 %</td><td>25 %</td><td>52 %</td><td>52 %</td></tr><tr><td>Embrapa (n = 16)</td><td>94 %</td><td>38 % (n = 8)</td><td>75 % (n = 8)</td><td>56 %</td></tr><tr><td>AGROSAVIA (n = 294)</td><td>83 %</td><td>52 %</td><td>58 %</td><td>30 %</td></tr></tbody></table>

illustrated by statements like "research is essential for the improvement of society" or "research must be aimed at closing gaps, generating impacts, and satisfying the needs of a community, a sector". In contrast, we find limited change in the perception of the roles within teams and organisation. This might be explained by a potential selection bias, in the sense that these participants have been engaging (voluntarily or not) into impact-evaluation activities, and are likely well aware of their role as part of the organisation. Last, about half of the participants observed

<footnote><sup>a</sup> The EI score corresponds to the reported level of engagement of participants into "impact-evaluation activities".</footnote>
organisational change, e.g., with regard to development and planning of interventions, that would result, in part, from the impact-related actions carried out in the organisation. In Cirad, observed organisational changes seem to translate into stronger communication about impact, change in the way research interventions are conceived, and implementation of a common lexicon around impact evaluation. Some respondents reported their views in how they thought a culture of impact relates to: in AGROSAVIA, terms like “change”, “transformation”, “appropriation”, “orientation”, “prioritisation”, “values”, “identity”, or “belonging” were used.

### 4.4.3. Cognitive and practical individual changes resulting from a culture of impact

We present the type of changes that the “engaged-population” has experienced as a result of the exposure with either *ex ante* or *ex post* evaluation activities (see Table 10). In terms of cognitive changes associated with *ex ante* activities, we find that a large majority of participants reported feeling more acquainted with impact notions and how to evaluate it. This change of knowledge includes, in particular, gain of impact-related suitable vocabulary, increased knowledge along impact evaluation methods and approaches, and impact pathways, as per Cirad’s responses. In AGROSAVIA, participants indicated a better understanding of the productive systems and a strengthening of their knowledge along evaluation methodologies. With regard to exposure to *ex post* evaluation activities specifically, AGROSAVIA’s respondents reported a gain of knowledge and a strengthening of *ex post* methodologies (Ambitec-Agro). Changes seem less pronounced with regard to capacity to interact with impacted actors, or to implement reflections and methods in an independent way. A bit less than half of the participants in Cirad mentioned that involvement into *ex ante* impact-evaluation activities has helped them increase their levels of interaction with other professions and disciplines of the organisation, and hence their level of understanding for those. Among those who experienced a change, some

reported to now see differently the role and place of impacted actors: they are increasingly considered by the respondents as centred actors in the interventions and not “only” as beneficiaries. One respondent from Cirad stated specifically that the use of *ex ante* approaches has enabled to “strengthen the transition of farmers from beneficiaries to actors, and to acquire the reflex of ‘actor-centred’ questioning: with whom? for whom? by whom? who should do what differently?” Moreover, 69% of respondents at AGROSAVIA describe a similar change: through these new interactions, they report a more active level of interdisciplinary participation, leading to i) a better understanding of the production system, which generates more comprehensive results: e.g., “today the focus of the projects has changed: we are called to formulate more comprehensive proposals, where several disciplines participate, making it necessary to make connections or alliances among researchers from different disciplines”, “collaborative interaction has improved the approach to proposals and their projected impact”; and ii) a co-responsibility in the formulation, development, and achievement of proposed goals: e.g., “interaction with other research teams generates a better construction and adoption of the technology”, “improved communication between areas and knowledge enables the formulation of most realistic schedule”.

With regard to changes in interactions resulting from engaging with *ex post* activities, several respondents from AGROSAVIA explain that evaluation has allowed them to understand the importance of working with other areas and domains of the corporation. People expressed “the Centre’s Social Balance team is multidisciplinary, besides we evaluate different Technology Offer thus complementing knowledge and information.”, “the evaluations are interdisciplinary works, so we are in constant interaction with people from other disciplines and professions and we are constantly learning.” This also includes changes in external interactions, which have, according to some respondents, reinforced their conviction of the importance of interacting with producers in order to better identify their perceptions, needs, and strengthen adoption of developed technologies. This is reflected in quotes like “it is always necessary to

<table><caption>**Table 10**
Proportion of “engaged-population” that reports cognitive and/or practical changes.</caption><thead><tr><th></th><th>Change in knowledge about impact and impact evaluation</th><th>Change in knowledge about the types of impacts to which research can contribute</th><th>Change in knowledge of principles, specificities, conditions of application of the method</th><th>Change in capacity to interact with the actors potentially impacted by the research</th><th>Change in capacity to use impact assessment methods, approaches independently</th><th>Change in interactions with other professions and/or disciplines at the institution</th><th>Change in the way of formulating & constructing research questions</th><th>Change in the way of planning future interventions</th><th>Change in the way to manage/ coordinate interventions and their steering</th></tr></thead><tbody><tr><td colspan="10"><strong>Exposure to ex post activities</strong></td></tr><tr><td>Cirad (n = 8)</td><td>75 %</td><td>50 %</td><td>62.5 %</td><td>37.5 %</td><td>50 %</td><td>87.5 %</td><td>37.5 %</td><td>62.5 %</td><td></td></tr><tr><td>Embrapa (n = 16)</td><td>78 % (n = 8)</td><td>75 % (n = 8)</td><td>63 % (n = 8)</td><td>63 % (n = 8)</td><td>38 % (n = 8)</td><td>63 % (n = 8)</td><td>25 % (n = 8)</td><td>38 % (n = 8)</td><td></td></tr><tr><td>AGROSAVIA (n = 26)</td><td>81 %</td><td>88 %</td><td>58 %</td><td>65 %</td><td></td><td>73 %</td><td></td><td>69 %</td><td></td></tr><tr><td colspan="10"><strong>Exposure to ex ante activities</strong></td></tr><tr><td>Cirad (n = 50)</td><td>84 %</td><td></td><td>74 %</td><td>48 %</td><td>32 %</td><td>44 %</td><td>54 %</td><td>30 %</td><td>24 %</td></tr><tr><td>AGROSAVIA (n = 202)</td><td>63 %</td><td></td><td>59 %</td><td></td><td></td><td>69 %</td><td>55 %</td><td>60 %</td><td></td></tr><tr><td colspan="10"><strong>Differentiation by level of EI (for ex ante)</strong></td></tr><tr><td>Cirad EI Low (n = 33)</td><td>82 %</td><td></td><td>67 %</td><td>52 %</td><td>21 %</td><td>36 %</td><td>58 %</td><td>36 %</td><td>30 %</td></tr><tr><td>Cirad EI High (n = 17)</td><td>88 %</td><td></td><td>88 %</td><td>41 %</td><td>53 %</td><td>59 %</td><td>49 %</td><td>18 %</td><td>12 %</td></tr><tr><td>AGROSAVIA EI Low (n = 127)</td><td>51 %</td><td></td><td>49 %</td><td></td><td></td><td>61 %</td><td>49 %</td><td>53 %</td><td></td></tr><tr><td>AGROSAVIA EI High (n = 75)</td><td>83 %</td><td></td><td>76 %</td><td></td><td></td><td>81 %</td><td>65 %</td><td>72 %</td><td></td></tr></tbody></table>

Note: Cells containing no data means that the survey did not integrate a comparative question (reminder: data collection tools were customized to each organisation).
know the perception directly of the producers to identify opportunities for improvement or additional actions to maximize results” or “the potentially affected actors provide insights into future reception of technological offer”.

More than half of participants who engaged in *ex ante* activities, in both Cirad and AGROSAVIA reported a change in the way of formulating and constructing research questions, illustrated in the following ways: “better planning based on concrete objectives and available resources”, “construct research questions that are consistent with the stakeholders’ visions”, “more relevant research questions in terms of their importance in the territory and focus on generating impact”, “forces the formulator to think prospectively and in terms of impact”, “the producer and the consumer are the essence of the Corporation, and the research questions should revolve around them”, or “higher concerns about impact explains change in the ways of formulating research questions aimed at adoption and subsequent impact”. The level of reported change in planning and management of research interventions is lower though. Yet, those who report a change, mention “co-construction of impact pathways that enable a higher level of efficiency in the way of piloting interventions” (Cirad), or ‘construction of more robust research proposals and greater success in external calls for project financing’ (AGROSAVIA).

Considering the results from the survey on staff involved with the Social Balance at Embrapa (n = 123), we find that half of the participants report that they very frequently collaborate with researchers from areas other than their own, and reflect with colleagues on how research and projects are conceived: 75 % answered that they often partake in reflection groups. Regarding the sharing of experiences and field knowledge, more than 80 % of people report to do it very often. Moreover, the majority of this sample reported carrying out discussions, and proposition of methods, tools, and approaches to analyse the impacts of research (e.g., on the use of reference methodology, on the results of the Social Balance report). Last, more than half of the participants reported to often build and share impact-related skills, and knowledge with others. Yet, 11 % claim never doing so.

#### 4.4.4. Factors of changes and opportunities

Among the population that engaged with *ex ante* activities, we analysed the effect of the level of engagement (EI score) on the experienced level of change (Table 10). We find that levels of changes along knowledge, capacity, and some research practices tend to be, in proportion, higher among the sample of population that more intensively/ frequently engaged with evaluation-activities. This finding is reassuring in the sense that it confirms the idea that for a culture of impact to impregnate a population, time and continuity of support is needed, together with level of recurrence of the activities.

Our methodological design was also an opportunity to gather thoughts on potential difficulties and suggestions from participants for strengthening these changes and further promote this culture. As part of difficulties, some respondents from Cirad mentioned that it is not easy to enter and get familiar with the proposed evaluation approaches, which appear complex. Others mention the amount of time necessary for engaging into these participatory processes, which is not always easy to accommodate with all parties involved. For AGROSAVIA some people consider that more training and support is needed to encourage impact-oriented focus: “we continue to do very basic and not applied research”, “expert staff in adoption and expected impacts of Technological Offers are scarce and do not allow all projects to be covered”, “there is a lack of training to include in the formulation of the projects how we can measure the impact of the possible technological offers that are generated”. Other difficulties lie in the interest of some researchers and the level of allocated resources, as example: “greater resources and infrastructure are required for impact research”. By contrast, other answers suggest the importance to have interdisciplinary teams and strengthening expert staff on impact issues: “projects should always be formulated by interdisciplinary teams”. On the side of Cirad, as transpires also from the interviews, we note the “importance of leaving this ambition as a dynamic work that is intellectually open and productive » (oral comm.), and the idea of “enlarging object of

studies (e.g., in *evaluation*) in order to enlarge the spectrum of agents and profiles feeling concerned by the culture of impact, and therefore being able to position themselves, within their field, in relation to other activities, and in terms of contribution to societal impacts» (oral comm.).

### 5. Discussion

The studied organisations have followed their own path in building a culture of impact. Embrapa developed and implemented multidimensional impact assessment methodologies since the last twenty-five years while Cirad started its journey fourteen years ago, with the design of specific participatory approaches for evaluating, planning, and reflecting with its partners on the impact of research activities. AGROSAVIA advanced, in the last seven years, in the institutionalisation of a global impact assessment strategy including systematic assessments and *ex ante* reflections. While the emergence of this culture is driven, in the three cases, by a desire to demonstrate usefulness of research and foster an organisational learning process (as shown by White et al. (2018)), these two factors unfolded differently in the cases and over time. Cirad chose to focus on a reflexive learning process to equip research teams to reflect, with partners and other actors, on the contribution of research activities to societal impacts, and the role and position of research. For Embrapa and AGROSAVIA, a strong driver to this culture is ‘Social Balance’ reporting in order to comprehensively communicate on socioeconomic benefits of the technologies and innovations developed through agricultural research. In relation to what Turner et al. (2022) propose, these two organisations seek primarily for ‘accountability’, to show both funders and society, and their supervising authorities, the contribution of science to society, while Cirad is rather guided by ‘analysis’, in an intention to understand effects of research and improve practices. Notably, these trajectories are anchored into different methodological choices, which are influenced by the organisational context, including its operational scope and configuration, and intervention modalities and philosophies. This very much echoes the findings of Klerkx et al. (2017) and Rijswijk et al. (2019) who highlighted how the institutional and economic/financial context may shape the culture and influence the deployment of new practices. For instance, given the diversity of settings within which Cirad operates, designing flexible, customized, and participatory reflexive approaches is important. The diversity of operating contexts being more geographically limited for AGROSAVIA and Embrapa, more standardised approaches may be suitable. Another contrasting element along the methodological approaches lies in the level of complexity that is aimed at being captured. Embrapa and AGROSAVIA tend to have a technological impact and adoption-oriented focus, in order to inform research development and transfer processes, whereas Cirad considers innovation process more widely, exploring its multiple dimensions and the changes its appropriation generates. This may again be influenced by their institutional settings: Cirad is organised in departments and units composed of various disciplines and addressing various types of innovations, while Embrapa and AGROSAVIA are organised along agricultural products and technology types, or regional development priorities, potentially leading to more focused inter-disciplinary exchanges and systemic designs.

We acknowledge several limitations to our study. First, our samples are heterogeneous across the organisations, which limits the extent to which they can be compared. Yet, this heterogeneity contributes to make this study unique, by depicting varying professional populations working on impact evaluation-related activities, in research organisations. Second, the sample may not be comprehensive in depicting culture of impact's trajectories and the veracity of effects on research practices. There are likely other cases that could add to the picture and would deserve considerations. Third, the “engaged-population” is supposedly familiar with impact-related principles and aware of their own role in the organisation, a potential selection bias. Thus, they may have implicitly implemented changes in their practices without reporting
them, leading to a possible “under-estimation” of the level of changes associated with a culture of impact. Fourth and similar to the limitations of *Rijswijk et al.* (2019), our study is not longitudinal, hence the difficulty to measure change in practices or in organisational identities. We relied on perceived changes of people in regard to their own experience or observations. Last, as also reflected by *Rijswijk et al.* (2019), it is sometimes difficult to disentangle the culture of impact and its associated changes, as the culture “is as much a consequence of change as it is a cause of other changes”. Further research could focus on monitoring the nature and degree of changes in research practices over time, as a result of this culture.

One important finding lies in the “infusion process” of the culture of impact across different layers of the organisation, pointing at the importance of looking at various organisational levels to understand how people may uptake and be willing to engage into “new” approaches (Klerkx et al., 2017). First and commonly to our cases, a central engine to this infusion is needed, which translates into dedication and specific staff to support and accompany others in performing impact evaluation related activities (reflections and applications), and keep up-to-date with literature development, in interaction with various levels of the organisation, including multiple levels of management. Indeed, supporting such organisational change implies specific expertise, including evaluation impact skills, communication skills, and ability to support others. This echoes the idea of “research impact practitioners” by Bayley and Phipps (2019) for fostering an “impact literacy” that can be gradually spread at individual and organisational levels, or the idea of “organisational legitimacy” around such a culture that imply new values to infuse through various levels of the organisation as well as broader institutional environments (Turner et al., 2023). “New professionalisms” to support this literacy have been long called for in agricultural research (Pretty and Chambers, 1993), also as a shift from accountability to learning of all stakeholders involved through evaluation (Douthwaite et al., 2017). Our findings show that the ‘impact teams’ of the studied organisations have a pivotal role in supporting the development of such “impact literacy”, and consist of unique entities in their composition (both research and support staff) and in their ability (or mission) to convey transversal organisational considerations and facilitate inter-disciplinary and inter-profession dialogues. Second, we find that the infusion process unfolds in different manners, suggesting different links between the organisational and the individual level strategies. Indeed, the evaluation applications vary from a strong level of formalisation - whereby technology impact assessments are integrated into the organisational performance assessment process, implying “mandatory” involvement of staff into evaluation activities -, to a weaker level along the belief that a deep and durable level of appropriation of evaluation principles is associated with a freedom of engaging, or not, with proposed approaches in the organisation (i.e., voluntariness principle). This difference seems to influence the degree and modalities of organisational capitalisation (i.e., the use of results from the evaluations into strategic planning and agenda priorities). A level of systematisation and standardisation in evaluation processes may be favorable to capitalisation of results for informing organisational strategic decisions (in line with Duffield and Whitty, 2016), by enabling to cover a wide and diverse range of objects considered for evaluations, and therefore produce a comprehensive organisational vision, at the difference of when the application of impact evaluation depends on what staff brings up as suggestion. Yet, a downside of standardised approaches may be the little room for flexibility and individual contributions to suit specific technologies or innovation processes. Clearly, the difficulty lies in finding the right balance between on the one hand strategic organisational-level drivers that promote systematisation and standardisation of impact evaluation approaches, and on the other hand suiting individual-level potential interests, specific innovations’ trajectories, and enabling tailored approaches, and thus facilitating internal learning processes. In fact, the “impact agenda” calls for both summative and formative functions of evaluation (Turner et al., 2022); this combination being,

moreover, conducive to the building of collective memories in the organisation (Ferré et al., 2022). Third and in regards to “what it takes” to build a culture of impact, our findings highlight the need for continuous investment and efforts by the various levels of the organisation. One could suggest that such effort may be higher in settings where evaluation activities are voluntary: as staff turns over, the level of efforts needs to be maintained to convince new recruits. Where activities are mandatory and follow routine procedures, the level of investment decreases as staff becomes acquainted with those.

No work had focused on examining the cognitive and practical changes that a culture of impact in agricultural research organisations translates within the staff exposed to evaluation-related activities. We find nuancing results. Cirad, Embrapa, and AGROSAVIA highlight significant gains in knowledge along evaluation principles, notions, and methods. With regard to capacities, those seem to be particularly enhanced when individuals participate to several experiences along *ex ante* or *ex post* evaluation approaches. Changes of actual practices are, in general, more incremental, with a smaller sample reporting changes on the way they interact with others, the way they design research questions, or plan interventions. Our findings clearly highlight that there is a learning process in place that generates first levels of changes at individual levels, yet demanding time (cf. Turner et al., 2023). Preskill and Boyle (2008) suggest that the organisation’s leadership, structures, and communication channels may influence the effectiveness of evaluation capacity building efforts. Interestingly, an indicator of the percolation of the culture of impact is the fact that what was initially perceived as an “extra layer” or an effort is generally being looked at as a new norm and a criterion embedded into the organisational landscape, and project-related practice. Furthermore, reported multi-faceted benefits of establishing a culture of impact consists of a unique opportunity to strengthen the community of researchers and practitioners, enlarge and reinforce partnerships, and to better argue why and what an organisation does. Thus, individual changes aligning with such a culture contribute to reinforce the organisations’ identity.

## 6. Conclusion and perspectives

Our study depicts the diverse shapes a “culture of impact” may take and the various manners it can be conceived, shaped, and implemented in agricultural research organisations. We considered three organisations having an ambition for establishing a culture of impact (Cirad, AGROSAVIA, Embrapa), and analysed the characteristics and trends relative to how they build and “practice” such a culture. We compared their respective historical trajectories in developing a culture of impact, their *modus operandi*, drivers, and the types of individual changes exposure to such a culture foster. Thus, we identified contextual circumstances and organisational elements that seem to favour or hinder the emergence of a culture of impact, as well as research practices that can promote its internalisation and consolidation.

This work enables to understand the multiple facets and implications of establishing a culture of impact in research organisations. We show that there is a variety of approaches that can fit different objects of evaluation, agents, and interests. Eventually, the uptake of the culture of impact is largely dependent on the way impact is perceived, evaluated, and discussed, and therefore on what the organisation and agents value. Embracing such a culture requires a level of openness to influence individual and collective practices, and a mutually shared and collective understanding of what this means and what it entails. This calls for consistency between organisational strategy, communication, visions, and planning, which then translate operationally through organisational mechanisms. Our study offers clues on opportunities to stimulate such a culture (e.g., identifying “champions”, recruiting staff having “change-oriented” mind-sets, focusing on soft skills). Yet, and just like for any culture, it remains a slow and long-term endeavour and the adhesion process is not straight forward. Particular attention is required on the integration of this culture with other organisational cultures, like the
“project” culture, which refers to tendency of research being organised around project modalities implying short-term cycles, compliance with funders’ requirements, and delivery of specific products. A culture of impact may fit with a project culture in the sense that it implies reflecting on specific impact contributions and the way that needs to be put into place in order to achieve it (through the so-called Theory of Change). In fact, for the case of Cirad that very much functions around the project modality, impact-related reflections are fostered by the need to responds to calls, attract funds, and design projects. Project’s dynamics may also act as a constraining frame for building a culture of impact as the latter requires thinking beyond the project’s goals and as part of a variety of research objects (e.g., partnerships, value chains). This tension between project culture and organisational culture deserves further exploration. Furthermore, as shown by Blundo-Canto et al. (2019) and White et al. (2018), learning is a key component of a culture of impact for encouraging continuous reflections on research practices and research positioning, enabling sharing of experiences and ideas at different organisational levels, feeding the organisational learning process at strategical level, but also fostering experimentations and interactions, and thus improving research practices contributing to societal impacts. This learning process is facilitated by accessibility to adequate tools, spaces, support staff, and a level of “decentralisation” through the role of intermediaries or “champions” that some staff may (consciously or not) endorse. Importantly, as transpiring here, culture of impact is an ever-evolving process: it is shaped by both people composing the culture and by the external context, therefore requiring permanent update along new opportunities and interests. Openness and adaptability of *modus operandi* are therefore key to adapt to future trajectories of the culture. This also calls for “pause and reflect” times allowing occasional assessment of evaluation capabilities in the organisation, perceptions towards this culture, possible resistances, and expectations, like this study does.

Building upon Blundo-Canto et al. (2019) and this analysis, we finally contribute to refine the definition of the “culture of impact” concept, as follows: an organisational aspiration to design, plan, and conduct research through the eyes of the types of impact it aims to contribute to, involving a conducive organisational environment (with tools, approaches, and communication spaces at various levels of the organisation, facilitating moments of reflections in the research process), and translating into the staff and partners feeling both consciously aligned within the research system and key to directing research activities in a way that contribute to the generation of socio-economic impacts. As part of this definition, we suggest that a culture of impact may contain different (sub)cultures with various dynamics covering different strands or organisational objectives, and allowing different paces. The co-existence of sub-cultures allows to match with and reach out to a larger scope of profiles, capacities, and interests, adjusting to their own practices and context of action (Blundo-Canto et al., 2019).

The development of a culture of impact aligns with recent agenda in agricultural research organisations that call for systemic innovation, and rethinking of organisations' function and responsibilities for enabling a shift towards more sustainable trajectories of development and addressing wicked and complex issues (Conti et al., 2024), considering a context of ecological and societal emergency too. This study is particularly useful for scientists and managers in research organisations who aim to integrate the evaluation on societal impacts within their practices and develop strategies to critically reflect on the contribution of research to societal impacts. Practical guidance on building a culture of impact could be stated as follows: i) exploring context-specific issues and path-dependencies in order to prepare for challenges and opportunities to the development of such a culture, ii) encouraging continuous inter- and intra-organisational dialogues and design of “standard routine procedures” for facilitating staff engagement and structuring a learning process; iii) engaging with the community at large, including researchers, technicians, engineers, supporting staff, who all have a role within innovation processes. Thus, adhesion of this diversity of profiles and

professions to this culture, in their own way, is key to foster effective changes of postures, and may be facilitated by offering a diversity of evaluation approaches able to suit various needs and evaluative capacities; and iv) ensuring a set of structural and variable elements, including dynamic human resources to propel it over time, as well as a combination of both voluntary and systematic/mandatory elements (cf. bottom-up and top-down dynamics as formulated by White et al., 2018).

## CRediT authorship contribution statement

**Marie Ferré:** Writing – review & editing, Writing – original draft, Visualization, Validation, Supervision, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization. **Genowefa Blundo-Canto:** Writing – review & editing, Writing – original draft, Supervision, Methodology, Funding acquisition, Formal analysis, Conceptualization. **Geraldo Stachetti Rodrigues:** Writing – review & editing, Validation, Supervision, Methodology, Investigation, Funding acquisition, Conceptualization. **María-Margarita Ramírez-Gómez:** Writing – review & editing, Validation, Supervision, Methodology, Investigation, Funding acquisition, Conceptualization. **Graciela Luzia Vedovoto:** Writing – review & editing, Methodology, Investigation, Formal analysis. **Roberto Manolio Valladão Flores:** Writing – review & editing, Supervision, Methodology, Investigation, Funding acquisition, Formal analysis. **Gonzalo-Alfredo Rodríguez-Borray:** Writing – review & editing, Validation, Supervision, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization. **Mirian Oliveira de Souza:** Writing – review & editing, Methodology, Investigation, Formal analysis. **Frédéric Goulet:** Writing – review & editing, Methodology, Investigation, Formal analysis, Conceptualization. **Ángela-Rocío Vásquez-Urriago:** Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization. **Juliana-Ivonne Sánchez-Lozano:** Writing – review & editing, Methodology, Investigation, Formal analysis. **Daniela Maciel Pinto:** Methodology, Investigation, Formal analysis. **Gregorio-Salomón Zambrano-Moreno:** Writing – review & editing, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. **María-Aidé Londoño-Arias:** Writing – review & editing, Methodology, Investigation, Formal analysis, Conceptualization. **Cristóbal-Alfonso Zapata-Tamayo:** Supervision, Funding acquisition, Formal analysis. **Aurelle de Romémont:** Writing – review & editing, Supervision, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgements

We acknowledge the support of the partner organisations (Cirad, Embrapa, AGROSAVIA) for covering the research time invested in this project, and the support from Agropolis Foundation in the frame of the project CIAR (project number 2002-239) for making the organisation of a workshop in Bogota-Colombia possible. This work was publicly funded through ANR (the French National Research Agency) under the "Investissements d'avenir" programme with the reference ANR-10-LABX-001-01 Labex Agro and coordinated by Agropolis Foundation (project number 2002-239). We thank all the participants to the interviews, surveys, and workshop activities, for their precious time and inputs. We thank Karine Nunes and Berline Africa Maagoum Soh for their earlier contributions to the work, and Selim Louafi for a final read.
We also thank three anonymous reviewers for valuable comments that
helped to refine the manuscript.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi.
org/10.1016/j.respol.2024.105140.

Data availability

Data will be made available on request.

References

Abdi, Kambiz, Senin, Aslan Amat, 2015. Empirical study on the effect of organizational culture on organization innovation. Asian Soc. Sci. 11, 114-128.

AGROSAVIA 2022: file:///C:/Users/mferre/Downloads/Ver Documento_38187.pdf
accessed 24.11.2024.

AGROSAVIA, 2023. Tejer impacto. Metodología para la evaluación ex ante de impacto de proyectos de investigación y desarrollo del sector agropecuario. Corporación Colombiana de Investigación Agropecuaria, Mosquera, Colombia.

Akhavan, P., Sanjaghi, M.E., Ojaghi, H., 2014. Examining the relationships between organizational culture, knowledge management and environmental responsiveness capability. Eur. J. Mark. 44 (2). https://doi.org/10.1108/VINE-07-2012-0026.

Akhtar, S., Arif, A., Rubi, E., Naveed, S., 2011. Impact of institutional learning on institutional performance: study of higher education institutes. Int. J. Acad. Res. 3 (5), 6.

Alegre, J., Chiva, R., 2008. Assessing the impact of organizational learning capability on product innovation performance: an empirical test. Technovation 28 (6), 315-326.

Alsabbagh, M., Khalil, A.H.A., 2017. The impact of institutional culture on institutional learning (an empirical study on the education sector in Damascus City). International Journal of Academic Research in Business and Social Sciences 7 (4), 579-600. https://doi.org/10.6007/IJARSSv7-i4/2834.

Avila, A.F.D., Rodrigues, G.S., Vedovoto, G. 2008. Avaliação dos Impactos de Tecnologias Geradas pela Embrapa: metodologia de referência. 1. ed. Brasília: Embrapa, 2008. v. 1. 189p. https://ainfo.cnpnia.embrapa.br/digital/bitstream/item/132174/1/MetodologiaReferenciaAvalImpactoEmbrapa.pdf.

Avila, A.F.D., Rodrigues, G.S., Vedovoto, G.L., Pentead, F.R., Fonseca, W.C. 2015. Embrapa's experience on the impact assessment of agricultural R&D: 15 years using a multidimensional approach. In: IMPAR CONFERENCE, 2015. Paris. Impacts of agricultural research: towards an approach of societal values: [Proceedings...]. Paris: INRA, 2015. 25.

Barret, D., Blundo-Canto, G., Dabat, M.H., Devaux-Spatarakis, A., Faure, G., Hainzelin, E., Mathé, S., Temple, L., Toillier, A., Triomphe, B., 2017. Guide méthodologique ImpresS: évaluation ex post des impacts de la recherche agronomique dans les pays du Sud. CIRAD, Montpellier. https://doi.org/10.19182/agritrop/00005 (96 pp.).

Bayley, J.E., Phipps, D., 2019. Building the concept of research impact literacy. Evidence & Policy 15 (4), 597-606. Retrieved Feb 8, 2024, from. https://doi.org/10.1332/174426417X15034894876108.

Bin, A., Gianonni, C., Mendes, P.J.V., Rio, C., Salles Filho, S.L.M., Bezerra, L.C., 2013. Organisation of research and innovation: a comparative study of public agricultural research institutions. J. Technol. Manag. Innov. 8 (95-96), 2013. https://doi.org/10.4067/S0718-27242013000300048.

Blundo-Canto, G., Triomphe, B., Faure, G., Barret, D., de Romémont, A., Hainzelin, E., 2019. Building a culture of impact in an international agricultural research organisation: process and reflective learning. Research Evaluation 28 (2), 136-144.

Blundo-Canto, G., Devaux-Spatarakis, A., Mathé, S., Faure, G., Cerdan, C., 2020a. Using a participatory theory driven evaluation approach to identify causal mechanisms in innovation processes. N. Dir. Eval. 167, 59-72.

Blundo-Canto, G., de Romémont, A., Hainzelin, É., Faure, G., Monier, C., Triomphe, B., Barret, D., Vall, É., 2020b. ImpresS ex ante. An approach for building ex ante impact pathways in development-oriented research. In: ImpresS ex Ante Methodological Guide (Second Version). CIRAD, Montpellier (74 pp.).

Bozeman, B., Youtie, J., 2017. Socio-economic impacts and public value of government-funded research: lessons from four US National Science Foundation initiatives. Research Policy 46 (8), 1387-1398. https://doi.org/10.1016/j.respol.2017.06.003.

Calvert, J., 2006. What's special about basic research? Sci. Technol. Hum. Values 31 (2), 199-220. © 2006 Sage Publications. https://doi.org/10.1177/0162243905283642.

Cayla, D., 2007. Organizational Learning: Adaptive Process or Managed Change Process?

Chiva, R., Alegre, J., Lapiedra, R., 2007. Measuring organisational learning capability among the workforce. Int. J. Manpow. 28 (3/4), 224-242. https://doi.org/10.1108/01437720107155227.

Choi, M., Ruona, W.E.A., 2011. Individual readiness for organisational change and its implications for human resource and organisation development. Hum. Resour. Dev. Rev. 10 (1), 46-73. https://doi.org/10.1177/1534484310384957.

Cirad 2017. Vision stratégique et ambitions du Cirad 2018-2028: Strategic vision and ambitions Cirad 2018-2028. https://www.cirad.fr/Media/espace-docutheque/docutheque/fichiers/vision-strategique accessed 31.08.2023.

Cirad 2018a. Le Cirad se mobilise pour déployer la culture de l'impact https://intranet.cirad.fr/les-rubriques-du-portail/les-actualites/2018/actus/impress.-le-cirad-se-mobilise-pour-deployer-la-culture-de-l-impact accessed 06.02.2023.

Cirad 2018b. Strategic Scientific and Partnership Objectives (OSSP) : Objectifs de stratégie scientifique et partenariale 2019-2023 [OSSP2] https://www.cirad.fr/view pdf/4162; https://www.cirad.fr/en/Media/espace-docutheque/docuthe que/fichiers/ossp2 accessed 31.08.2023.

Conti, C., Hall, A., Percy, H., Stone-Jovicich, S., Turner, J., McMillan, L., 2024. What does the agri-food systems transformation agenda mean for agricultural research organisations? Exploring organisational prototypes for uncertain futures. Glob. Food Sec. 40, 100733.

Cooper, S., 2014. Transformative evaluation: organisational learning through participative practice. The Learning Organisation 21 (2), 146-157.

Douthwaite, B., Hofferger, E., 2017. Towards a complexity-aware theory of change for participatory research programs working within agricultural innovation systems. Agr. Syst. 155, 88-102.

Douthwaite, B., Apgar, J.M., Schwarz, A.M., Attwood, S., Senaratna Sellamuttu, S., Clayton, T., 2017. A new professionalism for agricultural research for development. Int. J. Agric. Sustain. 15 (3), 238-252. https://doi.org/10.1080/14735903.2017.1314754.

Duffield, S., Whitty, J., 2016. Application of the systemic lessons learned knowledge model for organisational learning through projects. Int. J. Proj. Manag. 34, 1280. https://doi.org/10.1016/j.ijproman.2016.07.001.

EMBRAPA. 2018a. Workshop internacional promove interação entre especialistas em avaliação de impactos. https://www.embrapa.br/busca-de-noticias/-/noticia/79058 949/workshop-internacional-promove-interacao-entre-especialistas-em-avaliacao -de-impactos.

EMBRAPA. 2018b. Seminário internacional discute avaliação de impactos. https://www. embrapa.br/busca-de-noticias/-/noticia/79058753/seminario-discute-avaliaca o-de-impactos-e-traz-exemplos-do-cirad-e-da-embrapa?p_auth=ArWo4Uss.

EMBRAPA, 2020. VII Plano Diretor da Embrapa 2020-2030. Brasília, DF (31 pp. il. color).

EMBRAPA, 2023. Balanço social 2022. In: Superintendência de Comunicação, Superintendência de Estratégia, 26. ed. Embrapa, Brasília, DF. (67 pp.: il. Color). https://www.embrapa.br/balanco-social-2022.

Faure, G., Barret, D., Blundo-Canto, G., Dabat, M.H., Devaux-Spatarakis, A., Le Guerroué, J.L., Marquie, C., Mathé, S., Temple, L., Toillier, A., Triomphe, B., Hainzelin, E., 2018. How different agricultural re-search models contribute to impacts: evidence from 13 case studies in developing countries. Agr. Syst. 165, 128-136.

Faure, G., Blundo-Canto, G., Dabat, M.H., Devaux-Spatarakis, A., Le Guerroué, J.L., Mathé, S., Temple, L., Toillier, A., Triomphe, B., Hainzelin, E., 2020. A participatory method to assess the contribution of agricultural research to societal changes in developing countries. Research Evaluation 29 (2), 158-170.

Ferré, M., Blundo Canto, G., de Romémont, A., Maagoum Soh, B.A., Binot, A., 2022. How can the past inform and benefit research-for-development interventions? The value of reflexive and participatory approaches. In: Anthropologie & Développement 53, 155-174. https://doi.org/10.4000/anthropodev.1682.

Hainzelin, E., Barret, D., Faure, G., collectif ImpresS., 2016. Agriculture research in developing countries: from a “culture of promise” to a “culture of impact”. Policy Brief. https://agritrop.cirad.fr/583241/1/Impress%20Policy%20Brief.pdf (accessed 02.02.2024).

Hainzelin, E., Barret, D., Faure, G., Dabat, M.-H., Triomphe, B., 2017. Agricultural research in the Global South: steering research beyond impact promises. Perspective 42, 1-4. https://doi.org/10.19182/agritrop/00009.

Jerez-Gómez, P., Céspedes-Lorente, J., Valle-Cabrera, R., 2005. Institutional learning capability: a proposal of measurement. J. Bus. Res. 58 (6), 715-725. https://doi.org/10.1016/j.jbusres.2003.11.002.

Joly, P.-B., Matt, M., 2022. Towards a new generation of research impact assessment approaches. J. Technol. Transf. 47, 621-631. https://doi.org/10.1007/s10961-017-9601-0.

Joly, P.-B., Colinet, L., Gaunand, A., Lemarié, S., Matt, M., 2016. Agricultural research impact assessment: issues, methods and challenges. OECD Food, Agriculture and Fisheries Papers 98. https://doi.org/10.1787/5339e165-en.

Jyothibasu, C., Pradhan, B.B., Farooq, A., 2012. The learning entities in organisational learning – an empirical study. Int. J. Learn. Intellect. Cap. 9 (1/2), 211-225.

Klerkx, L., Seuneke, P., de Wolf, P., Rossing, W.A.H., 2017. Replication and translation of co-innovation: the influence of institutional context in large international participatory research projects. Land Use Policy 61, 276-292.

Kok, K.P.W., den Boer, A.C.L., Cesuroglu, T., van der Meij, M.G., de Wildt-Liesveld, R., Regeer, B.J., Broerse, J.E.W., 2019. Transforming research and innovation for sustainable food systems—a coupled-systems perspective. Sustainability 11, 7176.

Leeuwis, C., Klerkx, L., Schut, M., 2018. Reforming the research policy and impact culture in the CGIAR: integrating science and systemic capacity development. Glob. Food Sec. 16, 17-21.

Liao, S.-H., Wu, C., 2010. System perspective of knowledge management, institutional learning, and institutional innovation. Expert Syst. Appl. 37 (2), 1096-1103. https://doi.org/10.1016/j.eswa.2009.06.109.

Maru, Y., Sparrow, A., Stirzaker, R., Davies, J., 2018a. Integrated agricultural research for development (IAR4D) from a theory of change perspective. Agr. Syst. 165, 310-320.

Maru, Y.T., Sparrow, A., Butler, J.R.A., Banerjee, O., Ison, R., Hall, A., Carberry, P., 2018b. Towards appropriate mainstreaming of “Theory of Change” approaches into agricultural research for development: challenges and opportunities. Agr. Syst. 165, 344-353.
Matt, M., Gaunand, A., Joly, P.-B., Colinet, L., 2017. Opening the black box of impact – ideal-type impact pathways in a public agricultural research organisation. Res. Policy 46 (1), 207–218.

Mayne, J. 2008. Building an evaluative culture for effective evaluation and results management. ILAC Brief No. 20. Rome: Institutional Learning and Change (ILAC) Initiative.

Mayne, J., Johnson, N., 2015. Using theories of change in the CGIAR Research Program on Agriculture for Nutrition and Health. Evaluation 21, 407–428.

McKay, K., Kuntz, J.R.C., Näswall, K., 2013. The effect of affective commitment, communication and participation on resistance to change: the role of change readiness. N. Z. J. Psychol. 42 (1).

Mintzberg, H., Westley, F., 1992. Cycles of organizational change. Strateg. Manag. J. 13, 39–59. Special Issue: Fundamental Themes in Strategy Process Research. Published by: John Wiley & Sons Stable URL: http://www.jstor.org/stable/2486365.

Muscalu, E., 2014. Organizational culture change in the organisation. https://www.armyacademy.ro/reviste/rev4_2014/MUSCALU.pdf. (Accessed 23 October 2024).

Omotayo, F.O., 2015. Knowledge Management as an Important Tool in Institutional Management: A Review of Literature. Library Philosophy and Practice (e-journal).

Preskill, H., Boyle, S., 2008. Insights into evaluation capacity building: motivations, strategies, outcomes, and lessons learned. Can. J. Program Eval. 23 (3), 147–174. https://doi.org/10.3138/cjpe.0023.008.

Pretty, J.N., Chambers, R., 1993. Towards a Learning Paradigm: New Professionalism and Institutions for Agriculture. Discussion paper – Institute of Development Studies, University of Sussex.

Rebelo, T., Gomes, A.D., 2007. Organizational learning and the learning organization: reviewing evolution for prospecting the future. Learn. Organ. 15 (2008), 294–308.

Reed, M.S., Ferré, M., Martin-Ortega, J., Blanche, R., Dallimer, M., Holden, J., 2021. Evidencing impact from research: a typology of methods. Research Policy 50 (4).

Reed, M.A., Gent, S., Seballos, F., Glass, J., Hansda, R., Fischer-Møller, M., 2022. How can impact strategies be developed that better support universities to address twenty-first-century challenges? Research for All 6 (1), 24. https://doi.org/10.14324/RFA.06.1.24.

Rijswijk, K., Klerkx, L., Turner, J.A., 2019. Enacting digitalisation in AKIS: how New Zealand agricultural knowledge providers understand and respond to digital agriculture. NJAS - Wageningen Journal of Life Sciences 90-91, 100313.

Rodrigues, G.S., Campanhola, C., Kitamura, P.C., 2003. An environmental impact assessment system for agricultural R&D. Environ. Impact Assess. Rev. 23 (2), 219–244.

Rodrigues, G.S., Buschinelli, C.C. de A, Avila, A.F.D. 2010. An environmental impact assessment system for agricultural research and development II: institutional learning experience at Embrapa. Journal of Technology Management & Innovation 5 (4): 38–56, 2010.

Saari, E., Kallio, K., 2010. Developmental impact evaluation for facilitating learning in innovation networks. Am. J. Eval. v. 32, n. 2.

Schein, E.H., 1990. Organizational culture: what it is and how to change it. In: Evans, P., Doz, Y., Laurent, A. (Eds.), Human Resource Management in International Firms. Palgrave Macmillan, London. https://doi.org/10.1007/978-1-349-11255-5_4.

Schmid, O., Quiédeville, S., Sterly, S., Barjolle, D., Midmore, P., 2016. Research Brief: Analysis of Impact Pathways of Research on Agriculture. https://doi.org/10.13140/RG.2.2.21000.52484.

Schot, J., Steinmueller, W.E., 2018. Three frames for innovation policy: R&D, systems of innovation and transformative change. Research Policy 47, 1554–1567.

Seyedyousefi, N., Hosseini Fard, S.M., Tohidi, F., 2016. The role of institutional culture in knowledge management. Mediterr. J. Soc. Sci. https://doi.org/10.5901/mjss.2016.v7n5p412.

Spaapen, J.B., 2015. A new evaluation culture is inevitable. Organic Farming 1 (1), 36–37. https://doi.org/10.12924/of2015.01010036.

Stone-Jovicich, S., Percy, H., McMillan, L., Turner, J.A., Chen, L., White, T., 2019. Evaluating monitoring, evaluation and learning initiatives in the New Zealand and Australian agricultural research and innovation systems: the MEL2 framework. Evaluation Journal of Australasia 19 (1), 8–21. https://doi.org/10.1177/1035719X18823567.

Temple, L., Barret, D., Blundo Canto, G., Dabat, M.-H., Devaux-Spatarakis, A., Faure, G., Hainzelin, E., Mathé, S., Toillier, A., Triomphe, B., 2018. Assessing impacts of agricultural research for development: A systemic model focusing on outcomes. Research Evaluation 27 (2), 157–170. https://doi.org/10.1093/reseval/rvy005.

Templeton, G., Bruce, L., Charles, S., 2002. Development of a measure for the organizational learning construct. J. Manag. Inf. Syst. 19, 175–218.

Turner, J.A., Guesmi, B., Gil, J.M., Heanue, K., Sierra, M., Percy, H., Bortagaray, I., Chams, N., Milne, C., 2022. Evaluation capacity building in response to the agricultural research impact agenda: emerging insights from Ireland, Catalonia (Spain), New Zealand, and Uruguay. Eval. Program Plann. 94 (102127), 1–14. https://www.sciencedirect.com/science/article/abs/pii/S0149718922000817.

Turner, J.A., Percy, H., Hall, A., Klerkx, L., 2023. Re-orientating agricultural research to address complex challenges: legitimacy dynamics of a hybrid research organisation. J. Rural. Stud. 103, 103137.

Vedovoto, G.L., Marques, D.V., Valente, L.C.M., Rodrigues, G.S. 2022. Avaliação de impactos de tecnologias agropecuárias na Embrapa: dimensão Desenvolvimento Institucional. Jaguariúna, SP: Embrapa Meio Ambiente, 2022. 22p. (Embrapa Meio Ambiente. Documentos, 1516-4691, 136). https://ainfo.cntcia.embrapa.br/digital/bitstream/doc/1147435/1/Rodrigues-Avaliacao-impactos-2022.pdf.

Watts, J., Mackay, R., Horton, D., Hall, A., Douthwaite, B., Chambers, R., Acosta, A. 2007. Institutional Learning and Change: An Introduction. 19.

Watts, J., Horton, D., Douthwaite, B., La Rovere, R., Thiele, G., Prasad, S., Staver, C., 2008. Transforming impact assessment: beginning the quiet revolution of institutional learning and change. Exp. Agric. 44 (1), 21–35.

Weber, K.M., Rohracher, H., 2012. Legitimizing research, technology and innovation policies for transformative change: combining insights from innovation systems and multi-level perspective in a comprehensive ‘failures’ framework. Res. Policy 41 (6), 1037–1047.

Weißhuhn, P., Helming, K., Ferretti, J., 2018. Research impact assessment in agriculture - A review of approaches and impact areas. Research Evaluation 27, 36–42.

White, T., Percy, H., Small, B., 2018. Creating an evaluation culture through capacity building: a new frontier in a science organisation. In: Evaluation Matters—He Take To Te Aromatawai, 4. 2018 © New Zealand Council for Educational Research. https://doi.org/10.18296/em.0031, 2018. http://www.nzcer.org.nz/nzcerpress/evaluation-matters.

Wolf, B.M., Häring, A.-M., Hess, J., 2015. Strategies towards evaluation beyond scientific impact. Pathways not only for agricultural research. Organic Farming 1, 3–18.

Zhou, W., Hu, H., Shi, X., 2015. Does organizational learning lead to higher firm performance? An investigation of Chinese listing companies. Learn. Organ. Download 157 DOWNLOADS Since December 18, 2020 PlumX Metrics Included in Business Administration, Management, and Operations Commons Share Facebook LinkedIn WhatsApp Email Partager.