# Data-driven R&D&I management for societal impacts: Introduction and application of agroRadarEval

Daniela Maciel Pinto, Adriana Bin, Marie Ferre, James A. Turner, Géraldo

Stachetti Rodrigues, María Magdalena Costa, Miguel Sierra Pereiro, Juan

Mechelk, Aurelle de Romemont, Kevin Heanue

<figure><img src="image_1.png" /></figure>

## To cite this version:

Daniela Maciel Pinto, Adriana Bin, Marie Ferre, James A. Turner, Géraldo Stachetti Rodrigues, et al. Data-driven R&D&I management for societal impacts: Introduction and application of agroRadarEval. Journal of Technology Management and Innovation, 2024, 19 (4), pp.89-112. 10.4067/S0718-27242024000400089. hal-05183140

HAL Id: hal-05183140

https://hal.science/hal-05183140v1

Submitted on 23 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Data-Driven R&D&I Management for Societal Impacts: Introduction and Application of AgroRadarEval

Daniela Maciel Pinto¹* ID, Adriana Bin² ID, Marie Ferré³ ID, James A Turner⁴ ID,
Geraldo Stachetti Rodrigues⁵ ID, María Magdalena Costa⁶ ID, Miguel Sierra Pereiro⁶ ID, Juan Mechelk⁶ ID,
Aurelle de Romemont⁷, Kevin Heanue⁸ ID

**Abstract:** Recognizing evaluation results as a crucial source of information to support RD&I management, this article introduces 'AgroRadarEval', an interactive tool aimed at fulfilling theoretical, conceptual, and practical gaps concerning the systematization of the use of evaluation results in agricultural RD&I. Aligned with the principles of Responsible Research and Innovation (RRI) and Responsible Research Assessment (RRA), AgroRadarEval aims to support leaders and managers of RD&I in reflecting on the strengths and weaknesses of organizational capacities, culture, collaborations, processes, and communications that underlie the use of evaluation results in agricultural RD&I. AgroRadarEval is built along three support pillars: Evaluation Capacity Building, Impact-Oriented Evaluation Culture, and Reflective Learning, and is operationalized through eight interconnected dimensions: 1. participation and collaboration, 2. skills development, 3. promotion of an evaluation culture, 4. continuous feedback and adaptation, 5. integration with strategic planning, 6. monitoring, 7. influences of the external environment, and 8. communication. This study describes the development of the tool, its characteristics, illustrating its application in an agricultural RD&I organization. The study is targeted at leaders and managers of agricultural RD&I, evaluators, and researchers interested in research evaluation and enhancing the impact of RD&I.

**Keywords:** Use of evaluations results; Agricultural R&D management; Research Impact; Organizational management; Impact Assessment.

Submitted: October 21, 2024 / Approved: December 16, 2024

## 1. Introduction

Expectations for research, particularly that carried out by public institutions, have grown, especially regarding their capacity to generate innovations and promote socially relevant changes (Reed et al., 2021, 2022; Spaapen, 2015). This reflects a growing concern with social responsibility, ethics, and transparency in the use of public resources for science and technology (S&T) and research, development, and innovation (RD&I) activities, marking a paradigm shift that encourages the adoption of the principles of Responsible Research and Innovation (RRI) and Responsible Research Assessment (RRA) (Curry et al., 2022; Schuijff & Dijkstra, 2020). The principles of RRI aim to ensure that RD&I processes are conducted ethically, inclusively, and in a way that benefits society, encouraging reflection on economic, social, and environmental impacts (Felt, 2018). Complementarily, RRA focuses on ensuring that research evaluations adopt appropriate methods and diverse indicators, integrating the results into research agendas to guide RD&I strategies. Together, RRI and RRA create a framework where research processes and their evaluation work synergistically to ensure accountability, societal alignment, and transformative impacts.

In light of this, impact evaluation emerges as an indispensable tool that embodies the principles of RRI and RRA by providing evidence of both the actual and potential transformations brought about by research. It measures and demonstrates whether expectations have been met or exceeded, while promoting accountability, inclusivity, and alignment with societal needs (Carley & Bustelo, 2019; Reinertsen et al., 2022). Moreover, this evaluation provides a snapshot of the real effects of interventions, functioning as an information source that allows for strategic adjustments to ensure that research remains aligned with social demands (Lee et al., 2020). Thus, it can serve both to measure the generated effects and to support organisational planning and learning, indicating opportunities to optimise future results based on present evidence (Mackay & Horton, 2003).

In the context of agricultural research, maximising the use of these evaluation results is particularly relevant. In this field, impact evaluation has a long history, dating back to the 19th century (Evenson et al., 1979) and being formalised in the 1950s (Colinet, 2021). Originally focused on the economic impacts of research (Alston et al., 1995; Evenson, 2001), these evaluations have come to include environmental, social, cultural, and political impacts, thereby increasing their informational value for RD&I management (Horton & Mackay, 2003).

<footnote>(1) Embrapa Territorial. Scientific and Technological Policy (PPG-PCT), University of Campinas, Brazil</footnote>

<footnote>(2) School of Applied Sciences, University of Campinas, Brazil</footnote>

<footnote>(3) French Agricultural Research Centre for International Development (Cirad), Montpellier, France.</footnote>

<footnote>(4) Ethical Agriculture, AgResearch Ltd, New Zealand</footnote>

<footnote>(5) Embrapa Meio Ambiente, SP, Brazil</footnote>

<footnote>(6) Instituto Nacional de Investigación Agropecuaria (INIA), Parque Tecnológico LATU, Montevideo, Uruguay</footnote>

<footnote>(7) French Agricultural Research Centre for International Development (Cirad), Paris, France</footnote>

<footnote>(8) Evaluation Unit, Teagasc Mellows Campus Athenry, Co. Galway, Ireland</footnote>

<footnote>*Corresponding author: daniela.maciel@embrapa.br</footnote>
Despite this evolution, the use of evaluation results by agricultural RD&I institutions remains limited, more often directed at accountability than at organisational learning and strategic management (Joly et al., 2016). This limitation highlights the absence of practical frameworks that can operationalise RRI and RRA principles in agriculture, guiding organisations in leveraging evaluation evidence to address societal challenges. In part, one reason for this limitation is that the discussion on the use of results in the field of research evaluation is relatively recent (Milzow et al., 2019; Van der Most, 2010). While the topic has been addressed in programme and public policy evaluation since the 1960s, with theoretical, conceptual, and practical contributions (Alkin & Christie, 2004, 2023), in research evaluation there is a clear lack of systematised and structured approaches. In light of this, Lee (2010) emphasises the urgency of developing systematic frameworks to expand the use of evaluation results in research, which is corroborated by Pinto and Bin (2024).

With a focus on filling this gap, this study presents AgroRadarEval, a data-driven tool that directly operationalises the principles of RRI and RRA. Designed to support the management and leadership of agricultural RD&I organisations, AgroRadarEval enables the systematic use of evaluation results to align research with societal demands, fostering ethical, inclusive, and impactful practices. Considering the principles of RRI and RRA, the tool is grounded in theoretical and conceptual pillars and operationalised through dimensions and questions to be considered at the organisational level, with the aim of informing RD&I strategies and promoting research that is more responsive to societal demands, based on evaluation evidence. The question that guided the development of AgroRadarEval was: "How can the use of impact evaluation results be systematised to maximise the social impact of agricultural research?"

## 2. Theoretical Background

### State of the Art on the Use of Evaluation Results

This section highlights briefly how discussions on the use of evaluation results are organized in the fields of program and policy evaluation, research evaluation, and agricultural research evaluation, and how they are associated with current debates on impact-driven and socially responsive research, endorsed by the RRI and RRA movements.

Alkin and Christie (2004, 2023) organized the field of program and public policy evaluation into three main branches<fnref n="1" />, with the “Branch of Use” focusing on the debate surrounding the use of evaluation results. This structure goes beyond theoretical discussions, encompassing practical frameworks as well. For example, Stufflebeam’s Context, Input Process Product (CIPP) model (Stufflebeam, 1983, 2002) provides a comprehensive approach to use-oriented evaluations, while Preskill and Boyle (2008) propose Evaluation Capacity Building (ECB) with a strong emphasis on use. ECB refers to the deliberate choice, by an organization, to develop, maintain, and enhance evaluation competencies (Bourgeois & Cousins, 2013; Cousins et al., 2009).

In this branch, both theoretical and practical developments highlight the need for guidelines that promote feasible evaluations, and the capacity to assess an action (Rutman & Wholey, 1980) to support decision-making by program managers and policymakers. Patton’s “Utilization-Focused Evaluation (UFE)” expands the perspective on use-oriented evaluations, highlighting that they are not solely intended for policymakers but also for any potential user, positioning the evaluator as a strategic actor in the evaluation process. However, this approach is questioned by Alkin (Alkin et al., 1979) and Weiss (1972, 1998) who argue that the evaluator should not have the role of judging the results data.

On the other hand, Cousins (2014) advocates for a participatory approach at all levels of evaluation to ensure an effective connection between evaluation results and their users. At the same time, these authors and others in the field of program and policy evaluation explore evaluation results in terms of organizational learning. Weiss and Chelimsky, particularly, demonstrate the importance of evaluation results for the continuous improvement of organizational practices and for strategic decision-making (Chelimsky, 1977, 2015; Cousins & Leithwood, 1986; Weiss, 1979).

The ‘Branch of Use’ is wide and encompasses aspects related to the quality of the evaluation, including the choice of the methods and the evaluator’s capacity to implement them effectively. It also includes stakeholder involvement and the organizational capacity to conduct impact evaluations, considering various levels of the organization that may be involved in the process. This is particularly related to “ECB, with models such as those by Preskill and Boyle (2008, 2009) and Labin et al. (2012), which seek to support organizations in strengthening their evaluation capacity, ensuring that evaluation results are used to maximize the positive impact on the evaluated programs and policies. However, the ‘Branch of Use’ is fundamentally rooted in program and policy evaluation. In the field of research evaluation, as explored in the following sections, there is a need to advance discussions on the use of evaluation results by establishing elements that can foster this type of context.

### Use of Results from Research Impact Evaluation

In RD&I evaluation, an evolution in the use of results can be observed in the last years (Pinto and Bin, 2024), shifting away from a focus on accountability, to align with the principles of RRI and RRA. Beginning in the 1990s with theoretical and empirical investigations focused on the relationship between funding and accountability in RD&I (Hemlin, 1996; Horton, 1998; Luukkonen, 1995), the number of these types of studies increased from the 2000s onwards.

During this period, studies started to investigate the use of results as an element for organizational learning, with notable work related to agricultural RD&I, specifically linked to the Consultative Group on International Agricultural Research (CGIAR) (Horton & Mackay, 2003; Mackay & Horton, 2003; Patton & Horton, 2009). From 2010 onwards,

<footnote n="1">¹The branches organized by Alkin and Christie are: Branch of Methods, Branch of Values, and Branch of Use.</footnote>
discussions about the use of evaluation results in the field of agriculture research began to decrease, with the last mapped work by Joly et al. (2016), who did not specifically investigate the use, but explore case studies of five agricultural research organizations that showed that accountability and advocacy are the most frequent types of use of evaluation results regarding agricultural RD&I.

Yet, other types of organizations, such as funding agencies (Milzow et al., 2019; Reinhardt & Milzow, 2012) and organizations involved in academic research (Grant & Hinrichs, 2015; Kamenetzky et al., 2016; Reinertsen et al., 2022), have began to explore this topic more deeply. Among the discussions on academic research evaluation, the use of impact evaluation results as part of systems like the Research Excellence Framework (REF) plays a central role in guiding responsive research aimed at maximizing societal impact (Morgan et al., 2017).

Considering this context, Morgan et al. (2017) and Parks et al. (2019) identify the 'As of evaluation - analysis, advocacy, allocation, accountability, acclamation, and adaptation - as crucial for diversifying and adapting evaluation practices, increasing the value and relevance of research. This approach is closely related to the principles of RRI and RRA, with Wilford et al. (2016) highlighting that the use of evaluation results can enhance the social impact of research by incorporating anticipation, inclusion, reflexivity, and responsiveness.

The principles of RRI and RRA (Table 1) reflect aspects of evaluation theory and practice, such as the importance of creating, maintaining, and developing evaluative capacities (ECB), including stakeholders in RD&I actions and its evaluation, making results transparent and accessible, and communicating about them, which are identified as mechanisms to foster the use of evaluation results in research (Reinhardt & Milzow, 2012).

<table><caption>**Table 1.** Principles from RRI and RRA.</caption><thead><tr><td>Principle</td><td>RRI</td><td>RRA</td></tr></thead><tbody><tr><td>Ethics and Integrity</td><td>Prioritizes ethics, integrity, and social inclusion for valuable research.</td><td>Emphasizes ethics in evaluation, ensuring integrity.</td></tr><tr><td>Transparency</td><td>Promotes openness and transparency in research, based on concepts like Open Access and Open Science.</td><td>Ensures transparency of data and criteria.</td></tr><tr><td>Engagement</td><td>Encourages active participation of all stakeholders.</td><td>Values collaboration among research actors.</td></tr><tr><td>Reflexivity</td><td>Encourages constant self-assessment of research impacts to produce more directed research.</td><td>Promotes reflection on the practices and impacts of evaluations.</td></tr><tr><td>Diversity</td><td>Values the inclusion of diverse perspectives and approaches.</td><td>Ensures diversity and inclusion in evaluation practices.</td></tr><tr><td>Impact</td><td>Focuses on the direct social and environmental benefits of innovations.</td><td>Evaluates the real impact of research practices on society across various dimensions.</td></tr><tr><td>Anticipation</td><td>Adopts a forward-looking view that considers potential opportunities, risks, environmental concerns, etc.</td><td>Implements evaluation results to enable impactful actions.</td></tr><tr><td>Adaptation</td><td>Adapts methods and processes to social needs.</td><td>Reforms evaluation practices for ethical alignment.</td></tr><tr><td>Impact Monitoring</td><td>Evaluates the social and environmental impact of research.</td><td>Monitors and assesses the impact of implemented changes.</td></tr><tr><td>Empowerment</td><td>Develops capacities for responsible research.</td><td>Promotes organizational capacities for change.</td></tr></tbody></table>

Source: Design of the table inspired on: CoARA (2022); European Commission (2016); Himanen (2023); Wilford et al. (2016) and UKRI (2023).

Similarly, the emerging concept of 'Culture of Impact' in agricultural RD&I organizations (Blundo-Canto et al., 2019) resonates with the principles of RRI and RRA, emphasizing the importance of an organizational culture oriented towards the staff reflecting, in various manners, on the contributions of research activities to societal impacts. Although Blundo-Canto et al. (2019) do not directly address the principles of RRI and RRA, they highlight participatory evaluation as a deliberate strategy of organizations to foster understanding of impact contributions and the underlying generation mechanisms. In addition, the study emphasizes the importance of inclusive approaches by engaging with a wide range of stakeholders, encouraging reflexivity through continuous learning and adaptation, and promoting responsiveness by adjusting research processes to maximize societal benefits.

This suggests that a well-established culture of impact does also include organizational processes along the strategic use of evaluations' results to enhance RD&I practices. On a similar vibe, Ferre et al. (2023) recently investigated, in three agricultural RD&I organizations, internal practices and processes associated with a culture of impact's ambition. They

analysed how a culture of impact is implemented on the one hand, and experienced by staff on the other hand, including comparisons of the respective organisational mechanisms along the use of evaluation results. It appears clearly that a culture of impact does entail a strategic effort to integrate evaluations results into planning and daily operations. In practice, this can involve adjusting resource allocation based on the outcomes of evaluated projects, refining research agendas to better address societal needs, or improving stakeholder engagement processes (Patton & Horton, 2009).

Nevertheless, the literature lacks in-depth discussions on the systematization, at organizational level, of use of evaluation results as a process (Lee, 2010). The former author emphasizes the need to advance the debate on the use of results in research evaluation from an information management perspective, similar to the approach of Horton and Mackay (2003). Lee (2010) proposes a conceptual model for managing evaluation results by integrating the evaluation of S&T programs and policies with Korean research. This highlights the necessity for more practical approaches that can guide evaluation management based on the use of results.
Despite significant advancements in evaluation theory and practice, there is a notable gap in the literature on the systematization of evaluation result usage at the organizational level. This study seeks to address this gap by proposing a framework that integrates evaluation results into agricultural research organizations' R&D agendas, ensuring alignment with RRI and RRA paradigms. By advancing beyond accountability, this approach aims to promote a strategic, systematic, and impactful use of evaluations, contributing to societal needs.

### 3. Methodology

For the development of the AgroRadarEval tool, three main stages were followed, aimed at building a tool based on the existing literature and evaluation practices. These stages were: (1) literature review and study selection, (2) reading, categorization, grouping, and pattern identification, and (3) visual representation and tool automation.

#### (1) Literature Review and Study Selection

The literature review was conducted using the Scopus, Web of Science (WoS), and Google Scholar (GS) databases, chosen due to their quality and broad coverage of academic, technical, and scientific publications (Harzing & Alakangas, 2016). A total of 252 studies were identified based on descriptors designed to address the use of evaluation results in R&D and S&T (see Table A1 in Appendix A for the complete list of

descriptors and search terms). The PRISMA protocol (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) (Page et al., 2021) was adopted, following these procedures:

* Inclusion of four preselected studies not retrieved through the descriptors (n = 4) (Morgan et al., 2017; Parks et al., 2019; Joly et al., 2016; Van der Most, 2010).

* Exclusion of duplicates (n = 32) via R and manual review;

* Exclusion of records without complete metadata in the "Abstract" section (n = 81), using R;

* Exclusion of studies not directly related to the topic of S&T and R&D (n = 113);

* Inclusion of 26 studies directly related to the topic;

* Addition of 19 key references from the bibliographies of these 26 studies (n = 19);

* Inclusion of studies from Branch of Use and direct search in the journals Research Evaluation, Research Policy, and Evaluation and Program Planning related to the principles of RRI, RRA, and the impact evaluation of agricultural research (n = 29).

These steps resulted in 78 selected studies, comprising articles, monographs, book chapters, and reports (Figure 1).
<figure><img src="image_3.png" /><figcaption>Figure 1. PRISMA Flow Diagram of Study Selection Process for the AgroRadarEval Development.</figcaption></figure>
**(2) Reading, Categorization, Grouping, and Identification of Emerging Patterns**

The 78 studies were fully read, and for cross-analysis and organization of their contents, Bardin's (2011) approach was adopted. This approach is based on three cycles: 1. **Pre-analysis**, where the material is organized and the analyse objectives are defined; 2. **Exploration of Studies**, where the content is classified, grouping studies according to their similarities; and 3. **Results processing**, where inference, interpretation, and final organization of the content are conducted to define a structure, which enhances the achievement of the defined objectives.

For the **Pre-analysis**, three categories were defined (see in Table A2 in Appendix B): 1. Theoretical and Conceptual Elements, referring to concepts and theories directly related to the topic of interest, considering the types of evaluation (programme and public policy evaluation, research evaluation, and agricultural research evaluation), areas of knowledge, and types of uses and users; 2. Operational Elements, related to studies that proposed actions to operationalize the evaluation process based on the use of evaluation results; and 3. Evaluation Trends, referring to studies that point out how evaluation should evolve, particularly in relation to a transformation/impact agenda.

Based on the defined objectives, the “exploration of studies” phase was initiated, involving the classification of themes, central ideas, and practical aspects related to the use of evaluation results in each of the 78 studies. This classification was conducted using Excel and Miro, as detailed in Appendix A.

In the “results processing” phase, the studies were consolidated to construct the tool and its interactivity. The categories “Theoretical and Conceptual Elements” and “Evaluation Trends” defined the guiding concepts, referred to as “pillars”, essential for systematizing evaluation results under RRI and RRA principles. In the “Operational Elements” category, the thematic groupings, referred to as “dimensions”, and the practical aspects, derived from the 78 studies, were consolidated. These practical aspects, drawn from the literature, were converted into questions to directly assess representatives of agricultural research organizations. Figure 2 summarizes this process.

<figure><img src="image_4.png" /><figcaption>**Figure 2.** Summary of the results processing. Available at: https://bit.ly/3BNzmWH</figcaption></figure>

In this sense, the “dimensions” and “questions” resulted in the synthesis of the key elements for developing the interactivity of the AgroRadarEval framework and its subsequent automation. The final arrangement of the “results processing” indicated a hierarchical relationship, represented by a relational tree type (Figure 3), defined with the emergence of the “pillars” and the establishment of dimensions and

questions. According to Bazeley (2020), a relational tree, or hierarchical tree, organizes elements in a branching structure, where each node can have sub-levels or “children”. As shown in Figure 2, AgroRadarEval’s structure has three levels: “pillars” as main nodes (“parents”), dimensions as “child nodes”, and “questions” as the lowest level (“leaves”). Dimensions and questions operationalize the tool’s functionality.
<figure><img src="image_5.png" /><figcaption>**Figure 3.** Hierarchical relationship of AgroRadarEval. Available at: https://bit.ly/4f6Gz2i</figcaption></figure>

### (3) Validation of Dimensions and Questions

The validation process involved iterative consultations with evaluation experts, including the impact evaluation team at Embrapa Territorial and other professionals. Informal meetings and presentations, such as one conducted at the scientific conference ESOCITE<fnref n="2" />, were used to gather feedback. Specialists provided input on the proposed dimensions, which were well-received overall, leading to their retention. Some questions within specific dimensions were revised based on this feedback. Further details about the validation criteria and robustness testing of the tool are provided in Appendix A.

### (4) Visual Representation and Tool Automation

With the dimensions identified, a visual representation of the tool was produced through the customization of a vector image obtained from the Freepik website, using the Inkscape software. This design helps users reflect on organizational practices that support the use of evaluation results in strategic decision-making, ensuring clarity and precision for understanding and interpretation.

Based on the defined dimensions and questions, the tool's interactivity was developed using a Likert scale (Albaum, 1997; Nemoto & Beglar, 2014), enabling representatives of agricultural research organizations to respond on a scale from 1 (minimum) to 5 (maximum). The scale allows the calculation of maximum and average scores for each dimension. An electronic form was created in Google Forms to organize the questions by dimension, and responses were processed through an R script using the Shiny package (Posit.co, 2023), as shown in Figure 4.

<figure><img src="image_6.png" /><figcaption>**Figure 4.** Workflow of data collection, processing, and visualization. Available at: https://bit.ly/4f73wCF</figcaption></figure>

<footnote n="2">² PINTO, D. M.. AGRORADAREVAL: framework focado no uso dos resultados da avaliação da pesquisa para o maior impacto.. In: JORNADA LATINO-AMERICANA DE ESTUDOS SOCIAIS DA CIÊNCIA E TECNOLOGIA, 2024, Campinas. Anais..., 2024.</footnote>
The R script automates the capture of data from the form, normalizes the calculations based on the adopted quantitative structure, and generates graphical and textual visualizations. The processed data is presented through a radar chart, which facilitates the comparison between the maximum and average score observed for each dimension. Additionally, two lists providing information on areas of “highlights” and those “in need of improvement” are displayed alongside the chart. This arrangement was implemented through the creation of an online environment on the ‘shinyapps.io’ platform, which simplifies access, the use of the form, and the immediate visualization of results. The developed script, the raw and final data, as well as other files used in this study are available on GitHub Repository.

## Methodological Limitations

Despite efforts to ensure transparency and replicability, the methodology has limitations, primarily the subjectivity inherent in literature analysis. While protocols like PRISMA and validated descriptors were used,

tasks such as reading, categorizing, and interpreting results depend on researchers' knowledge and experience, potentially introducing biases. This subjectivity is particularly relevant when analyzing complex concepts and categorizing studies into the three proposed categories. Tools like Excel and Miro were employed to reduce biases, but interpretations still reflect individual perspectives.

# 4. Results

## Theoretical and conceptual elements and Evaluation trends

From the categories of ‘theoretical and conceptual elements’ and ‘evaluation trends,’ three key support pillars emerged. They were associated with different themes and central ideas, as well as practice aspects, to enhance the use of evaluation results in agricultural RD&I, in line with the RRI and RRA paradigms. As we can see at Figure 5, they are: 1. Evaluation Capacity Building (ECB), 2. Impact-Oriented Evaluation Culture, and 3. Reflective Learning.

<figure><img src="image_7.png" /><figcaption>**Figure 5.** Key Support Pillars for Enhancing the Use of Evaluation Results in Agricultural RD&I. Available at: https://bit.ly/4dV4OiZ</figcaption></figure>

“ECB” is recognized as a strategic concept focused on providing structures, tools, knowledge, skills, and organizational support to integrate evaluation practices into daily processes. It encompasses training, knowledge acquisition, and collaborative strategies to foster sustainable evaluation capacity and organizational learning. Theoretical models by Preskill and Boyle (2008) and Labin et al. (2012) emphasize ECB’s alignment with strategic decisions, a premise supported by OECD (2022, 2023), CoARA (2022), and research from White et al. (2018) and Turner et al. (2022).

The “Impact-Oriented Evaluation Culture” focuses on fostering impactful research practices through concepts like outcome-oriented evaluations, impact pathways, performance measurement, theory of change, and accountability. Particularly, approaches such as “Culture of impact” and “Pathways to Impact” demonstrate this premise in the

field of agricultural research. Studies such as Reed et al. (2021, 2022), Stilgoe and Guston (2016), Felt (2018), Owen et al. (2020), and Spaapen (2015) highlight the role of data-driven decision-making, aligned with RRI and RRA principles. Impact-focused studies in agricultural RD&I by Douthwaite et al. (2003, 2023) and Alvarez et al. (2010, 2014) also emphasize this trend.

Reflective Learning, often highlighted in program evaluations (Chelimsky, 1977, 2015; Patton, 2020; Weiss, 1998), emphasizes critical reflection, adaptive learning, and organizational reflexivity. Studies by Felt (2018), Schuijff & Dijkstra (2020), and Von Schomberg (2019) reinforce its importance for continuous improvement and learning through feedback and participatory processes. This pillar underpins a strategic evaluation approach in agricultural RD&I, fostering transformative use of evaluation results for greater impact.
## Operational Elements

Based on the concepts associated with the three pillars and 290 notes from the analyzed sample, we clustered studies into eight dimensions: 1. Participation and Collaboration; 2. Skill Development; 3. Promotion of an Evaluation Culture; 4. Feedback and Continuous Adaptation; 5. Integration with Strategic Planning; 6. Monitoring of Recommendations Implementation; 7. External Environment; and 8. Communication. These dimensions were linked to practical aspects, which were transformed into 94 questions. Their relationships can be viewed at < https:// bit.ly/agroradareval_en, page: "Pillars, Dimensions and Questions" >.

Some concepts overlap, such as organizational learning, participation and collaboration, and strategic planning. However, for organizing the tool, the dimensions were associated with the three pillars, as shown in Figure 5, and visually represented in a circular design emphasizing their interdependence (Figure 6). Depicted as a puzzle, this representation highlights that all components are essential and mutually reinforcing. Each puzzle piece represents a dimension, and together they form a cohesive whole, emphasizing the integrative nature of the tool.

<figure><img src="image_8.png" /><figcaption>**Figure 6.** Dimensions of the AgroRadarEval. Available at: Framework</figcaption></figure>

The eight dimensions, though distinct in focus, work together toward the shared goal of strengthening organizational capacity to effectively use impact evaluation results. They support participatory and collaborative management of evaluative activities, applying knowledge to foster responsible and impactful research, while promoting ongoing cycles of learning and improvement through feedback and monitoring. In this manner, **Participation and Collaboration** comprises 14 questions aimed at engaging diverse stakeholders in the continuous evaluation process of R&D interventions, as highlighted by Patton (2008; 2002) and Stockmann et al. (2020, 2022). These stakeholders include the evaluation team, managers, researchers, targeted stakeholders, and other interested parties. This dimension fosters reflection on practical actions to identify demands and potential uses of evaluation results.

**Skill Development** reunites 11 focused on building or strengthening evaluation competencies through organizational strategies, from identifying training needs to implementing competency development processes. Training targets both the evaluation team and other organizational members. This dimension draws on Preskill and Boyle's (2008) multidisciplinary ECB model and Labin et al.'s (2012) needs-activities-outcomes framework to enhance the integration and utility of evaluation results within the organization.

**Promotion of an evaluation culture** aligns with organizational change and the institutionalization of evaluation practices as integral to strategy and operations. It fosters a culture that values evaluation and decision-making based on results (Patton & Horton, 2009; Blundo-Canto et al., 2019). This includes establishing spaces for dialogue on evaluation outcomes and forming communities of practice to ensure evaluation processes consistently support RD&I management and decision-making.
**Feedback and Continuous Adaptation** fosters an evaluative process involving contributions from various actors on evaluation results. It develops mechanisms for feedback and process adaptation to align evaluations with stakeholder needs and priorities, promoting continuous learning (Schönbrodt et al., 2022) and enhancing research and impact-oriented practices (Reed et al., 2021; 2022).

**Integration with Strategic Planning** embeds evaluations and their results into the strategic planning process, aligning outcomes and impacts with organizational objectives, especially in agricultural RD&I management (Labin et al., 2012; Morgan Jones et al., 2017; Weißhuhn et al., 2018). Using evaluation results to guide decision-making enables organizations to adjust strategies, prioritize investments, and realign projects to meet key goals. Active engagement from leaders and RD&I managers is crucial for ensuring evaluations drive continuous improvement and alignment.

**Monitoring the Implementation of Recommendations** ensures the evaluation cycle extends beyond result presentation, emphasizing continuous and adaptive follow-up. This involves clear indicators, stakeholder feedback, and alignment with strategic and impact goals. Closely tied to research activities and evaluation outcomes, monitoring relies on strategic indicators (KPIs) (Douthwaite et al., 2003; 2023) to verify the effectiveness of implemented changes, address challenges promptly, and strengthen research impact.

The **External Environment** significantly influences organizational decisions and operations (Stockmann, 2020, 2022). Factors like regulations, public policies, technological trends, market demands, affirmative agendas, and socio-economic challenges shape research orientation and the use of evaluation results. Productivity issues, global challenges, and stakeholder pressures (Thornton et al., 2018) further affect evaluation acceptance and implementation. Engaging stakeholders, such as policymakers, farmers, and development agencies, is essential to integrating evaluation results into decision-making, enhancing long-term impacts on productivity and sustainability (Thornton et al., 2017).

Finally, **Communication** is a cross-cutting dimension essential for effective use of impact evaluation results (Patton, 2002; 2008). It encompasses strategies, channels, and practices for disseminating results to diverse audiences, ensuring they are accessible, clear, and actionable. Effective communication ensures evaluation results are understood and usable by stakeholders through targeted, comprehensible messages.

To align each dimension with RRI and RRA principles, we compared their core components to key aspects such as inclusivity, reflexivity, sustainability, societal engagement, fairness, transparency, and societal impact. Table 2 summarizes this association, linking studies, RRI/RRA principles, and the number of questions created. The number of questions varies by dimension due to differing scopes; for instance, 'Participation and Collaboration' involves identifying actors and their needs, while 'Integration with Strategic Planning' is more internally focused.
<table><caption>**Table 2.** Description of Dimensions</caption><thead><tr><th>Pillar</th><th>Dimension</th><th>Authors</th><th>RRI/RRA Principles</th><th>Number of related Questions</th></tr></thead><tbody><tr><td rowspan="5">Evaluation Capacity Building (ECB)</td><td>Skill Development</td><td>Labin et al. (2012); Preskill & Boyle (2008); Bourgeois; Cousins (2009); Deniston (1980); Horton (2003)</td><td>Ethics and Integrity Empowerment</td><td>12</td></tr><tr><td>Integration with Strategic Planning</td><td>Horton (1998; 2003); White et al. (2018); Turner et al. (2022); Regan (2021); Joly et al. (2014; 2016); Alston (1995; 2010); Evenson et al. (1979); Labin et al. (2012); Preskill e Boyle (2008)</td><td>Anticipation</td><td>9</td></tr><tr><td>Communication</td><td>OECD (2022; 2023); Reed et al. (2021; 2022); Bayley (2023); Bayley e Phillips (2019; 2023); Milzow et al. (2019); Stockmann et al. (2020; 2021); Felt (2018); Fteval (2003); Morgan et al. (2016; 2017); Parks et al. (2019)</td><td>Transparency</td><td>14</td></tr><tr><td>Participation and collaboration</td><td>Cousins & Leithwood (1996); King Mertens; Stufflebeam (1983; 2002); Patton (2008; 1988; 2002); Weiss (1972; 1998); Chelimsky (1977); Stockmann et al. (2020; 2022); Rutman; Wholey (1980)</td><td>Diversity Engagement</td><td>14</td></tr><tr><td>Promotion of an evaluation culture</td><td>Morgan et al. (2016; 2017); OECD (2022; 2023); Blundo et al. (2019); Milzow et al. (2019); Schionbrodt et al. (2022); Reed et al. (2021; 2022); Spaapen (2015); Bayley (2023); Bayley e Phillips (2019; 2023); Strand et al. (2015); Fteval (2003); OECD (2022; 2023); Von Schomberg et al. (2013)</td><td>Ethics and Integrity</td><td>13</td></tr><tr><td rowspan="2">Impact-Oriented Evaluation Culture</td><td>Monitoring of recommendation for implementation</td><td>Joly et al. (2014; 2016); Weishuhn et al. (2018); Horton (1998); Pingali (2001); Horton; Patton (2009); Paul (2020); Lee et al. (2020); Saari; Kalio (2012); Blundo et al. (2019); White et al. (2018); Turner et al. (2022); Regan (2021); Alvarez et al. (2010); Douthwaite et al. (2010; 2023); Mackay; Horton (2003); Horton; Mackay (2003); Horton (1998; 2003); Alvarez et al. (2010; 2014)</td><td>Transparency Impact Impact Monitoring Anticipation</td><td>11</td></tr><tr><td>Feedback and continuous adaptation</td><td>Labin et al. (2012); Mertens; Patton (2008; 1988; 2002); Weiss (1972; 1998); Chelimsky (1977); Patton (2008; 1988; 2002); Stockmann et al. (2020; 2022)</td><td>Reflexivity Adaptation</td><td>10</td></tr><tr><td>Reflective Learning</td><td>External environment</td><td>Fteval (2003); Schuijff; Dijkstra (2020); Felt (2018); Von Schomberg et al. (2013); Reed et al. (2021; 2022)</td><td>Diversity Impact</td><td>11</td></tr><tr><td colspan="3">TOTAL OF QUESTIONS</td><td></td><td>94</td></tr></tbody></table>
## AgroRadarEval Interactivity

AgroRadarEval includes a total of 94 questions designed to assess interactions that promote the effective use of evaluation results in RD&I for impactful actions. These questions can be answered based on the tool's quantitative scale, which employs a spectrum from "Level 1 to 5", where 1 indicates the lowest value or degree of agreement/achievement in the organization, and 5 represents the highest value or full agreement/ achievement (see Table B1 in Appendix B. To further contextualize, the scale not only measures the reported presence of practices or skills in the organization but also their effectiveness and level of development within the agricultural evaluation context. This scale guides strategic

decision-making by identifying areas for improvement and aligning efforts with desired outcomes, supporting continuous improvement and impactful evaluations.

Thus, the set of questions for each dimension allows the creation of a score, which is defined by calculating both maximum and average values, based on the number of questions within each dimension. Additionally, the normalization of average score is performed, as detailed in Table 3. These scores act as performance indicators, allowing for an objective and comparative evaluation of the conditions of the agricultural R&D environment in relation to the proposed questions.

<table><caption>Table 3. Established Maximum and average scores per dimension.</caption><thead><tr><td>Dimension</td><td>Questions</td><td>Established Maximum Score</td><td>Established Average Score</td><td>Weight</td><td>Normalized Average Value</td></tr></thead><tbody><tr><td>Participation and Collaboration</td><td>14</td><td>70</td><td>35</td><td>1,43</td><td>50</td></tr><tr><td>Competence Development</td><td>12</td><td>60</td><td>30</td><td>1,67</td><td>50</td></tr><tr><td>Promotion of an Evaluation Culture</td><td>13</td><td>65</td><td>32,5</td><td>1,54</td><td>50</td></tr><tr><td>Feedback and Continuous Adaptation</td><td>10</td><td>50</td><td>25</td><td>2,00</td><td>50</td></tr><tr><td>Integration with Strategic Planning</td><td>9</td><td>45</td><td>22,5</td><td>2,22</td><td>50</td></tr><tr><td>Monitoring the Implementation of Recommendations</td><td>11</td><td>55</td><td>27,5</td><td>1,82</td><td>50</td></tr><tr><td>External Environment</td><td>11</td><td>55</td><td>27,5</td><td>1,82</td><td>50</td></tr><tr><td>Communication</td><td>14</td><td>70</td><td>35</td><td>1,43</td><td>50</td></tr></tbody></table>

The assigned weight is related to dividing 100 by the maximum value, and should not be confused with any representation of importance or hierarchy among the dimensions. Furthermore, the normalized maximum and medium values are essential for visually analyzing the results, considering how close or far an organization is from the established benchmarks.

## Automation and Data Collection

Operationally, AgroRadarEval functions through an electronic form (see Figures B1-A and B1-B in Appendix C) hosted in a web environment via an R script updating every 60 seconds. The form presents questions arranged by dimension and should ideally be answered by representatives with relevant expertise (see Table B2 in Appendix C for detailed profile characteristics). The web environment, available at: https://bit.ly/agroradareval_en, includes five pages:

<figure><img src="image_9.png" /><figcaption>Figure 7. "Evaluation Radar" Page.</figcaption></figure>
In the "Evaluation Radar" page, on the left side, it presents a selection box that allows identifying the organization, and in the center, a radar chart, followed by "Highlights" and "Areas for Improvement." This section textually highlights the main strengths identified and the areas that require attention based on the answered questions. This representation corresponds to the response values for each question, with values higher than 3 classified as "highlight areas" and values lower than 3 as "areas for improvement." Additionally, the interface offers a "Download data" option, allowing users to export the data for more detailed analyses or internal presentations.

## Pilot - Application of the Tool

The AgroRadarEval tool was applied to employees from the Brazilian Agricultural Research Corporation (Embrapa), allowing for the evaluation of performance in the different dimensions, as detailed in Table 4. The application of the tool took place in February 2024 and involved two agricultural R&D evaluation experts, one focused on "Innovation and Technology Transfer" and the other on "Research and Development", both working in impact evaluations at Embrapa.

<table><caption>Table 4. Presentation of the values achieved, compared to the normalized average and maximum values</caption><thead><tr><td>Dimension</td><td>Established Median Score</td><td>Established Maximum Score</td><td>Weight</td><td>Organization Score</td><td>Weighted Organization</td><td>%</td></tr></thead><tbody><tr><td>Participation and Collaboration</td><td>35</td><td>70</td><td>1,43</td><td>39</td><td>57,14</td><td>10,26</td></tr><tr><td>Skill Development</td><td>30</td><td>60</td><td>1,67</td><td>32</td><td>53,30</td><td>6,25</td></tr><tr><td>Promotion of an Evaluation Culture</td><td>32,5</td><td>65</td><td>1,54</td><td>44</td><td>67,90</td><td>26,14</td></tr><tr><td>Feedback and Continuous Adaptation</td><td>25</td><td>50</td><td>2,00</td><td>23,5</td><td>47,00</td><td>-6,38</td></tr><tr><td>Integration with Strategic Planning</td><td>22,5</td><td>45</td><td>2,22</td><td>21,5</td><td>47,70</td><td>-4,65</td></tr><tr><td>Monitoring of Recommendation Implementation</td><td>27,5</td><td>55</td><td>1,82</td><td>26</td><td>47,20</td><td>-5,77</td></tr><tr><td>External Environment</td><td>27,5</td><td>55</td><td>1,82</td><td>33,5</td><td>60,90</td><td>17,91</td></tr><tr><td>Communication</td><td>35</td><td>70</td><td>1,43</td><td>37,5</td><td>53,50</td><td>6,67</td></tr></tbody></table>

The results are illustrated in Figures 8, 9 and 10 revealing that the organization performed above average in five dimensions: "Promotion

of an Evaluation Culture," "External Environment," "Participation and Collaboration," "Skill Development," and "Communication."

<figure><img src="image_10.png" /><figcaption>Figure 8. Visual presentation of results.</figcaption></figure>
The "Promotion of an Evaluation Culture" dimension stood out the most, with a performance of 26% above average. This result suggests a commitment to creating an environment that values evaluation and evidence-based decision-making. Since the late 1990s, Embrapa has established its process for measuring the impact of its technology, which is recognized in Brazil and is the basis for the company to publish its annual Social Balance report (Avila et al., 2008). Another notable dimension is "External Environment," with a performance 18% above the established median score, indicating the organization's ability to adapt, interact, and respond to external demands, such as market trends, public policies, and socioeconomic challenges.

The dimensions “Participation and Collaboration,” “Skill Development,” and “Communication” showed moderate performance. In the case of the first, although the dimension recorded an increase of 10.3% above average, it indicates that there is still room for improvement in fostering greater internal and external participation and collaboration. For the dimensions “Communication” and “Skill Development”, with increases above average of 6.3% and 6.7%, respectively, there is an indication of a good level of engagement and effort in training and effective communication, but with more room to expand these dimensions.

On the other hand, the “Feedback and Continuous Adaptation” dimension showed a result 6.4% below average, suggesting the need to improve feedback mechanisms and adaptability to evaluations and results, which may compromise the use of evaluation evidence in the context of RD&I management and organizational organization. The “Integration with Strategic Planning” dimension followed a similar trend, with a performance 4.7% below average, indicating that the alignment between evaluations and the organization’s strategic planning can

be improved, especially regarding the applicability of evaluation results to support R&D management and organizational learning. Similarly, “Monitoring the Implementation of Recommendations,” with 5.8% below average, highlights the importance of strengthening the follow-up and implementation of recommendations resulting from evaluations.

A comparative analysis between the highest-performing dimension, “Promotion of an Evaluation Culture”, and the lowest-performing, “Feedback and Continuous Adaptation”, reveals gaps that emphasize the organization’s strengths in fostering evaluation practices but highlight weaknesses in adaptive processes. This gap of 32.5% suggests a misalignment between organizational culture and its operational adaptability, which could hinder the full use of evaluation results in RD&I management. These data come automatically after completing the survey and indicate areas of strength and potential for the organization, as well as aspects that require more attention and development.

As part of the validation process, the two evaluation experts reviewed the dimensions and corresponding questions to ensure alignment with organizational objectives and strategic priorities. The experts also assessed the relevance and clarity of each question, providing feedback that refined the tool's usability. However, it was noted that some dimensions, such as “Monitoring the Implementation of Recommendations” may require further contextual adaptation for broader application.

Figure 9 highlights the dimensions where the organization excels, pinpointing the specific questions/aspects that scored above the median level of 3 (on a 1 to 5 scale) within each dimension. In contrast, Figure 10 underscores areas requiring improvement, identifying questions/aspects that reveal the need for stronger feedback mechanisms.
<figure><img src="image_11.png" /><figcaption>Figure 9. Highlights</figcaption></figure>
<figure><img src="image_12.png" /><figcaption>Figure 10. Areas for improvement.</figcaption></figure>
## 6. Discussion

The AgroRadarEval tool is the result of an extensive literature review that covered theoretical, conceptual, trends and practical aspects to create a resource capable of facilitating analysis for RD&I managers and leaders of agricultural research organizations. This comprehensive review synthesized key principles, particularly in relation to RRI and RRA, ensuring alignment with the most relevant trends in evaluation and impact analysis. By integrating these principles, AgroRadarEval contributes to the theoretical understanding of how evaluation frameworks can be systematically aligned with organizational strategies to enhance societal impact, extending the literature on the strategic use of evaluation in RD&I contexts.

In this sense, AgroRadarEval's primary value lies in its ability to foster responsible research practices, offering a structured, data-driven approach that helps organizations align their RD&I activities with societal needs through strengthening its evaluation capacity and integrating these evaluation processes with other elements of RDI management. By integrating RRI and RRA principles, the tool helps organizations reflect on practices, make informed decisions, and focus on impact generation aligned with strategic goals. It streamlines RD&I evaluation by organizing efforts into actionable dimensions, enabling identification of strengths and gaps while fostering continuous learning for greater societal impact (Blundo-Canto et al., 2019). While designed for agricultural research, the structured approach and its focus on dimensions such as capacity building, impact-oriented evaluation culture, and reflective learning suggest potential applicability in other RD&I sectors, such as health, education, or energy, which also demand impact-oriented evaluations.

Research on the use of evaluation results has gained prominence in the field of research evaluation (Pinto & Bin, 2024b), highlighting the strategic role that evaluation-derived information plays in RD&I management. As agricultural research increasingly needs to demonstrate its impact across social, environmental, economic, and cultural dimensions (Weishuhn et al., 2018), integrating evaluation into strategic planning becomes essential.

AgroRadarEval emphasizes the need to address the scarcity of public resources and the growing demand for more impactful and applied solutions (Reed et al., 2021; 2022). It highlights that evaluation results should guide organizational learning in agricultural R&D to achieve long-term, high-impact innovation and development strategies. The scarcity of resources, particularly human resources, can be a challenge, but not an insurmountable one. According to Preskill and Boyle (2008) and Labin et al. (2012), intelligent management is key to overcoming these barriers, utilizing adaptive and innovative strategies to optimize existing resources and demonstrating the value of investment in evaluation aimed at driving impact. Attention should be given to six key aspects that support greater impact from research through the use of evaluation results, namely:

1. Strategic Prioritization: Knowing in which key areas evaluation is most needed and can have the greatest benefit. This may mean predefining the impact of projects and programs that may be strategic and thus better align with the organization's long-term goals.

2. Internal Capacity Building: Investing in the training and development of the team's evaluation skills can be more cost-effective than waiting to hire new employees, which in many cases requires the release and approval of public representatives who set S&T policies. Furthermore, developing internal skills promotes the sustainability and resilience of the organization. In this regard, it is worth not only training organization members who deal directly with evaluation but also those who are on the front lines conducting research that should change the reality (Turner et al. 2022). Training these research professionals is often more focused on understanding what high-impact research should be rather than on evaluation itself. This is entirely linked to continuous learning for creating an impact culture based on lessons learned.

3. Collaborations and Partnerships: Establishing partnerships with other universities, or organizations can allow for the sharing of resources, knowledge, and experience. This involves not only expanding the knowledge base but also the exchange of experiences and mutual support for carrying out evaluation and R&D management activities.

4. Use of Technology and Data: Systematizing actions and automating processes for storing, collecting, and analyzing data can reduce manual workload and make the evaluation process more efficient. Technology can also facilitate large-scale and more frequent data collection. Currently, there are numerous free tools that can be used to support the realization and organization of evaluations, for example, to collect data: ODK Collect, KoboToolbox, DataWinners; and to analyze data: RStudio, KNIME, Orange, among others.

5. Incremental and Phased Evaluation: One possibility is to implement the framework in phases, which allows the organization to progressively build capacities and demonstrate the value of each stage of the process concerning organizational learning. Additionally, it renews the logic of the framework, respecting the uniqueness of each organization and understanding AgroRadarEval as an integrated construct in constant evolution and continuous, reflective, and contextual adaptation.

6. Demonstrating Value: Showing the results of the evaluation as improvements in the efficiency, effectiveness, and sustainability of programs can help secure stakeholder support and potentially unlock more resources. This involves the evaluation team's effort to find the best way to communicate the benefits of an impact-centered approach to the organization's leadership, funders, and policymakers, potentially increasing support and funding for these initiatives.
These aspects, while tailored to agricultural R&D, offer insights that are broadly relevant for organizations in other sectors facing similar constraints and impact-oriented goals. The study aims to fill the gap highlighted by Lee (2010) and Pinto and Bin (2004) regarding the lack of systematization in the evaluation process focused on use, concerning S&T and RD&I. Considering the principles of RRI and RRA, the proposed tool is based on the concepts of ECB, Impact-Oriented Evaluation Culture, and Reflective Learning, serving as a tool to support agricultural R&D organizations in diagnosing their evaluation process to strategically use their results. Furthermore, by addressing these principles through a practical framework, AgroRadarEval bridges theoretical gaps related to systematization in evaluation processes, expanding the applicability of these concepts across diverse contexts of innovation and research management.

## 7. Final Considerations

AgroRadarEval uniquely bridges gaps in the literature by automating the systematization of evaluation results for agricultural RD&I while operationalizing RRI and RRA principles, offering a theoretical contribution to the integration of evaluation and impact strategies. The automation proposed by AgroRadarEval is an important differentiator, allowing organizations to systematize and utilize evaluation results in a more consistent and impact-aligned manner, facilitating large-scale adoption and promoting efficient and accessible use of evaluation outcomes.

AgroRadarEval provides RD&I organizations with a scalable framework for enhancing impact, offering actionable insights for integrating evaluation results into strategic planning and decision-making. As the tool is used in different contexts, there will be opportunities to test and refine it further, allowing for continuous improvement and adaptation. This iterative process not only offers the potential to mature the tool but also to tailor it to the specific needs and realities of diverse agricultural RD&I organizations. In addition, it could serve as a knowledge base, enabling organizations to track and analyze their performance over time as they adapt and evolve their organizational contexts. Future development could include the integration of new technological advancements and the incorporation of user feedback, ensuring that AgroRadarEval remains relevant and effective over time.

Anchored in the idea that evaluation is a continuous process and not an end in itself, AgroRadarEval supports the path to operational excellence and the social relevance of agricultural research, aligned with the principles of RRI and RRA. Its application is expected to help organizations refine their practices, align strategies, and maximize the impact of evaluation results in their R&D management and activities.

The application of the presented tool was limited to a Brazilian organization, leaving room for future studies to explore it in a wider variety of contexts and countries, analyzing its effectiveness and adaptability. Future work should also address potential challenges, such as resource constraints and stakeholder engagement, by providing guidelines tailored to diverse organizational contexts and capacities. Additionally, future research can investigate how the proposed dimensions interact

with each other and how improvements in one dimension can positively influence others, promoting a broader and more integrated impact. For advancements in the tool, a maturity model can be employed to assist organizations in more effectively building their systematization of the evaluation process with a focus on using its results.

Finally, This study targets leaders and managers of agricultural RD&I organizations, evaluators, and scholars of research evaluation, encouraging a proactive approach to systematizing the evaluation process, with a vision for data-driven, evidence-based decision making. It is recognized that this tool is in its initial phase, requiring deeper engagement with the concrete challenges of the agricultural sector and its representation in research. Therefore, future research should focus on applying the tool across various agricultural RD&I contexts, refining its features and expanding its capabilities. Expanding engagement with a wider range of stakeholders, including policymakers and funders, can enrich the tool's applicability and ensure alignment with broader sectoral priorities.

## 8. References

* Albaum, G. (1997). The Likert Scale Revisited. *Market Research Society Journal*, 39(2), 1-21. https://doi.org/10.1177/147078539703900202

* Alkin, M. C., & Christie, C. A. (2004). An evaluation theory tree. *Evaluation roots: Tracing theorists' views and influences*, 2(19), Artigo 19.

* Alkin, M. C., & Christie, C. A. (2023). *Evaluation Roots: Theory Influencing Practice*. Guilford Publications.

* Alkin, M. C., Daillak, R., White, P., & White, P. (1979). *Using evaluations: Does evaluation make a difference?*

* Alston, J. M., Norton, G. W., & Pardey, P. G. (1995). Science under Scarcity: Principles and Practice for Agricultural Research Evaluation and Priority Setting. *American Journal of Alternative Agriculture*, 10(4), Artigo 4. https://doi.org/10.1017/S0889189300006597

* Alvarez, S., Douthwaite, B., Thiele, G., Mackay, R., Córdoba, D., & Tehelen, K. (2010). Participatory Impact Pathways Analysis: A practical method for project planning and evaluation. *Development in Practice*, 20(8), 946-958. https://doi.org/10.1080/09614524.2010.513723

* Alvarez, S., Jost, C., Schuetz, T., Förch, W., Schubert, C., & Kristjanson, P. M. (2014). *Lessons in Theory of Change from the Introductory Training on Theories of Change, Impact Pathways and Monitoring & Evaluation*. https://hdl.handle.net/10568/52992

* AVILA, A. F. D., RODRIGUES, G. S., & VEDOVOTO, G. L. (2008). * Avaliação dos impactos de tecnologias geradas pela Embrapa: Metodologia de referência*. Brasília, DF: Embrapa Informação Tecnológica: Embrapa. Secretaria de Gestão e Estratégia, 2008. http://www.alice.cnptia.embrapa.br/handle/doc/99957
Bardin, L. (2011). Content analysis. *São Paulo: Edições*, 70(279), 978-8562938047.

Bazeley, P. (2020). *Qualitative data analysis: Practical strategies*.

Blundo-Canto, G., Triomphe, B., Faure, G., Barret, D., De Romemont, A., & Hainzelin, E. (2019). Building a culture of impact in an international agricultural research organization: Process and reflective learning. *Research Evaluation*, 28(2), Artigo 2.

Bourgeois, I., & Cousins, J. B. (2013). Understanding Dimensions of Organizational Evaluation Capacity. *American Journal of Evaluation*, 34(3), 299-319. https://doi.org/10.1177/1098214013477235

Carley, M. J., & Bustelo, E. (2019). *Social impact assessment and monitoring: A guide to the literature*.

Chelimsky, E. (1977). Improving the Use of Evaluation: An Agency Perspective. *The Prison Journal*, 57(1), 13-18. https://doi.org/10.1177/003288557705700103

Chelimsky, E. (2015). A strategy for improving the use of evaluation findings in policy. *Evaluation use and decision making in society: A tribute to Marvin C. Alkin*, 73-89.

CoARA. (2022). *Agreement on Reforming Research Assessment: Now open for signature*. https://research-and-innovation.ec.europa.eu/news/all-research-and-innovation-news/agreement-reforming-research-assessment-now-open-signature-2022-10-03_en

Colinet, L. (2021). *ASIRPA: Societal impact of research*. INRA. https://www6.inrae.fr/asirpa_eng/ASIRPA-project/Method

Cousins, J. B., Elliott, C., Amo, C., Bourgeois, I., Chouinard, J., Goh, S. C., & Lahey, R. (2009). Organizational Capacity To Do and Use Evaluation: Results of a Pan-Canadian Survey of Evaluators. *Canadian Journal of Program Evaluation*, 23(3), 1-35. https://doi.org/10.3138/cjpe.0023.002

Cousins, J. B., Goh, S. C., Elliott, C. J., & Bourgeois, I. (2014). Framing the Capacity to Do and Use Evaluation. *New Directions for Evaluation*, 2014(141), 7-23. https://doi.org/10.1002/ev.20076

Cousins, J. B., & Leithwood, K. A. (1986). Current empirical research on evaluation utilization. *Review of educational research*, 56(3), 331-364.

Curry, S., Gadd, E., & Wilsdon, J. (2022). *Harnessing the Metric Tide: Indicators, infrastructures & priorities for UK responsible research assessment*. [Report]. Research on Research Institute. https://doi.org/10.6084/m9 figshare.21701624.v2

Douthwaite, B., Kuby, T., Van De Fliert, E., & Schulz, S. (2003). Impact pathway evaluation: An approach for achieving and attributing impact in complex systems. *Agricultural systems*, 78(2), 243-265.

Douthwaite, B., Proietti, C., Polar, V., & Thiele, G. (2023). Outcome Trajectory Evaluation (OTE): An Approach to Tackle Research-for-Development's Long-Causal-Chain Problem. *American Journal of Evaluation*, 44(3), 335-352. https://doi.org/10.1177/10982140221122771

European Commission. (2016). *Principles for Responsible Innovation: Building the trustworthiness of technology*. https://ec.europa.eu/information_society/newsroom/image/document/2016-4/sixth_cop_plenary_meeting_-_presentation_hilary_sutcliffe_matter_13334.pdf

Evenson, R. E. (2001). *Economic impacts of agricultural research and extension* (Vol. 1, p. 573-628). Elsevier. https://doi.org/10.1016/S1574-0072(01)10014-9

Evenson, R. E., Waggoner, P. E., & Ruttan, V. W. (1979). Economic Benefits from Research: An Example from Agriculture. *Science*, 205(4411), Artigo 4411. https://doi.org/10.1126/science.205.4411.1101

Felt, U. (2018). Responsible research and innovation. *Handbook of genomics, health and society*. https://books.google.com.br/books?hl=pt-BR&lr=&id=X3dZDwAAQBAJ&oi=fnd&pg=PT184&dq=Responsible+Research+and+Innovation&ots=nBvV5rI3JF&sig=-JbRQTppsqOopNw3w1ytXnJwHv0k

Ferre, M., Gomez, M. M. R. G., Vedovoto, G. L., Canto, G. B., Chocanta, B. E. A., Marques, D. V., Rodríguez-Borray, G.-A., Souza, M. O. de, Urriago, A. R. V., Rodrigues, G. S., Sánchez-Lozano, J.-I., Zambrano-Moreno, G.-S., Arias, M. a. L., Zapata-Tamayo, C.-A., Goulet, F., Proietti, C., & Romémont, A. de. (2023). *Culture of impact in agricultural research organizations: What for and how? - insights from a cross-analysis of Cirad, Embrapa, and Agrosavia institutes*. https://www.alice.cnptia.embrapa.br/alice/handle/doc/1159885

Grant, J., & Hinrichs, S. (2015). *The nature, scale and beneficiaries of research impact: An initial analysis of the Research Excellence Framework (REF) 2014 impact case studies*.

Harzing, A.-W., & Alakangas, S. (2016). Google Scholar, Scopus and the Web of Science: A longitudinal and cross-disciplinary comparison. *Scientometrics*, 106(2), Artigo 2. https://doi.org/10.1007/s11192-015-1798-9

Hemlin, S. (1996). Research on research evaluation. *Social Epistemology*, 10(2), 209-250. https://doi.org/10.1080/02691729608578815

Himanen, L. (2023). *Responsible research assessment principles for research information on management*.
Horton, D. (1998). Disciplinary roots and branches of evaluation: Some lessons from agricultural research. *Knowledge and Policy*, 10(4), Artigo 4. https://doi.org/10.1007/BF02912498

Horton, D., & Mackay, R. (2003). Using evaluation to enhance institutional learning and change: Recent experiences with agricultural research and development. *Agricultural Systems*, 78(2), 127–142.

Joly, P.-B., Colinet, L., Gaunand, A., Lemarié, S., & Matt, M. (2016). *Agricultural research impact assessment: Issues, methods and challenges*.

Kamenetzky, A., Hinrichs-Krapels, S., Wooding, S., & Grant, J. (2016). An analysis of the impact of research supported by the UK National Institute of Health Research. *London: The Policy Institute at King's College*.

Labin, S. N., Duffy, J. L., Meyers, D. C., Wandersman, A., & Lesesne, C. A. (2012). A research synthesis of the evaluation capacity building literature. *American Journal of Evaluation*, 33(3), Artigo 3.

Lee, J. (2010). How to manage and utilize the information from R&D evaluation? *PICMET 2010 TECHNOLOGY MANAGEMENT FOR GLOBAL ECONOMIC GROWTH*, 1–23.

Lee, S., Díaz-Puente, J., & Vidueira, P. (2020). Enhancing Rural Innovation and Sustainability Through Impact Assessment: A Review of Methods and Tools. *Sustainability*, 12(16), Artigo 16. https://doi.org/10.3390/su12166559

Luukkonen, T. (1995). The impacts of research field evaluations on research practice. *Research Policy*, 24(3), 349–365.

Mackay, R., & Horton, D. (2003). Expanding the Use of Impact Assessment and Evaluation in Agricultural Research and Development. *Agricultural Systems*, 78, 143–165. https://doi.org/10.1016/S0308-521X(03)00124-0

Milzow, K., Reinhardt, A., Söderberg, S., & Zinöcker, K. (2019). Understanding the use and usability of research evaluation studies. *research evaluation*, 28(1), 94–107.

Morgan Jones, M., Manville, C., & Chataway, J. (2017). Learning from the UK's research impact assessment exercise: A case study of a retrospective impact assessment exercise and questions for the future. *The Journal of Technology Transfer*, 1–25.

Nemoto, T., & Beglar, D. (2014). Likert-scale questionnaires. *JALT 2013 conference proceedings*, 1–8. https://jalt-publications.org/sites/default/files/pdf-article/jalt2013_001.pdf

OECD. (2022). *Agricultural policy monitoring and evaluation*. https://www.oecd.org/agriculture/topics/agricultural-policy-monitoring-and-evaluation/

OECD. (2023). *Developing evaluation capacities*. https://www.oecd.org/dac/evaluation/evaluatingcapacitydevelopment.htm

Owen, R., Macnaghten, P., & Stilgoe, J. (2020). Responsible research and innovation: From science in society to science for society, with society. *Em Emerging Technologies* (p. 117–126). Routledge.

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., & Brennan, S. E. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. *International journal of surgery*, 88, 105906.

Parks, S., Rincon, D. R., Parkinson, S., & Manville, C. (2019). *The changing research landscape and reflections on national research assessment in the future*. RAND. https://view.ckcest.cn/AllFiles/ZKBG/Pages/301/RAND_RR3200.pdf

Patton, M. Q. (2020). Evaluation Use Theory, Practice, and Future Research: Reflections on the Alkin and King AJE Series. *American Journal of Evaluation*, 41(4), Artigo 4. https://doi.org/10.1177/1098214020919498

Patton, M. Q., & Horton, D. (2009). Utilization-focused evaluation for agricultural innovation. *ILAC Brief*.

Posit.co. (2023). *Shiny*. https://shiny.posit.co/

Preskill, H., & Boyle, S. (2008). A multidisciplinary model of evaluation capacity building. *American journal of evaluation*, 29(4), Artigo 4.

Preskill, H., & Boyle, S. (2009). Insights into Evaluation Capacity Building: Motivations, Strategies, Outcomes, and Lessons Learned. *Canadian Journal of Program Evaluation*, 23(3), 147–174. https://doi.org/10.3138/cjpe.0023.008

Reed, M. S., Ferre, M., Martin-Ortega, J., Blanche, R., Lawford-Rolfe, R., Dallimer, M., & Holden, J. (2021). Evaluating impact from research: A methodological framework. *Research Policy*. https://doi.org/10.1016/j.respol.2020.104147

Reed, M. S., Gent, S., Seballos, F., Glass, J., Hansda, R., & Fischer-Møller, M. (2022). How can impact strategies be developed that better support universities to address twenty-first-century challenges? *Research for All*, 6(1). https://www.diva-portal.org/smash/record.jsf?pid=diva2:1713411

Reinertsen, H., Bjørkdahl, K., & McNeill, D. (2022). Accountability versus learning in aid evaluation: A practice-oriented exploration of persistent dilemmas. *Evaluation*, 28(3), 356–378.

Reinhardt, A., & Milzow, K. (2012). *Evaluation in research and research funding organisations: European practices*.
Rutman, L., & Wholey, J. S. (1980). *Planning useful evaluations: Evaluability assessment*. Sage Publications Beverly Hills, CA.

Schönbrodt, F., Gärtner, A., Frank, M., Gollwitzer, M., Ihle, M., Mischkowski, D., Phan, L. V., Schmitt, M., Scheel, A. M., & Schubert, A.-L. (2022). *Responsible Research Assessment I: Implementing DORA for hiring and promotion in psychology*. https://psyarxiv.com/rgh5b/download?format=pdf

Schuijff, M., & Dijkstra, A. M. (2020). Practices of Responsible Research and Innovation: A Review. *Science and Engineering Ethics*, **26**(2), 533–574. https://doi.org/10.1007/s11948-019-00167-3

Spaapen, J. B. (2015). A new evaluation culture is inevitable. *Organic Farming*, 1(1), Artigo 1.

Stilgoe, J., & Guston, D. (2016). *Responsible research and innovation*. MIT Press. https://discovery.ucl.ac.uk/id/eprint/10052401/1/Stilgoe_Guston_responsible_innovation_2017.pdf

Stockmann, R., Meyer, W., & Szentmarjay, L. (2022). *The institutionalisation of evaluation in the Americas*. Springer Nature.

Stockmann, R., Meyer, W., & Taube, L. (2020). *The institutionalisation of evaluation in Europe*. Springer Nature.

Stufflebeam, D. L. (1983). The CIPP Model for Program Evaluation. Em G. F. Madaus, M. S. Scriven, & D. L. Stufflebeam, *Evaluation Models* (p. 117–141). Springer Netherlands. https://doi.org/10.1007/978-94-009-6669-7_7

Stufflebeam, D. L. (2002). The CIPP Model for Evaluation. Em D. L. Stufflebeam, G. F. Madaus, & T. Kellaghan (Orgs.), *Evaluation Models* (Vol. 49, p. 279–317). Kluwer Academic Publishers. https://doi.org/10.1007/0-306-47559-6_16

Thornton, P., Schuetz, T., Förch, W., Cramer, L., Abreu, D., Vermeulen, S., & Campbell, B. (2017). Responding to global change: A theory of change approach to making agricultural research for development outcome-based. *Agricultural Systems*, **152**, 145–153. https://doi.org/10.1016/j.agsy.2017.01.005

Turner, J. A., Guesmi, B., Gil, J. M., Heanue, K., Sierra, M., Percy, H., Bortagaray, I., Chams, N., & Milne, C. (2022). Evaluation capacity building in response to the agricultural research impact agenda: Emerging insights from Ireland, Catalonia (Spain), New Zealand, and Uruguay. *Evaluation and Program Planning*, 102127.

UKRI. (2023). *Framework for responsible research and innovation*. https://www.ukri.org/who-we-are/epsrc/our-policies-and-standards/framework-for-responsible-innovation/

Van der Most, F. (2010). Use and non-use of research evaluation: A literature review. *CIRCLE Paper*, 2010/16. http://wp.circle.lu.se/upload/CIRCLE/workingpapers/201016_vanderMost.pdf

Von Schomberg, R. (2019). Why responsible innovation. *International handbook on responsible innovation: a global resource*, 12–32.

Weiss, C. H. (1972). Utilization of evaluation: Toward comparative study. *Evaluating action programs: Readings in social action and education*, 318–326.

Weiss, C. H. (1979). The Many Meanings of Research Utilization. *Public Administration Review*, 39(5), Artigo 5. https://doi.org/10.2307/3109916

Weiss, C. H. (1998). Have we learned anything new about the use of evaluation? *The American Journal of Evaluation*, 19(1), Artigo 1.

Weißhuhn, P., Helming, K., & Ferretti, J. (2018). Research impact assessment in agriculture—A review of approaches and impact areas. *Research Evaluation*, 27(1), Artigo 1.

White, T., Percy, H., & Small, B. (2018). Creating an evaluation culture through capacity building: A new frontier in a science organisation. *Evaluation Matters–He Take Tō Te Aromatawai*, 4, 111–135.

Wilford, S., Fisk, M., & Stahl, B. C. (2016). *Guidelines for responsible research and innovation*. De Montfort University.
# Appendix A - Methodology Details

<table><caption>Table A1. Descriptors used to identify studies.</caption><thead><tr><th>Expression</th><th>Scopus</th><th>WoS</th><th>GS</th><th>Total</th></tr></thead><tbody><tr><td>( "use of evaluation" OR "utilization of evaluation" OR "using evaluation" ) AND ( "research" OR "R&D" OR "RD" OR "research and devel*" OR "innov*" )</td><td>28</td><td>20</td><td>107</td><td>155</td></tr><tr><td>( "use of assess*" OR "utilization of assess*" OR "using assess*" ) AND ( "research" OR "R&D" OR "RD" OR "research and devel*" OR "innov*" )</td><td>11</td><td>5</td><td>74</td><td>90</td></tr><tr><td>"evaluation utilization" OR "assessment utilization" AND "research and development" OR "R&D" OR "RD" OR "Innovation" OR "Science" OR "Science and technology" OR "ST"</td><td>1</td><td>0</td><td>6</td><td>7</td></tr><tr><td>TOTAL</td><td>40</td><td>25</td><td>187</td><td>252</td></tr></tbody></table>

<table><caption>Table A2. Description of Categories and how they contribute to development of AgroRadarEval.</caption><thead><tr><th>Category</th><th>Description</th><th>Objective</th></tr></thead><tbody><tr><td>Theoretical and Conceptual Elements</td><td>The study is related to theories and concepts regarding the effective use of evaluation results in R&D contexts, considering the impact of research.</td><td>To provide a solid theoretical and conceptual base that aligns the tool with the fundamental literature on the use of evaluation results, highlighting indispensable concepts.</td></tr><tr><td>Operational Elements</td><td>Refers to studies that have already been explored in the literature related to the use of evaluation results, mainly, models and frameworks.</td><td>To assist in determining applicability of AgroRadarEval, regarding the use of evaluation to support R&D.</td></tr><tr><td>Evaluation Trends</td><td>The study presents new methodologies, concepts, or practices in agricultural R&D evaluation.</td><td>To provide innovative and updated perspectives, contributing to the improvement of the tool.</td></tr></tbody></table>

## (2) Reading, Categorization, Grouping, and Identification of Emerging Patterns

In Excel, the specific contributions of each study were catalogued, including information such as the field of knowledge, types of evaluation use, environments studied, and users involved. In Miro, three boards were created to organise the studies according to the type of evaluation (programme and public policy evaluation, research evaluation, and agricultural research evaluation). These boards allowed for the identification of the themes and central ideas of each study, using notes (post-its) and tags. This enabled the grouping of studies by thematic similarities and the identification of practical aspects that should be considered by an R&D institution aiming to systematise the use of evaluation results with a focus on a research agenda aimed at generating impacts and social transformations.

## (3) Validation of Dimensions and Questions

### 3.1 Criteria for expert selection included:

* Academic or professional experience in impact evaluation, particularly in agricultural research.

* Representation of diverse perspectives within the evaluation community, including practitioners, academics, and policymakers.

* Familiarity with concepts such as RRI and RRA or Culture of Impact.

The feedback was incorporated through iterative refinement, ensuring the framework's alignment with practical and theoretical expectations.

### 3.2 Robustness Testing of the Tool

The robustness of AgroRadarEval was tested by:

* Conducting internal consistency checks for the proposed dimensions and their corresponding questions.

* Simulating data input scenarios to evaluate the tool's capacity to differentiate organizational performance levels across dimensions.

* Presenting the tool to target users from agricultural research organizations to assess usability and clarity.

* Further adjustments to the tool were informed by these tests, ensuring that its interactivity and outputs align with its intended purpose.
# Appendix B - Results Details

<table><caption>Table B1. Adopted scale.</caption><thead><tr><td>Level</td><td>Description</td></tr></thead><tbody><tr><td>1</td><td>This is the lowest level of the scale. It represents a total or nearly total lack of the characteristic or parameter being evaluated. It may indicate strong disagreement with a statement, the absence of a skill, or the lowest score on a metric.</td></tr><tr><td>2</td><td>Indicates a minimal presence of the characteristic or skill assessed. Although it is higher than level 1, it still shows a significant need for improvement or development.</td></tr><tr><td>3</td><td>Represents an intermediate level. It can be seen as an average or balancing point. Indicates that the characteristic or skill is present but may not be fully developed and/or optimized.</td></tr><tr><td>4</td><td>Denotes a high degree of the characteristic or skill in question. Shows that the criterion evaluated is well-established, although there may still be room for improvement.</td></tr><tr><td>5</td><td>This is the highest level of the scale. Indicates excellence, total agreement, or the complete presence of the skill or characteristic evaluated. Represents the ideal or maximum achievement regarding the parameter in question.</td></tr></tbody></table>

<figure><img src="image_13.png" /><figcaption>Figure B1-A. Form.</figcaption></figure>

Available at: https://forms.gle/iJR6EV9Lz7iavbdk6

<figure><img src="image_14.png" /><figcaption>Figure B1-B. Evaluation Radar's page</figcaption></figure>

Available at: https://khi7yy-daniela-maciel0pinto.shinyapps.io/agroradareval_en/

<table><caption>Table B2. Profile Characteristics to Fill the AgroRadarEval.</caption><thead><tr><td>Nº</td><td>Characteristic</td><td>Description</td></tr></thead><tbody><tr><td>1</td><td>Knowledge of organizational processes</td><td>Understanding of all internal processes of the organization, especially in areas of research, development, and innovation.</td></tr><tr><td>2</td><td>Familiarity with internal evaluation practices</td><td>Understanding of how evaluations are conducted within the organization, including metrics and indicators used.</td></tr><tr><td>3</td><td>Strategic vision of R&D</td><td>Understanding of the organization's R&D objectives, as well as strategies designed to achieve them, including the development of new R&D programs and projects.</td></tr><tr><td>4</td><td>Knowledge of agricultural sector trends</td><td>Understanding of the trends and challenges in the agricultural sector that can influence R&D practices.</td></tr><tr><td>5</td><td>Ability for critical evaluation</td><td>Ability to critically evaluate internal practices and the organization's position regarding the use of evaluation evidence.</td></tr><tr><td>6</td><td>Familiarity with concepts of impact and innovation in research</td><td>Understanding of how research and its evidence can generate impact and innovation in the context of the organization.</td></tr></tbody></table>

Source: Design of the table inspired by Labin et al. (2012).

Identifying knowledgeable participants to use AgroRadarEval who together can cover all of these characteristics is ideal to enable the collected information to reflect a comprehensive understanding of the processes, practices, and strategic challenges of R&D and evaluation, as

well as the use of their results within the organization. The goal is to ensure that the responses are well-founded, considering the internal R&D environment and existing evaluative practices, as well as contemporary challenges in the agricultural sector.