# Challenges to ethical public engagement in research funding: a perspective from practice

Kalli Giannelos, Martijn Wiarda, Neelke Doorn

► To cite this version:

Kalli Giannelos, Martijn Wiarda, Neelke Doorn. Challenges to ethical public engagement in research funding: a perspective from practice. Open Research Europe, 2024, 4 (179), 23 p. 10.12688/openre-seurope.18126.2. hal-05017408

HAL Id: hal-05017408

https://sciencespo.hal.science/hal-05017408v1

Submitted on 2 Apr 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

CASE STUDY

**REVISED** Challenges to ethical public engagement in research
funding: a perspective from practice

[version 2; peer review: 2 approved, 1 approved with reservations]

Kalli Giannelos<sup>1,2</sup>, Martijn Wiarda<sup>1,3</sup>, Neelke Doorn<sup>3</sup>

**v2** First published: 13 Aug 2024, **4:179**
https://doi.org/10.12688/openreseurope.18126.1

Latest published: 06 Nov 2024, **4:179**
https://doi.org/10.12688/openreseurope.18126.2

**Abstract**

European research funding organizations (RFOs) are increasingly experimenting with public engagement in their funding activities. This case study draws attention to the challenges they face in preparing, implementing, and evaluating ethical public engagement in the context of setting funding priorities, formulating calls for proposals, and evaluating project proposals. We discuss challenges related to seven themes: (1) recruiting participants; (2) commitments and expectations; (3) meaningful dialogue and equal engagement; (4) accommodating vulnerability; (5) funding call formulations; (6) lack of expertise in engagement ethics; and (7) uncertainty, resource constraints, and external factors. To address these challenges, we propose the following seven interventions: (1) developing comprehensive recruitment strategies with experienced recruiters and community organizations; (2) establishing clear communication of roles, expectations, and outcomes through codes of conduct; (3) training mediators to address power imbalances; (4) designing flexible engagement methods and providing tailored support; (5) implementing collaborative feedback loops for inclusive funding call formulation; (6) enhancing ethical standards through internal expertise and external advisory inputs; and (7) developing adaptive strategies for flexible and ethical public engagement. These recommendations emphasize the need for context-adaptive insights to support funding organizations to implement ethical public engagement activities, even when faced with organizational constraints and a lack of ethical expertise.

**Open Peer Review**

Approval Status

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

1. Lucy Carter<sup>1,2</sup>, CSIRO Environment, Brisbane, Australia

2. Sarah Hatch, Cardiff University, Cardiff, UK

3. Loreta Tauginienė<sup>1,3</sup>, Kazimieras Simonavicius University, Vilnius, Lithuania

Any reports and responses or comments on the article can be found at the end of the article.

<footnote n="1">¹Governance and Regulation Chair, University of Paris Dauphine-PSL, Paris, Île-de-France, 75016, France</footnote>

<footnote n="2">²Centre for Political Research, Sciences Po, Paris, Île-de-France, 75007, France</footnote>

<footnote n="3">³Faculty of Technology, Policy and Management, Delft University of Technology, Delft, The Netherlands</footnote>
**Keywords**

engagement, participation, responsible research and innovation,
research funding, ethics, public engagement

<figure><img src="image_7.png" /></figure>

This article is included in the Horizon 2020
gateway.

<figure><img src="image_8.png" /></figure>

This article is included in the Research on
Research gateway.

<figure><img src="image_9.png" /></figure>

This article is included in the Research Ethics
and Integrity collection.

**Corresponding author:** Kalli Giannelos (kalli.giannelos@sciencespo.fr)

**Author roles:** Giannelos K: Conceptualization, Supervision, Writing - Original Draft Preparation, Writing - Review & Editing; Wiarda M: Conceptualization, Supervision, Writing - Original Draft Preparation, Writing - Review & Editing; Doorn N: Conceptualization, Supervision, Validation, Writing - Original Draft Preparation, Writing - Review & Editing

**Competing interests:** No competing interests were disclosed.

**Grant information:** This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No [872441](Participatory Real Life Experiments in Research and Innovation Funding Organisations on Ethics [PRO-Ethics]).

*The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.*

**Copyright:** © 2024 Giannelos K et al. This is an open access article distributed under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

**How to cite this article:** Giannelos K, Wiarda M and Doorn N. **Challenges to ethical public engagement in research funding: a perspective from practice [version 2; peer review: 2 approved, 1 approved with reservations]** Open Research Europe 2024, **4**:179 https://doi.org/10.12688/openreseurope.18126.2

**First published:** 13 Aug 2024, **4**:179 https://doi.org/10.12688/openreseurope.18126.1
**REVISED** Amendments from Version 1

In the revised version of the manuscript titled ‘Challenges to Ethical Public Engagement in Research Funding’, we have made specific changes to improve clarity, depth, and practical applicability based on reviewer feedback.

Firstly, the introduction has been expanded to provide a more comprehensive context for public engagement within research funding. Key terms, such as ‘public engagement’, are now explicitly defined to ensure consistent understanding across disciplines.

Secondly, each of the seven identified challenges to ethical public engagement has been elaborated in greater detail. This includes additional contextual explanations and examples to clarify challenges like recruitment difficulties, managing power imbalances in dialogue, and dealing with resource constraints. These enhancements offer readers a clearer view of how these issues play out in real-life scenarios, making the challenges more relatable and grounded.

The recommendations section has been substantially strengthened by incorporating more specific, actionable guidance. Each recommendation, such as developing recruitment strategies and training mediators, now includes practical steps for implementation. This revision provides detailed strategies for inclusive call formulations and bias reduction in participant selection, which makes the recommendations more actionable for research funding organizations.

Additionally, structural and stylistic changes have been made to improve readability and flow. Certain sections have been rephrased for conciseness and coherence, particularly where ethical standards and adaptive engagement strategies are discussed. These refinements contribute to a better comprehension of the arguments presented.

The authors wish to extend their sincere thanks to the three reviewers for their constructive comments, which played a crucial role in strengthening this manuscript. Their comments have led to significant improvements, enhancing both the clarity and impact of this paper.

Any further responses from the reviewers can be found at
the end of the article

**Introduction**

Research funding organizations (RFOs) across Europe are increasingly experimenting with forms of public engagement (e.g. citizen panels) in their efforts to set funding priorities, formulate funding calls, and evaluate project proposals (Den Oudendammer et al., 2019; Rowe et al., 2010). In this paper, ‘public engagement’ refers to the interaction with non-traditional stakeholders, such as citizens and civil society organizations (Dreyer et al., 2021; Kahane et al., 2013), in research and innovation activities led by RFOs. We recognize that the term may vary in meaning across different sectors, disciplines, and countries, but for clarity, we use ‘public engagement’ to encompass all forms of involvement RFOs might undertake. These experiments focus on including ‘non-traditional’ stakeholders such as citizens, communities, and civil society organizations, alongside ‘traditional’ ones like researchers, experts, and policymakers. These engagement processes mainly take the forms of communication, consultation, and participation, granting

the public a certain degree of power (Arnstein, 1969; Rowe & Frewer, 2000; Rowe & Frewer, 2005). However, communication and consultation are typically top-down processes, where public empowerment remains limited and participation is often more symbolic than substantive (IAP2, 2018).

Despite the potential of public engagement, several challenges hinder its success: these include issues related to recruitment, engagement ethics, and managing public expectations. Additionally, variations in research governance models across regions affect how and by whom engagement activities are conducted<fnref n="1" />. However, when executed effectively, public engagement can lead to more competitive and desirable outcomes (Fraaije & Flipse, 2020) and allow RFOs to explore societal values and worldviews that cannot be determined top-down or a priori (Bauer et al., 2021). Although the public may lack technoscientific expertise, they often possess complementary know-how and context-specific experiences that are crucial for solving societal challenges (Ravetz, 1999). For instance, informal caregivers may offer valuable insights into innovations that could improve a patient’s quality of life: such ‘experts by experience’ can thus improve the social robustness of innovation (Nowotny, 2003).

This emerging trend is inspired by various research fields, such as (Participatory) Technology Assessment (Durant, 1999; Schot & Rip, 1997), Responsible Research and Innovation (Stilgoe et al., 2013; Von Schomberg, 2013; Wiarda et al., 2021), Ethical Legal and Social Implications/Aspects Research (Fisher, 2005), and more. These fields endorse upstream public engagement on the basis of various normative, instrumental, and substantive grounds (Stirling, 2008; Wilsdon & Willis, 2004).

However, it is essential that public engagement is meaningful, fair, and effective (Ayre et al., 2018; Meskens, 2020). The uptake of engagement in research funding frequently raises ethical issues requiring explicit consideration (Van Bekkum & Hilton, 2014). Concerns may relate to issues like exploitation, vulnerability, representation, and inequality. These concerns often do not have a clear-cut solution and are exacerbated by practical constraints. Addressing these challenges requires explicit attention in both academia and practice.

Broader insights into participatory processes across various funding activities at both local and central government levels are well-documented in the scholarly literature. These processes, such as participatory budgeting and public involvement in budget formulation, have been explored in decentralized contexts (Nasution & Lutfi, 2022), local government practices (O’Hagan et al., 2020), and at the central government level (Ríos & Benito, 2017). In the specific context of research funding, the majority of studies focus on public engagement in health and medical research. Scholars have examined definitions of public engagement (van Bekkum et al., 2016), how patient and public

<footnote n="1">¹ In some regions, particularly Australia and the Indo-Pacific, public engagement is often delegated to research organizations.</footnote>
involvement can improve research design and funding applications in the health sector (McMillan et al., 2018), as well as the benefits of effectively implementing public engagement in health research funding organizations (Richards et al., 2024). Additionally, research highlights the importance of qualitative engagement to achieve rigor, representation, and reflexivity in funding initiatives (Rolfe et al., 2018), as well as challenges in involving patients in health and social care research grants (Foster et al., 2024). Studies also address patient engagement in cancer research funding allocation, including strategies for funding high-cost cancer drugs while managing trade-offs and fostering acceptance (Bentley et al., 2018; Taccone et al., 2023). The disproportionate focus on health and medical research in recent studies highlights the need for exploring broader public engagement across other sectors of research and innovation funding, which this study aims to address.

To explore these public engagement challenges, eight European RFOs from Austria (FFG), Belgium (Innoviris), Czech Republic (TACR), Germany (VDI/VDE), Lithuania (RCL), Norway (RCN), Romania (UEFISCDI), and Spain (CDTI) formed a consortium called PRO-Ethics. Over four years, they exchanged experiences on the ethical preparation, implementation, and evaluation of public engagement in research funding. This paper examines the difficulties these RFOs encountered in implementing public engagement processes, by describing these challenges in depth and illustrating how they manifest in practice.

While these European RFOs identified various ‘best practices’, this case study focuses on the challenges faced during implementation. These difficulties were first identified by RFOs through self-reflections and collectively discussed during three

‘cross-learning workshops’. This paper describes these challenges in depth, how they return in practice, and suggests how they can be addressed. The aim of this paper is to encourage academic efforts that could help RFOs organize public engagement more responsibly. In the following section, we will first describe seven major challenges to public engagement, after which we illustrate some of these through the example of VDI/VDE (German RFO), before suggesting recommendations to address these challenges.

## Challenges to ethical public engagement in research funding

Through a series of real-life experiments, several challenges to ethical public engagement have been reported by European RFOs. These challenges relate to seven themes (see Figure 1) that will be detailed in this section. While these challenges are presented in the context of RFOs, they are not unique to this sector. Many of the issues identified are experienced across the broader research and innovation community. Researchers, institutions, and other organizations engaging the public in any stage of the research cycle face similar hurdles, making these findings relevant beyond RFOs. The research question guiding this process was to identify, at a more granular level, challenges arising from the real-life implementation of public engagement across a diverse range of research and innovation cases funded by European RFOs. The following seven challenges were identified through a series of cross-learning workshops and discussions held among the eight participating RFOs. These workshops involved self-reflections and collective analyses of public engagement processes. Data was analyzed using an inductive thematic analysis, with verbatim transcriptions coded at the sentence and paragraph levels, providing the foundation for identifying key themes and challenges.

<figure><img src="image_10.png" /><figcaption>Figure 1. Seven main challenges identified in the ethical public engagement in research funding.</figcaption></figure>
## Challenge 1: recruiting participants

Recruiting participants for public engagement activities is a well-known challenge not only for RFOs but for the broader research community as well. RFOs strive to gather diverse public representation to ensure a wide range of perspectives in their public engagement activities (Van Bekkum & Hilton, 2014). Defining appropriate representation for a given context is often complex, as it must take into account various socio-economic factors such as background, education, age, religion, ethnicity, and gender identity (Den Oudendammer et al., 2019).

This selection process raises several questions such as intersectionality, since individuals may identify with multiple characteristics. Potential representatives might categorize themselves based on their own understanding of their identity, leading to possible criticism and concerns about unequal treatment. The concept of ‘appropriate’ public representation is therefore highly contested, and some argue that a representation accurately reflecting society may reinforce existing societal politics (Pratt, 2019). In some cases, an overrepresentation of minorities might even be necessary to mitigate power imbalances.

The context-specific nature of engagement means these challenges cannot be resolved with a standardized approach. Even when representation issues are addressed, potential participants may still be unwilling to engage. RFOs face a tension between which public actors should be represented and who is willing and able to participate, given their capacity and resources.

This challenge extends beyond RFOs, and there is a growing literature on how to increase diversity in public engagement across the entire research and innovation sector, covering for instance citizen science (Brouwer & Hessels, 2019) or biomedical research (Brown et al., 2023). In health and social care, for example, recruitment often focuses on individuals with ‘lived experience’ of specific health conditions. National resources such as the UK’s National Institute for Health and Care Research (NIHR, 2021) provide practical frameworks to support inclusivity in research engagement. These strategies are relevant for any researcher or organization aiming to foster more diverse public engagement.

In practice, RFOs often use pragmatic approaches such as snowball sampling and partnering with multiplier organizations to recruit participants. While these methods can mitigate bias, they cannot fully eliminate it (Viswanathan et al., 2004). To address this, RFOs should collaborate with social scientists or independent research experts in the design of engagement processes that reduce various forms of bias and potential conflicts of interest.

## Challenge 2: commitments and expectations

Managing commitments and expectations is a significant challenge for RFOs because their goals often differ from those of the public. Participants might have different views on how much influence they should have on research funding decisions, leading to potential disagreements and causing some to decline or withdraw from engagement processes.

Even when expectations align, participants’ commitments might still suffer if RFOs do not adequately accommodate their needs. Participation often depends on specific engagement forms and whether these align with participants’ characteristics—such as disabilities—, and resources—such as time. For example, participants with disabilities may require particular accommodations, and those with limited time may find it hard to commit to lengthy processes.

To address these issues, RFOs find it helpful to identify and communicate everyone’s needs and expectations regarding the roles, scope, purpose, process, and outcomes of the engagement before launching the process. Establishing clear codes of conduct can also help ensure that all participants understand and agree on these aspects. By explicitly addressing and aligning commitments and expectations, RFOs can foster more effective and inclusive public engagement.

## Challenge 3: meaningful dialogue and equal engagement

Ensuring meaningful dialogue and equal engagement is crucial for obtaining the public’s input, but it is often challenged by heterogeneous perspectives that can lead to misinterpretation and conflict. In deliberative formats, discussions between public representatives and traditional stakeholders are particularly prone to power imbalances. Some stakeholders may dominate conversations due to their personalities, knowledge, or institutional roles, creating an imbalance (e.g., citizens vs. scientists). Such imbalances can disrupt the dialogue and limit the involvement of less dominant participants.

To address these challenges, several RFOs adopt strategies to mitigate knowledge-based power imbalances. These strategies include thematic warm-ups to prepare participants with introductory sessions on the topic to level the knowledge field and information management by selectively providing or withholding information to ensure all participants have a more equal understanding of the subject (Zapata, 2009). Additionally, the use of neutral mediators (top-down moderation) helps guide dialogues, manage conflicts, and encourage participation from less vocal representatives (Rubinelli & von Groote, 2017). Media-tors play a key role in neutralizing power discrepancies during discussions and ensuring balanced deliberations (Davies, 2013).

The effectiveness of these strategies hinges on the presence of mutual trust between stakeholders and the mediators. Trust ensures that dialogues are constructive and inclusive, allowing for balanced and equitable engagement.

## Challenge 4: accommodating vulnerability

Engagement challenges also extend to the inclusion of vulnerable groups (Brown et al., 2017). This issue is particularly prevalent when funding processes address real-life problems, as public actors affected by these problems may face social injustice, financial issues, or other disadvantages. In these contexts, vulnerability is difficult to define and understand (Amann & Sleigh, 2021; Latif et al., 2018). It is helpful to consider
aspects contributing to participants' vulnerability, such as their resources, capabilities, experiences, and identities.

Participants generally have a better understanding of their own vulnerabilities. Therefore, it can be beneficial for RFOs to rely on participants' self-assessments rather than making assumptions themselves. One essential aspect of addressing vulnerability is promoting inclusion through compensation. Providing financial or non-financial compensation for participants' time and expenses is a recognized standard for public engagement. Compensation helps remove barriers to engagement, particularly for individuals from diverse socio-economic backgrounds, and ensures that vulnerable participants can contribute without facing additional financial burdens.

RFOs should adopt compensation as part of their public engagement activities, alongside other measures such as providing translators and improving accessibility. These efforts will ensure that vulnerable groups are fully included and supported throughout the engagement process.

### Challenge 5: funding call formulations

RFOs develop calls and strategies to allocate public funding to recipients such as universities, consortia, and researchers (Lepori et al., 2023). Public engagement in this context often involves formulating these funding calls and strategies. RFOs frequently struggle to meaningfully involve both traditional stakeholders (e.g., scientists) and public participants (e.g., citizens) in creating calls and strategies that are scientifically and socially relevant (Den Oudendammer et al., 2019). RFOs generally consider three options for engagement: (1) public participants suggest strategies and calls, which are then selected and scientifically embedded by traditional stakeholders; (2) traditional stakeholders propose strategies and calls, which are then selected and contextualized by public participants; or (3) the proposal, selection, contextualization, and scientific embedding are done collectively.

Each option has its own advantages and disadvantages. The experience of RFOs in our consortium indicates that collective interactions (option 3) often lead to power imbalances due to differences in social status and expertise. When public participants propose socially relevant strategies and calls, their scientific relevance is often perceived as low. Conversely, allowing traditional stakeholders to suggest strategies and calls, followed by selection and contextualization by public participants (option 2), has proven helpful. However, this approach risks turning into tokenism if public participants have limited decision-making power. Therefore, the appropriate engagement method is context-dependent.

### Challenge 6: lack of expertise in engagement ethics

Ensuring the ethical soundness of public engagement requires an expertise in both ethics and public engagement. While the associated skills and knowledge may improve engagement, RFOs often lack this expertise (Giannelos et al., 2022). However, it is important to clarify that ethical engagement does not necessarily require the creation of a specialized field of

'engagement ethics'. Instead, ethical public engagement should be built on professionalism, fairness, transparency, and the availability of adequate resources. These principles should guide all public engagement activities, rather than relying solely on specialized ethical expertise.

While external experts, such as ethicists and facilitators, can contribute to improving engagement quality, over-reliance on this external expertise can create unnecessary barriers to participation and responsibility. All professionals involved in engagement should embrace ethical standards through values such as respect and fairness, rather than considering ethics as a specialized domain. This ensures that ethical engagement is not seen as a complex or burdensome process.

RFOs should focus on using flexible, 'learning-by-doing' approaches that remain open to feedback from participants, while adhering to established protocols, guidelines, and codes of conduct. Rather than creating additional layers of expertise, public engagement can be safeguarded by adopting these core principles and utilizing frameworks such as the UK Standards for Public Involvement. This allows ethical public engagement without imposing unnecessary burdens or specialized expertise.

### Challenge 7: uncertainty, resource constraints, and external factors

Even when the previously mentioned challenges are addressed, ethical engagement can still suffer from organizational constraints (Amann & Sleigh, 2021). Comprehensive planning for engagement is beneficial, but RFOs often deal with high degrees of uncertainty. Nearly all RFOs in our consortium found their engagement processes more resource-consuming than initially anticipated.

Moreover, a wide range of external factors, such as regulations, significantly impact how RFOs prepare, implement, and evaluate public engagement. For example, some RFOs are required to follow strict governmental protocols that can make public engagement more rigid and less adaptable. These uncertainties, resource constraints, and external factors necessitate a high degree of organizational flexibility, which is challenging to achieve.

To navigate these issues, RFOs must develop adaptive strategies and be prepared to adjust their plans and resources as needed, ensuring that engagement processes remain ethical and effective despite external pressures.

### Challenges in practice: the case of an RFO

The seven aforementioned challenges are difficult to address by RFOs. In relation to the first challenge, engaging experienced recruiters might alleviate some of these challenges, but it is unlikely to resolve them all. The diverse and often conflicting expectations between RFOs and the public, coupled with the difficulty of accommodating participants' varying needs and vulnerabilities, make managing commitments and expectations a significant hurdle. Ensuring meaningful dialogue and
equal engagement is complicated by power imbalances and the need for effective mediation. Accommodating vulnerable groups requires a nuanced understanding and sensitive approach, which can be resource-intensive. Formulating funding calls that balance scientific and social relevance involves navigating complex stakeholder dynamics. The lack of expertise in engagement ethics further complicates the process, necessitating external input and ongoing learning. Lastly, the unpredictability of resource constraints and external factors such as regulatory requirements adds another layer of complexity. These multifaceted challenges necessitate adaptive, context-specific strategies and highlight the ongoing need for innovative solutions to foster ethical and effective public engagement in research funding.

To illustrate these challenges, we use the example of a German RFO (VDI/VDE Innovation + Technik GmbH, hereafter ‘VDI/VDE’) tasked with developing a call for project proposals to support informal caregivers through interactive technologies. The RFO set up a citizen advisory board consisting of 15 caregivers, responsible for evaluating and selecting proposals. This RFO faced several challenges, including difficulty in recruiting diverse caregivers, managing expectations and commitments, and accommodating vulnerabilities.

Recruiting caregivers was difficult due to selection bias (challenge 1) and potential participants’ previous negative experiences with administrative bodies. Time constraints limited the input some caregivers could provide, exacerbating knowledge-based power imbalances. Additionally, mismatching expectations and rigid protocols demotivated some participants (challenge 2). What is more, joining the advisory board would inhibit them from providing care and would thus put their patient(s) at risk (challenge 4). Few caregivers could afford a substitute caregiver and as a response, VDI/VDE reimbursed any care expenses that caregivers incurred. Compensation was also available for travel and accommodation costs. Nevertheless, the RFO’s administrative processes caused substantial delays of up to several weeks – in the reimbursements, again imposing a financial burden on some of the caregivers.

Also, some board members were consequently more familiar with the projects than others, thus exacerbating knowledge-based power imbalances (challenge 3). Several caregivers were therefore unable to convincingly voice their opinions, and a few acknowledged feeling undervalued or even intimidated. Other caregivers deemed their responsibilities too limited and pleaded for greater influence when mentoring projects. For example, some caregivers provided their professional expertise (i.e., knowledge of IT and engineering) even though they were invited by the board to provide their experiences as caregivers. As a result, rigid protocols (challenge 7) and mismatching expectations (challenge 2) may have demotivated some participants.

This real-life case illustrates how a single public engagement process by a research and innovation funding organization can encounter multiple, overlapping challenges simultaneously. It underscores the importance of providing targeted guidance to research and innovation funders on how to effectively navigate

and address these diverse challenges in public engagement. The following section will provide actionable recommendations to help research and innovation funders effectively address these challenges and foster more inclusive, ethical, and efficient public engagement practices.

## Discussion

The seven challenges identified from the aggregated feedback received by the RFOs during their public engagement processes underscore the need for context-specific guidance. Each difficulty encountered necessitates careful attention and scrutiny tailored to the specific circumstances of each case.

European RFOs are increasingly experimenting with public engagement in funding processes, aspiring to uphold ethical values such as justice, equality, and safety. Despite the benefits, challenges related to ethical public engagement persist, requiring context-adaptive insights and tools to meaningfully and inclusively engage the public. Despite decades of research on ethics and upstream engagement, RFOs still face challenges that are too context-specific to be addressed in a standardized manner. These challenges create gaps between how public engagement in research funding *should* be organized and *can* be organized.

Based on the analysis of our seven challenges faced by RFOs in ethical (therefore also inclusive) public engagement, several recommendations can be made to improve their current practices: see Table 1, below.

### Recommendation 1: recruitment strategies with experienced recruiters and community organizations

RFOs should develop comprehensive recruitment strategies that leverage the expertise of professional recruiters, social scientists, and local community organizations to ensure diverse and representative participation. Social scientists, for instance, can play a dual role: not only in designing recruitment strategies that reduce bias but also in brokering partnerships between stakeholders and facilitating effective engagement processes. These strategies should include thorough analyses to identify underrepresented groups, implementing outreach programs tailored to the specific needs and preferences of diverse populations, offering incentives and removing participation barriers such as providing compensation for the time dedicated. Additionally, this would involve establishing long-term relationships with community leaders and organizations to build trust and ensure sustained engagement.

### Recommendation 2: Clear communication of roles, expectations, and outcomes through codes of conduct

RFOs should establish and disseminate detailed codes of conduct that clearly define the roles, expectations, and outcomes for all participants. This would typically include hosting preliminary orientation sessions to explain the engagement process and the targeted objectives. Additionally, it would involve providing written guidelines to ensure all participants understand their commitment and the impact of their involvement, and regularly updating participants on progress and outcomes. Furthermore, creating feedback mechanisms to allow all
<table><caption>**Table 1. Recommendations for ethical public engagement in response to the challenges detected.**</caption><thead><tr><td>Challenges</td><td>Recommendations</td></tr></thead><tbody><tr><td>Recruiting participants</td><td>Recruitment strategies with experienced recruiters and community organizations</td></tr><tr><td>Commitments and expectations</td><td>Clear communication of roles, expectations, and outcomes through codes of conduct</td></tr><tr><td>Meaningful dialogue and equal engagement</td><td>Training of mediators to address power imbalances</td></tr><tr><td>Accommodating vulnerability</td><td>Flexible engagement methods and tailored support</td></tr><tr><td>Funding call formulations</td><td>Collaborative feedback loops for inclusive funding call formulation</td></tr><tr><td>Lack of expertise in engagement ethics</td><td>Enhancing ethical standards through internal expertise and external advisory inputs</td></tr><tr><td>Uncertainty, resource constraints, and external factors</td><td>Developing adaptive strategies for flexible and ethical public engagement</td></tr></tbody></table>

participants to voice any potential concerns or suggestions would ensure ongoing alignment and adaptation of expectations on both sides.

### Recommendation 3: training of mediators to address power imbalances

RFOs should invest in comprehensive training for mediators to foster equitable participation, or perhaps outsource this activity. This includes providing mediators with proper training in conflict resolution, active listening, and cultural competencies. To achieve balanced participation, structured dialogue techniques are essential. Examples of such techniques might include small group discussions, thematic warm-ups, or break-out sessions, depending on the context. A wide variety of other methods exist and the appropriate approach depends on the specific engagement scenario. Regular assessment and refinement of mediation strategies through participant feedback and mediator reflections would also be essential.

### Recommendation 4: flexible engagement methods and tailored support

RFOs should design flexible engagement methods that accommodate the needs of vulnerable groups by offering various participation formats (e.g., in-person, virtual, hybrid), so as to take into consideration different preferences and constraints. This would imply providing tailored support such as sign language interpreters, accessible venues, and assistive technologies. Additionally, ensuring financial compensation for participants' time and expenses could also be beneficial, as is establishing dedicated support teams to assist vulnerable participants throughout the engagement process.

### Recommendation 5: collaborative feedback loops for inclusive funding call formulation

RFOs should implement collaborative feedback loops that integrate input from public participants and traditional stakeholders

at multiple stages of the funding call formulation process. This could be achieved by conducting joint workshops and focus groups to co-create funding priorities and call criteria. Additionally, implementing iterative review cycles where drafts of funding calls are shared with stakeholders for feedback and improvement would be beneficial. Utilizing digital platforms to facilitate continuous input and collaboration would help ensure all voices are heard. Furthermore, ensuring transparency in how public input is integrated into the final funding calls, with clear explanations of the final decisions made, would also contribute to this aim.

### Recommendation 6: enhancing ethical standards through internal expertise and external advisory inputs

RFOs can enhance their ethical standards by fostering professionalism and embracing core values such as fairness, equality, and respect in their public engagement activities. Rather than establishing dedicated engagement ethics committees, which might overlap with the role of research ethics committees, RFOs can adopt streamlined processes that ensure ethical public engagement while maintaining flexibility.

This can be achieved by providing ongoing ethics training for staff involved in public engagement activities and fostering a culture of ethical awareness. Collaborating with external advisory boards or ethicists can be useful to review and enhance engagement strategies where needed, helping ensure continuous alignment with ethical principles. Regular reviews of engagement practices, through flexible audits or assessments, can support ongoing improvements and maintain high standards of ethical engagement.

If more structured oversight is considered beneficial, RFOs could explore establishing a shared advisory board accessible to multiple organizations, ensuring a consistent approach while avoiding duplication of efforts across individual RFOs.
## Recommendation 7: developing adaptive strategies for flexible and ethical public engagement

RFOs should develop adaptive strategies to manage uncertainty and resource constraints while maintaining ethical standards. This target could be achieved by creating flexible engagement frameworks that can be adjusted based on emerging needs and feedback, to address unforeseen challenges and ensure the continuity of engagement activities, to monitor and respond to external factors (e.g., regulatory changes, societal shifts), and regularly reviewing and updating engagement plans to incorporate lessons learned and best practices.

Implementing these practical recommendations could help RFOs bridge the existing gaps, in view of ethical public engagement. This could in turn contribute to fostering more ethical and effective funding practices. These findings should be considered with an understanding of their limited generalizability, given our purposive sampling strategy.

## Concluding remarks

European Research Funding Organizations (RFOs) face multiple, interrelated challenges in executing ethical public engagement. These challenges stem from recruiting a representative public, managing diverse commitments and expectations, ensuring meaningful dialogue, accommodating vulnerable participants, formulating inclusive funding calls, and addressing a lack of engagement ethics expertise—all within the constraints of limited resources and external pressures. To bridge these gaps, RFOs must develop context-specific strategies and adopt adaptive, flexible approaches that prioritize ethical standards and inclusivity.

In response to these challenges, we have provided recommendations for each of the challenges. These seven recommended practices—leveraging experienced recruiters, clear communication through codes of conduct, mediator training, flexible engagement methods, collaborative feedback loops, enhancing ethical standards, and developing adaptive strategies—could help RFOs enhance their engagement practices. These improvements can lead to more ethical, and effective public participation, ultimately fostering a more just and equitable research funding process.

Further, RFOs can draw on established resources and experiences from organizations in fields such as health and social care, as well as from international examples, to guide and enhance their public involvement strategies. Collaborative efforts between academia, funders, and public involvement practitioners can play a critical role in addressing these persistent challenges. Iterative learning and cross-sectoral cooperation will help RFOs bridge the gap between the ideal and practical organization of public engagement, ensuring that ethical, and effective practices become embedded in the research funding process.

## Ethical approval and consent

Ethical approval and consent were not required.

## Data availability

The primary data discussed in this paper, which served as the basis for our analysis and reflections, originates from the Pilots of PRO-Ethics, an H2020 project. These Pilots were internal to the consortium, and the results are publicly available on the project's website. See: https://pro-ethics.eu/outputs; and https://pro-ethics.eu/pilots/

## Author contributions

- Kalli Giannelos: conceptualization; supervision; writing (original draft); writing (review & editing)

- Martijn Wiarda: conceptualization; supervision; writing (original draft); writing (review & editing)

- Neelke Doorn: conceptualization; supervision; validation; writing (original draft); writing (review & editing)

## Acknowledgments

The authors of this manuscript would like to thank all the members of the PRO-Ethics consortium for their collaboration on what served as the initial basis of this study. Throughout this collaboration, feedback, iterative loops, and constructive dialogue served to outline the challenges described in this paper, test the assumptions, and refine the observations.

## References

Amann J, Sleigh J: Too vulnerable to involve? Challenges of engaging vulnerable groups in the co-production of public services through research. *Int J Pub Admin*. 2021; 44(9): 715-727.

### Publisher Full Text

Arnstein SR: A ladder of citizen participation. *J Am Ins Plann*. 1969; 35(4): 216-224.

### Publisher Full Text

Ayre ML, Wallis PJ, Daniell KA: Learning from collaborative research on sustainably managing fresh water: implications for ethical research-practice engagement. *Ecol Soc*. 2018; 23(1): 6.

### Publisher Full Text

Bauer A, Bogner A, Fuchs D: Rethinking societal engagement under the heading of Responsible Research and Innovation: (novel) requirements

and challenges. *J Res Innov*. 2021; 8(3): 342-363.

### Publisher Full Text

Bentley C, Costa S, Burgess MM, et al.: Trade-offs, fairness, and funding for drugs: key findings from a deliberative public engagement event in British Columbia, Canada. *BMC Health Serv Res*. 2018; 18(1): 339.

### PubMed Abstract | Publisher Full Text | Free Full Text

Brouwer S, Hessels LK: Increasing research impact with citizen science: the influence of recruitment strategies on sample diversity. *Public Underst Sci*. 2019; 28(5): 606-621.

### PubMed Abstract | Publisher Full Text

Brown K, Ecclestone K, Emmel N: The many faces of vulnerability. *Soc Pol Soc*. 2017; 16(3): 497-510.

### Publisher Full Text
Brown KE, Fohner AE, Woodalh EL: Beyond the individual: community-centric approaches to increase diversity in biomedical research. *Clin Pharmacol Ther.* 2023; **113**(3): 509-517.

PubMed Abstract | Publisher Full Text

Davies SR: The rules of engagement: power and interaction in dialogue events. *Public Underst Sci.* 2013; **22**(1): 65-79.

PubMed Abstract | Publisher Full Text

Den Oudendammer WM, Noordhoek J, Abma-Schouten RY, et al.: Patient participation in research funding: an overview of when, why and how amongst Dutch health funds. *Res Involv Engagem.* 2019; **5**(1): 33.

PubMed Abstract | Publisher Full Text | Free Full Text

Dreyer M, Kosow H, Bauer A, et al.: Public engagement with research: citizens' views on motivations, barriers and support. *Research for All.* 2021; **5**(2): 302-19.

Publisher Full Text

Durant J: Participatory technology assessment and the democratic model of the public understanding of science. *Sci Pub Pol.* 1999; **26**(5): 313-319.

Publisher Full Text

Fisher E: Lessons learned from the Ethical, Legal and Social Implications program (ELSI): planning societal implications research for the National Nanotechnology Program. *Technol Soc.* 2005; **27**(3): 321-328.

Publisher Full Text

Foster A, Caunt S, Schofield H, et al.: Evaluating a grant development public involvement funding scheme: a qualitative document analysis. *Res Involv Engagem.* 2024; **10**(1): 57.

PubMed Abstract | Publisher Full Text | Free Full Text

Fraaije A, Flipse SM: Synthesizing an implementation framework for responsible research and innovation. *J Respons Innov.* 2020; **7**(1): 113-137.

Giannelos K, Reber B, Doorn N: Responsive ethics and participation: science, technology and democracy. London/New York: ISTE Ltd/John Wiley & Sons, 2022.

Publisher Full Text

International Association for Public Participation (IAP2): IAP2's Public Participation Spectrum. 2018; [Accessed 16 October 2024].

Reference Source

Kahane D, Lopstone K, Herriman J, et al.: Stakeholder and citizen roles in public deliberation. *Journal of Public Deliberation.* 2013; **9**(2).

Publisher Full Text

Latif A, Tariq S, Abbasi N, et al.: Giving voice to the medically under-served: a qualitative co-production approach to explore patient medicine experiences and improve services to marginalized communities. *Pharmacy (Basel).* 2018; **6**(1): 13.

PubMed Abstract | Publisher Full Text | Free Full Text

Lepori B, Jongbloed B, Hicks D: Introduction to the handbook of public funding of research: understanding vertical and horizontal complexities. In: B. Lepori, B. Jongbloed, & D. Hicks (Eds.), *Handbook of Public Funding of Research*. Massachusetts: Edward Elgar Publishing, 2023; 1-19.

Publisher Full Text

McMillan B, Fox S, Lyons M, et al.: Using patient and public involvement to improve the research design and funding application for a project aimed at fostering a more collaborative approach to the NHS health check: the CaVIAR project (better Care Via Improved Access to Records). *Res Involv Engagem.* 2018; **4**: 18.

PubMed Abstract | Publisher Full Text | Free Full Text

Meskens G: Ethical considerations on stakeholder engagement in radiological risk governance. *Radioprotection.* 2020; **55**: S259-S262.

Nasution MM, Lutfi A: Public involvement in budget transparency in terms of opportunities and challenges. *Public Administration Journal.* 2022; **12**(2): 170-179.

Publisher Full Text

National Institute for Health and Care Research (NIHR): Reaching Out: a practical guide to being inclusive in public involvement in health research. 2021; [Accessed 17 October 2024].

Reference Source

Nowotny H: Democratising expertise and socially robust knowledge. *Sci Pub Pol.* 2003; **30**(3): 151-156.

Publisher Full Text

O'Hagan A, MacRae C, Hill O'Connor C, et al.: Participatory budgeting, community engagement and impact on public services in Scotland. *Public Money Manage.* 2020; **40**(6): 446-456.

Publisher Full Text

Pratt B: Constructing citizen engagement in health research priority-setting to attend to dynamics of power and difference. *Dev World Bioeth.* 2019; **19**(1): 45-60.

PubMed Abstract | Publisher Full Text

Ravetz JR: What is Post-Normal Science. *Futures.* 1999; **31**: 647-653.

Reference Source

Richards DP, Twomey R, Flynn T, et al.: Patient engagement in a Canadian health research funding institute: implementation and impact. *BMJ Open.* 2024; **14**(7): e082502.

PubMed Abstract | Publisher Full Text | Free Full Text

Ríos AM, Benito B: Factors explaining public participation in the central government budget process. *Aust J Publ Admin.* 2017; **76**(1): 48-64.

Publisher Full Text

Rolfe D, Ramsden VR, Banner D, et al.: Using qualitative Health Research methods to improve patient and public involvement and engagement in research. *Res Involv Engagem.* 2018; **4**: 49.

PubMed Abstract | Publisher Full Text | Free Full Text

Rowe G, Frewer LJ: Public participation methods: a framework for evaluation. *Sci Technol Hum Val.* 2000; **25**(1): 3-29.

Publisher Full Text

Rowe G, Frewer LJ: A typology of public engagement mechanisms. *Sci Technol Hum Val.* 2005; **30**(2): 251-290.

Publisher Full Text

Rowe G, Rawsthorne D, Scarpello T, et al.: Public engagement in research funding: a study of public capabilities and engagement methodology. *Public Underst Sci.* 2010; **19**(2): 225-239.

PubMed Abstract | Publisher Full Text

Rubinelli S, von Groote PV: Stakeholder dialogue as a deliberation for decision making in health policy and systems: the approach from argumentation theory. *Am J Phys Med Rehabil.* 2017; **96**(2 Suppl 1): S17-S22.

PubMed Abstract | Publisher Full Text

Schot J, Rip A: The past and future of constructive technology assessment. *Technol Forecast Soc Change.* 1997; **54**(2-3): 251-268.

Publisher Full Text

Stilgoe J, Owen R, Macnaghten P: Developing a framework for responsible innovation. *Res Pol.* 2013; **42**(9): 1568-1580.

Publisher Full Text

Stirling A: "Opening up" and "closing down": power, participation, and pluralism in the social appraisal of technology. *Sci Technol Hum Val.* 2008; **33**(2): 262-294.

Publisher Full Text

Taccone MS, Baudais N, Wood D, et al.: Co-creation of a patient engagement strategy in cancer research funding. *Res Involv Engagem.* 2023; **9**(1): 86.

PubMed Abstract | Publisher Full Text | Free Full Text

Van Bekkum JE, Fergie GM, Hilton S: Health and medical research funding agencies' promotion of public engagement within research: a qualitative interview study exploring the United Kingdom context. *Health Res Policy Syst.* 2016; **14**: 23.

PubMed Abstract | Publisher Full Text | Free Full Text

Van Bekkum JE, Hilton S: UK research funding bodies' views towards public participation in health-related research decisions: an exploratory study. *BMC Health Serv Res.* 2014; **14**(1): 318.

PubMed Abstract | Publisher Full Text | Free Full Text

Viswanathan M, Ammerman A, Eng E, et al.: Community-based participatory research: assessing the evidence. *Evid Rep Technol Assess (Summ).* AHRQ Publication N° 4-E022-1. Agency for Healthcare Research and Quality, Rockville, MD, 2004; (99): 1-8.

PubMed Abstract | Free Full Text

Von Schomberg R: A vision of responsible research and innovation. In: R. Owen, J. Bessant, & M. Heintz (Eds.), *Responsible Innovation: Managing the Responsible Emergence of Science and Innovation in Society*. Newark: John Wiley & Sons, 2013.

Publisher Full Text

Wiarda M, van de Kaa G, Yaghmaei E, et al.: A comprehensive appraisal of responsible research and innovation: from roots to leaves. *Technol Forecast Soc Change.* 2021; **172**: 121053.

Publisher Full Text

Wilsson D, Willis R: See-through science. Why public engagement needs to move upstream. London: Demos, 2004.

Publisher Full Text

Zapata MA: Deliberating across differences: planning futures in cross-cultural spaces. *Policy Soc.* 2009; **28**(3): 197-209.

Publisher Full Text
# Open Peer Review

Current Peer Review Status: ✅ ? ✅

## Version 2

Reviewer Report 08 November 2024

https://doi.org/10.21956/openreseurope.20362.r45900

© 2024 Tauginienė L. This is an open access peer review report distributed under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

<figure><img src="image_11.png" /></figure>

Kazimieras Simonavicius University, Vilnius, Lithuania

Thank you for the opportunity to evaluate the revised manuscript.

The suggestions were well addressed.

**Competing Interests:** No competing interests were disclosed.

**Reviewer Expertise:** citizen science, social responsibility, RRI, research ethics, research integrity, public engagement

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Reviewer Report 08 November 2024

https://doi.org/10.21956/openreseurope.20362.r45899

© 2024 Carter L. This is an open access peer review report distributed under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

<figure><img src="image_12.png" /></figure>

CSIRO Environment, Brisbane, Queensland, Australia

Thank you to the authorship team who've provided comprehensive responses to reviewers' remarks. The paper is much more impactful now.

**Competing Interests:** No competing interests were disclosed.
**Reviewer Expertise:** Social science; engagement research and practice; interdisciplinary research approaches

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

## Version 1

Reviewer Report 21 October 2024

https://doi.org/10.21956/openreseurope.19594.r44961

© 2024 Tauginienė L. This is an open access peer review report distributed under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

<figure><img src="image_13.png" /></figure>

Loreta Tauginienė

<figure><img src="image_14.png" /></figure>

Kazimieras Simonavicius University, Vilnius, Lithuania

The article is interesting and relevant to practitioners and other readers. It is well-structured and reads with ease.

Here are some suggestions for further improvement.

**Section 1.** The readers might benefit from the concise literature review on public engagement (PE) in funding from a broader perspective (e.g. practice of municipalities). This might be accommodated either in the Section 1 or made as a separate section. Please also consider including the latest findings on PE in research funding.

**Section 2.** It is unclear how challenges were identified in methodological terms. What were methods of data collection used? How was the data collected analyzed? Any limitations? Then, research questions are unclear.

The relevance and scope of each challenge is well described combining data of the project partners with supportive references.

**Section 3.** The real-life case is well-described. It addresses most of challenges. Any lessons learned that might be useful for the next section on recommendations?

**Section 4.** It is suggested considering adding "and inclusive" (*ethical and inclusive engagement*) because it appears to have the same importance here.

**Other observations:** Statements and Declarations should be complemented with *Conflict of Interest*.

**Is the background of the case's history and progression described in sufficient detail?**

No

**Is the work clearly and accurately presented and does it cite the current literature?**

No

**If applicable, is the statistical analysis and its interpretation appropriate?**
Not applicable

**Are all the source data underlying the results available to ensure full reproducibility?**

No

**Are the conclusions drawn adequately supported by the results?**

Yes

**Is the case presented with sufficient detail to be useful for teaching or other practitioners?**

Yes

**Competing Interests:** No competing interests were disclosed.

**Reviewer Expertise:** citizen science, social responsibility, RRI, research ethics, research integrity, public engagement

**I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.**

Author Response 25 Oct 2024

**Kalli GIANNELOS**

Dear Reviewer, We sincerely thank you for your detailed and constructive feedback. Your valuable insights have significantly contributed to improving our manuscript, and we are grateful for the opportunity to address them. Below is a summary of the revisions made in response to your suggestions:

**Comment:** "Section 1. The readers might benefit from the concise literature review on public engagement (PE) in funding from a broader perspective (e.g. practice of municipalities). This might be accommodated either in the Section 1 or made as a separate section. Please also consider including the latest findings on PE in research funding."

**Response:** Thank you for highlighting this. We have added a paragraph in Section 1 that provides a concise literature review on public engagement in funding, with a focus on broader examples such as participatory budgeting, which is a key area of study. Additionally, we clarified the current state of research on public engagement in research funding, emphasizing the prevalence of studies in the health and biomedical sectors. This important finding supports the relevance of our work, as it broadens the existing scholarly literature by examining public engagement in research and innovation funding across a wider range of sectors.

**Comment:** "Section 2. It is unclear how challenges were identified in methodological terms. What were methods of data collection used? How was the data collected analyzed? Any limitations? Then, research questions are unclear. The relevance and scope of each challenge is well described combining data of the project partners with supportive references." **Response:** The revised text at the beginning of section 2 now clarifies the methodology used for both data collection
and analysis, outlines the research question, and a sentence at the end of the Discussion (section 4) mentions the limitations of this approach.

**Comment:** "Section 3. The real-life case is well-described. It addresses most of challenges. Any lessons learned that might be useful for the next section on recommendations?" **Response:** In response, we have added a section discussing the lessons learned from the real-life case. This addition highlights how a single public engagement process by a research and innovation funding organization can face multiple, overlapping challenges simultaneously. We have also emphasized the need for providing targeted guidance to funders on how to effectively navigate these challenges to ensure more inclusive, ethical, and efficient public engagement practices (which is addressed in the subsequent section).

**Comment:** "Section 4. It is suggested considering adding "and inclusive" (ethical and inclusive engagement) because it appears to have the same importance here."

**Response:** Thank you for your suggestion. At the beginning of Section 4, we have clarified that ethical public engagement inherently includes inclusivity, emphasizing that a public engagement process cannot be considered ethical without being inclusive. This distinction highlights 'ethical' as an overarching category that encompasses inclusivity.

**Comment:** "Other observations: Statements and Declarations should be complemented with Conflict of Interest."

**Response:** There are no conflicts of interest related to this paper. The Statements and Declarations section also addresses the typical concerns regarding competing interests, ethical approval, and data availability. We hope these revisions adequately address your concerns and further improve the clarity and relevance of the manuscript. Once again, thank you for your invaluable feedback.

**Competing Interests:** No competing interests were disclosed.

Reviewer Report 10 October 2024

https://doi.org/10.21956/openreseurope.19594.r43871

© 2024 Hatch S. This is an open access peer review report distributed under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

<figure><img src="image_15.png" /></figure>

Sarah Hatch

School of Medicine, Cardiff University, Cardiff, Wales, UK

Thank you for providing an opportunity to review this paper. I feel this case study offers important learning points for the research community. I hope the following comments are helpful to the authors in strengthening this work.

Regarding terminology used in this case study – be mindful that in certain sectors and in different countries ‘public engagement’ can mean different things to different people. There is a lot of
literature available on this. In health and social care research in the UK, for example, public
involvement and public engagement are both used and mean different things. At the outset of the
paper, it maybe helpful to define your meaning of 'public engagement' to improve understanding
across disciplines, sectors, countries etc.

The seven highlighted challenges exist for a much wider range of organizations and have a much
broader scope than just Research Funding Organisations.

Challenge 1 - recruiting members of the public to get involved in any aspect of research is a
challenge for the research community as a whole not just for RFOs. There is growing literature on
how to encourage a more diverse public to get involved in the research cycle process which would
be relevant to RFOs or any organisation/researcher wanting to involve members of the public in
their work.

When working with health and care researchers, the priority is to attract representation from
members of the public who have ‘lived experience’ of a particular health condition/social care
aspect that is being researched. Certain essential criteria need to be clearly articulated in a role
description which then can be promoted widely to identified key stakeholders including charities;
community groups; libraries; companies who facilitate contact with hard-to-reach groups.
For RFOs, fund scope/area will help to inform appropriate representation, so selection is not just
based on different socio-economic factors.

There are a range of resources out there (that could be referenced) to support inclusivity in
research. For example, [Reaching Out: A guide to being inclusive in public involvement](http://learningforinvolvement.org.uk)
and [learningforinvolvement.org.uk](http://learningforinvolvement.org.uk)

In the UK, we also have the UK Standards for Public Involvement which include six standards
which make up a framework for what good public involvement in research looks like. These can be
adapted to different situations and are designed to be used as a guide/tool to help researchers
and research organisations identify what they are doing well and what needs improving. The six
standards are focussed on Inclusive Opportunities; Working Together; Impact; Governance;
Communications; Support and Learning. Similar guidance maybe available in other European
countries?

Availability of budget often dictates how much involvement will be possible (numbers of people to be recruited to take part etc).

Challenge 2 is included in the 'Working Together' standard of the UK Standards for Public Involvement.

## Challenge 6

Engagement ethics. Use of the word 'ethics' here could be confusing. Professionalism and adequate resource (human and financial) are key to delivering quality public engagement. This does not rely on specific expertise but practitioners who conduct themselves and their work in a professional manner and have access to adequate resource to support their engagement work.

**Challenges in practice: The case of an RFO**

'The lack of expertise in engagement ethics further complicates the process, necessitating external input and ongoing learning'.

My concern here is that you are creating an unnecessary barrier to conducting public
involvement/engagement? Creating a field of expertise 'engagement ethics' lends itself to being a
specialised area requiring expert knowledge. I believe that this is potentially misleading. All
working professionals should take responsibility for going about their work in a professional and
ethical manner embracing values of 'respect' 'fairness'; and 'transparency'. I worry that stating a
'lack of engagement ethics expertise' could easily be presented as an excuse for not doing public
engagement work, effectively acting as an additional barrier.
Case Study - highlights 'involvement' challenges experienced across the sector. These are not new and happen outside of RFOs.

'Ensuring financial compensation for participants' time and expenses could also be beneficial' - To support inclusivity offering compensation (financial or non-financial) is a recognised UK Standard for public involvement that all researchers/organisations should adopt when planning a public involvement activity with patients and members of the public.

**Recommendation 6 - Enhancing ethical standards through internal expertise and external advisory inputs**

Once again, my concern here is that this could potentially create an unnecessary barrier and should not be needed if organisations operate professionally embracing values of fairness, equality, respect etc.

Ethical public engagement - This term raises several queries:-

* I fear the use of this term will cause confusion in a growing debate concerning Public Involvement and Engagement versus Research Ethics (from higher education institutional research ethics committees). Public involvement and public engagement in research does not require research ethics but should be planned and conducted in a professional manner embracing EDI principles and taking into account risk. Public Involvement and Engagement professionals working alongside academic colleagues ensure that public involvement and engagement activity is delivered in a way that meets the recognised quality (UK) standards and plans and prepares for any risks associated with the activity. There is a growing debate around creating a recognised simple streamlined process acceptable to journal editors and funders to avoid researchers having to put their public engagement and involvement activity unnecessarily through local University Research Ethics Committees.

* Recommending the creation of dedicated engagement ethics committees could be confused with research ethics committees and appear overly burdensome.

* Would they be necessary in all RFOs? Would the level of engagement in RFOs warrant the establishment of dedicated 'engagement' ethics committee? Could a simpler process be put in place? If deemed necessary, potentially an overarching Committee accessible to all RFOs would be a more effective solution, avoiding duplication of effort across RFOs.

Resources developed by academia, funders and the public in the UK to address the identified challenges (UK Standards for Public Involvement) to public involvement in research would be equally applicable and helpful to RFOs working to engage the public in the formulation of funding calls. Resources from other countries may also exist and be helpful here.

Several funding bodies in the health and social care field have embedded public involvement in their funding processes (National Institute of Health Research; Health Research Authority; Health and Care Research Wales) as well as health charities (Cancer Research UK; Diabetes UK). European RFOs could potentially learn from the experience of these organisations in guiding and embedding patient and public involvement into organisational processes.

I hope my review comments are helpful. Happy to respond to any author queries.

**Is the background of the case's history and progression described in sufficient detail?**

Partly

**Is the work clearly and accurately presented and does it cite the current literature?**

Partly

**If applicable, is the statistical analysis and its interpretation appropriate?**
Not applicable

**Are all the source data underlying the results available to ensure full reproducibility?**

Yes

**Are the conclusions drawn adequately supported by the results?**

Partly

**Is the case presented with sufficient detail to be useful for teaching or other practitioners?**

Yes

**Competing Interests:** No competing interests were disclosed.

**Reviewer Expertise:** Patient and Public Involvement and Engagement Professional in field of health research and education.

**I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.**

Author Response 25 Oct 2024

**Kalli GIANNELOS**

Dear Reviewer, We sincerely appreciate your thorough and constructive feedback. Your insights have been invaluable in refining our manuscript, and we are grateful for the opportunity to incorporate them. Below is a summary of the revisions made in response to your comments: **Comment:** "Regarding terminology used in this case study - be mindful that in certain sectors and in different countries 'public engagement' can mean different things to different people. There is a lot of literature available on this. In health and social care research in the UK, for example, public involvement and public engagement are both used and mean different things. At the outset of the paper, it maybe helpful to define your meaning of 'public engagement' to improve understanding across disciplines, sectors, countries etc."

**Response:** The introduction has been revised to clarify the meaning of "public engagement" within the context of this study. We acknowledge that the term may vary across sectors and countries, and this definition aims to ensure clarity for readers from different disciplines.

**Comment:** "The seven highlighted challenges exist for a much wider range of organizations and have a much broader scope than just Research Funding Organisations."

**Response:** We have expanded the manuscript to reflect that the seven challenges identified are experienced by a wide range of organizations beyond just RFOs. This change has been made in section 2 to ensure the broader relevance of our findings.

**Comment:** "Challenge 1 - recruiting members of the public to get involved in any aspect of research is a challenge for the research community as a whole not just for RFOs. There is growing literature on how to encourage a more diverse public to get involved in the research cycle process which would be relevant to RFOs or any organisation/researcher wanting to involve members of the public in their work.

When working with health and care researchers, the priority is to attract representation from
members of the public who have 'lived experience' of a particular health condition/social care aspect that is being researched. Certain essential criteria need to be clearly articulated in a role description which then can be promoted widely to identified key stakeholders including charities; community groups; libraries; companies who facilitate contact with hard-to-reach groups. For RFOs, fund scope/area will help to inform appropriate representation, so selection is not just based on different socio-economic factors. There are a range of resources out there (that could be referenced) to support inclusivity in research. For example, *Reaching Out: A guide to being inclusive in public involvement (learningforinvolvement.org.uk)*. In the UK, we also have the UK Standards for Public Involvement which include six standards which make up a framework for what good public involvement in research looks like. These can be adapted to different situations and are designed to be used as a guide/tool to help researchers and research organisations identify what they are doing well and what needs improving. The six standards are focussed on Inclusive Opportunities; Working Together; Impact; Governance; Communications; Support and Learning. Similar guidance maybe available in other European countries? Availability of budget often dictates how much involvement will be possible (numbers of people to be recruited to take part etc)."

**Response:** In response to your feedback, we have broadened the discussion of recruitment in Challenge 1 to acknowledge that this issue extends beyond RFOs. We have also incorporated references to relevant literature on recruitment diversity and inclusivity, including frameworks from health and social care research.

**Comment:** "Challenge 6. Engagement ethics. Use of the word 'ethics' here could be confusing. Professionalism and adequate resource (human and financial) are key to delivering quality public engagement. This does not rely on specific expertise but practitioners who conduct themselves and their work in a professional manner and have access to adequate resource to support their engagement work. Challenges in practice: The case of an RFO. 'The lack of expertise in engagement ethics further complicates the process, necessitating external input and ongoing learning'. My concern here is that you are creating an unnecessary barrier to conducting public involvement/engagement? Creating a field of expertise 'engagement ethics' lends itself to being a specialised area requiring expert knowledge. I believe that this is potentially misleading. All working professionals should take responsibility for going about their work in a professional and ethical manner embracing values of 'respect' 'fairness'; and 'transparency'. I worry that stating a 'lack of engagement ethics expertise' could easily be presented as an excuse for not doing public engagement work, effectively acting as an additional barrier."

**Response:** We have revised Challenge 6 to clarify that ethical public engagement does not necessitate the creation of a specialized field. Instead, the revision highlights the role of professionalism, fairness, and transparency in guiding ethical engagement, avoiding unnecessary barriers.

**Comment:** "Ensuring financial compensation for participants' time and expenses could also be beneficial' - To support inclusivity offering compensation (financial or non-financial) is a recognised UK Standard for public involvement that all researchers/organisations should adopt when planning a public involvement activity with patients and members of the public." **Response** : Your suggestion regarding compensation for participants has been addressed. The text in Challenge 4 has been revised to emphasize that compensation is a recognized standard for inclusive public engagement, with examples drawn from established UK practices and beyond.

**Comment:** "Recommendation 6 - Enhancing ethical standards through internal expertise and external advisory inputs. Once again, my concern here is that this could potentially create an
unnecessary barrier and should not be needed if organisations operate professionally embracing values of fairness, equality, respect etc. Ethical public engagement – This term raises several queries:

* I fear the use of this term will cause confusion in a growing debate concerning Public Involvement and Engagement versus Research Ethics (from higher education institutional research ethics committees). Public involvement and public engagement in research does not require research ethics but should be planned and conducted in a professional manner embracing EDI principles and taking into account risk. Public Involvement and Engagement professionals working alongside academic colleagues ensure that public involvement and engagement activity is delivered in a way that meets the recognised quality (UK) standards and plans and prepares for any risks associated with the activity. There is a growing debate around creating a recognised simple streamlined process acceptable to journal editors and funders to avoid researchers having to put their public engagement and involvement activity unnecessarily through local University Research Ethics Committees.

* Recommending the creation of dedicated engagement ethics committees could be confused with research ethics committees and appear overly burdensome.

Would they be necessary in all RFOs? Would the level of engagement in RFOs warrant the establishment of dedicated 'engagement' ethics committee? Could a simpler process be put in place? If deemed necessary, potentially an overarching Committee accessible to all RFOs would be a more effective solution, avoiding duplication of effort across RFOs.

**Response:** In line with your suggestion, we have revised Recommendation 6 to avoid the creation of dedicated engagement ethics committees. Instead, we propose more flexible ethical processes, with the possibility of a shared advisory board to provide structured oversight if needed.

**Comment:** "Resources developed by academia, funders and the public in the UK to address the identified challenges (UK Standards for Public Involvement) to public involvement in research would be equally applicable and helpful to RFOs working to engage the public in the formulation of funding calls. Resources from other countries may also exist and be helpful here.

Several funding bodies in the health and social care field have embedded public involvement in their funding processes (National Institute of Health Research; Health Research Authority; Health and Care Research Wales) as well as health charities (Cancer Research UK; Diabetes UK).

European RFOs could potentially learn from the experience of these organisations in guiding and embedding patient and public involvement into organisational processes." **Response:** We have integrated references to international resources and experiences that can inform and enhance public engagement strategies. The revised conclusion emphasizes the importance of cross-sectoral and international collaboration to address these challenges. We trust these revisions address your concerns and enhance the clarity and relevance of the manuscript. Thank you once again for your invaluable feedback.

**Competing Interests:** No competing interests were disclosed.
© 2024 Carter L. This is an open access peer review report distributed under the terms of the [Creative Commons Attribution License](https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

<figure><img src="image_16.png" /></figure>

Lucy Carter

<figure><img src="image_17.png" /></figure>

CSIRO Environment, Brisbane, Queensland, Australia

The paper offers insights into the challenges faced by European RFOs in conducting ethical public engagement activities. While the main point being made concerns the lack of governance around how RTOs plan and deliver good public engagement, the challenges are not only specific to RFOs but to all research partnerships. The recommendations the authors make are laudable, and draw from existing literature. I did wonder whether the recommendations would be mute in other non-European contexts given the different institutional models globally.

I offer some suggestions for refinement in the paper below:

**A little more explanation is needed please on:** The methods used to collate the identified 7 challenges. It was unclear whether these were generated from a series of discussions/reflections. A little more detail is needed in terms of the data and the analysis that went into formulating these challenges. Perhaps 2-3 lines would be enough.

**Introduction, para 1, final sentence:** Just be careful here, communication and consultation processes almost always grant no power to publics (i.e. these methods are often one-way, top-down processes where empowerment is unlikely) See this useful spectrum from IAP2
[https://www.iap2.org/page/SpectrumEvolution](https://www.iap2.org/page/SpectrumEvolution)

**Final paragraph under Challenge 1:** I'm not convinced of the selection bias argument. Sampling-related bias can be reduced but never eliminated. It's also likely to be encountered by staff working in RFOs as well as experienced recruiters. Suggest that the authors make a stronger argument here by simply advocating for inclusion of appropriate expertise in social research (like social scientists, or similar) to design research processes that RFOs undertake. You could also argue that independent research expertise would likely reduce bias of other kinds, as well as perceived conflicts of interest.

**Challenge 3** would benefit from inclusion of some references.

**Recommendation 1:** I would add 'social scientists' to the mix of recommended experts. This group can also broker partnerships and facilitate engagement processes.

**Recommendation 3:** Here you list several specific methods. While these might be OK, they might also not be the right ones for the context. Perhaps just add a statement here that these are examples. In any qualitative methods textbook, you would find dozens of method examples, all designed for diverse scenarios.

**Finally,** As an Australian researcher, our RFOs contract out almost all deliberate public engagement initiatives to science or university organisations, sometimes in partnership. While
some Australian RFOs may conduct consultation-type engagement on their own, historically, deeper engagement has been conducted by research organisations and designed and delivered by expert researchers. Under this model, research ethics approval processes along with academic and engagement expertise ensures the design of quality research processes.

The concept of RFOs conducting deep research themselves is foreign in the research ecosystem in Australia, and much of the Indo-Pacific region. It was difficult for me to understand this difference in research governance in the paper initially and so I wondered whether it was possible to clarify somewhere in the introduction that globally, there are other research governance models in play (and hence where your identified challenges might not be relevant.)

Happy to respond to any author queries about this review.

**Is the background of the case's history and progression described in sufficient detail?**

Partly

**Is the work clearly and accurately presented and does it cite the current literature?**

Partly

**If applicable, is the statistical analysis and its interpretation appropriate?**

Not applicable

**Are all the source data underlying the results available to ensure full reproducibility?**

No source data required

**Are the conclusions drawn adequately supported by the results?**

Partly

**Is the case presented with sufficient detail to be useful for teaching or other practitioners?**

Yes

**Competing Interests:** No competing interests were disclosed.

**Reviewer Expertise:** Social science; engagement research and practice; interdisciplinary research approaches

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Author Response 25 Oct 2024

**Kalli GIANNELOS**

Dear Reviewer, We would like to sincerely thank you for your insightful and constructive feedback. Your comments have been invaluable in helping us refine the manuscript, particularly by clarifying key points and strengthening the arguments presented. Below is a
summary of the revisions made in response to your critique:

**Comment:** "A little more explanation is needed please on: The methods used to collate the identified 7 challenges. It was unclear whether these were generated from a series of discussions/reflections. A little more detail is needed in terms of the data and the analysis that went into formulating these challenges. Perhaps 2-3 lines would be enough."

**Response:** You raised concerns about the clarity regarding how the seven challenges were generated. To address this, we have revised the second section to provide additional details, explaining that the challenges were identified through a series of cross-learning workshops and discussions among the eight participating RFOs. We also clarified that the data was analyzed using an inductive thematic analysis. Verbatim transcriptions were coded at the sentence and paragraph level, providing the foundation for identifying the key themes and challenges.

**Comment:** "Introduction, para 1, final sentence: Just be careful here, communication and consultation processes almost always grant no power to publics (i.e. these methods are often one-way, top-down processes where empowerment is unlikely) See this useful spectrum from IAP2 https://www.iap2.org/page/SpectrumEvolution"

**Response:** To highlight the fact that communication and consultation processes often do not grant significant power to the public, the manuscript now includes a reference to the IAP2 Public Participation Spectrum, clarifying that communication and consultation are often one-way, top-down mechanisms with limited public empowerment.

**Comment:** "Final paragraph under Challenge 1: I'm not convinced of the selection bias argument. Sampling-related bias can be reduced but never eliminated. It's also likely to be encountered by staff working in RFOs as well as experienced recruiters. Suggest that the authors make a stronger argument here by simply advocating for inclusion of appropriate expertise in social research (like social scientists, or similar) to design research processes that RFOs undertake. You could also argue that independent research expertise would likely reduce bias of other kinds, as well as perceived conflicts of interest."

**Response:** The original manuscript's argument on selection bias was questioned, and it was suggested that social scientists or independent research experts could help in designing more robust research processes. The revised version of Challenge 1 now advocates for the inclusion of social scientists or independent research experts in the design of engagement processes to mitigate various forms of bias and potential conflicts of interest.

**Comment:** "Challenge 3 would benefit from inclusion of some references."

**Response:** As recommended, we have updated Challenge 3 to incorporate relevant references that support the discussion on power imbalances and strategies for fostering meaningful dialogue. These citations substantiate the proposed methods for addressing knowledge-based power imbalances.

**Comment:** "Recommendation 1: I would add 'social scientists' to the mix of recommended experts. This group can also broker partnerships and facilitate engagement processes."

**Response:** The suggestion to include social scientists explicitly in Recommendation 1 has been implemented. Social scientists are now highlighted as key experts who not only contribute to recruitment strategies but also broker partnerships and facilitate effective engagement processes.
**Comment:** "Recommendation 3: Here you list several specific methods. While these might be OK, they might also not be the right ones for the context. Perhaps just add a statement here that these are examples. In any qualitative methods textbook, you would find dozens of method examples, all designed for diverse scenarios."

**Response:** The original version of Recommendation 3 listed specific methods, which might not be universally appropriate. We have revised the text to clarify that the methods mentioned are examples and that a variety of other approaches may be more suitable depending on the engagement context.

**Comment:** "Finally, As an Australian researcher, our RFOs contract out almost all deliberate public engagement initiatives to science or university organisations, sometimes in partnership. While some Australian RFOs may conduct consultation-type engagement on their own, historically, deeper engagement has been conducted by research organisations and designed and delivered by expert researchers. Under this model, research ethics approval processes along with academic and engagement expertise ensures the design of quality research processes. The concept of RFOs conducting deep research themselves is foreign in the research ecosystem in Australia, and much of the Indo-Pacific region. It was difficult for me to understand this difference in research governance in the paper initially and so I wondered whether it was possible to clarify somewhere in the introduction that globally, there are other research governance models in play (and hence where your identified challenges might not be relevant.)."

**Response:** It was noted that the challenges identified may not apply in non-European contexts due to different research governance models. To address this, we added a statement in the Introduction acknowledging that in regions such as Australia and the Indo-Pacific, RFOs often delegate public engagement to research organizations, making some of the challenges discussed less applicable in these areas. The manuscript has been thoroughly revised in response to all the points raised in your review. We extend our gratitude once again for your thoughtful and detailed feedback, which has significantly enhanced the quality, clarity, relevance, and rigor of this work.

**Competing Interests:** No competing interests were disclosed.