DOI: 10.3886/icpsr38493
title: Developing and Validating Quantitative Measures to Assess Community Engagement in Research: Addressing the Measurement Challenge, United States, 2017-2020
title: Archival Version

description: Community-engaged research is an umbrella term for forms of research that have community and stakeholder engagement as a core principle, for example, patient-centered outcomes research (PCOR), participatory action research, and community-based participatory research. However, the implementation and category of community engagement can vary across a spectrum from minimal engagement to fully collaborative engagement. A major methodological gap is the assessment of stakeholder engagement from a stakeholder perspective. Evaluation of the impact of stakeholder engagement on research development, implementation, and outcomes requires the development and validation of tools that assess engagement. This study's objective was to develop and validate comprehensive and condensed versions of a survey instrument that will be completed by stakeholders and used to assess engagement in research studies from a stakeholder perspective.
description: To develop and validate comprehensive and condensed versions of a survey instrument that will be completed by stakeholders and used to assess engagement in research studies from a stakeholder perspective.
description: A literature review on community engagement in research was conducted to identify existing measures of engagement. Using the engagement principles (EPs), a comprehensive community engagement measure was identified and developed, and its construct validity was then evaluated. The Disparities Elimination Advisory Committee (DEAC), a community-academic partnership at the Siteman Cancer Center and the Washington University in St. Louis School of Medicine, served as the community advisory board guiding the work. The DEAC and the Patient Research Advisory Board (PRAB), a community group trained in research methods, served as community partners and assisted in determining the components to include in the measurement tool: engagement categories and EPs. These partners also advised on the development of survey items that would demonstrate the presence of EPs and on the recruitment and retention of participants to use the measurement tool. To assess content validity, a panel of national experts in stakeholder-engaged research (SER) were convened, and used a 5-round Delphi process to reach agreement on engagement categories and their definitions, EPs and their definitions, and items associated with each EP. The DEAC and the PRAB provided feedback on the work of the Delphi panel. The expert review panel comprised a range of stakeholders (e.g., patients, caregivers, advocacy groups, clinicians, researchers). After initiation of the Delphi process, community members who had participated in patient- or community-engaged research across the community engagement spectrum were recruited to participate in a longitudinal, web-based study that consisted of 4 surveys over an 18-month period. As the assessment tool was developed to measure categories of community engagement, the surveys permitted the investigators to examine the reliability and validity of the tool, which was named the Research Engagement Survey Tool (REST). The performance of each item on the tool was also evaluated. Two types of psychometric properties of the survey items were examined: internal consistency and convergent validity with other scales (e.g., Trust in Medical Researchers Scale). After the Delphi process and between Surveys 3 and 4, cognitive interviews about the survey items to improve the reliability and validity of REST were completed. REST has two scoring approaches. The first approach, aligned with EPs, is scored by taking the mean of the 3 to 5 items for each EP to obtain EP-specific scores and then taking the overall mean of the EP means to obtain an overall REST score. The second scoring approach aligns REST with the categories of engagement, developed during the Delphi process, based on the percentage of item responses that are aligned with each of 5 categories: (1) outreach and education, (2) consultation, (3) cooperation, (4) collaboration, and (5) partnership. The scoring approaches were applied to the 32-item and 9-item versions of the REST. Finally, the implementation of the final version of REST (long form) in 20 project teams were studied. These 20 project teams worked alongside the study to administer REST to their stakeholders in 26 separate PCOR and SER projects; the project teams completed surveys and interviews that examined barriers to and facilitating factors for implementing REST.
description: A panel of national experts in SER were convened, and a 5-round Delphi process was applied to reach agreement on engagement categories and their definitions, EPs and their definitions, and items associated with each EP. After initiation of the Delphi process, community members who had participated in patient- or community-engaged research across the community engagement spectrum were recruited to participate in a longitudinal, web-based study that consisted of 4 surveys over an 18-month period. The variables present in these data represent results from these efforts.
description: Convenience, snowball sampling methods.
description: Datasets: DS0: Study-Level Files DS1: Delphi Final Consensus Survey Dataset DS2: Delphi Survey 1 Dataset DS3: Delphi Survey 2 Dataset DS4: Delphi Survey 3 Dataset DS5: Implementation Partner/Stakeholder Survey Dataset DS6: Project Team Baseline Survey Dataset DS7: Project Team Follow Up Survey Dataset DS8: Longitudinal Participant Surveys Dataset
description: Adults (age 18 or older) involved in community-engaged research. Smallest Geographic Unit: Zip code
description: mixed mode; on-site questionnaire; web-based survey