Assessing Awareness, Engagement and Training Needs in Open Research Practices. A Study of Irish Researchers in Social Sciences across Quantitative and Qualitative Methods.
Creators: name: Lakhzoum, Dounia
Publisher: name: Zenodo
Year: 2024

description: Assessing awareness, engagement and training needs in Open Research Practices: A Study of Irish Researchers in Social Sciences across Quantitative and Qualitative Methods

Dounia Lakhzoum, Dermot Lynott, and the Tropic Consortium

Maynooth University, Ireland

Background

In recent years, Open Research has emerged as a prominent concern and gold standard across various disciplines (Hardwicke et al., 2021), spurred by the failure to successfully replicate studies across numerous academic disciplines (Tincani & Travers, 2019). Such failures have prompted efforts to evaluate replicability across diverse fields, ranging from economics (Camerer et al., 2016) to biology (Errington et al., 2021), resulting in considerable variations in successful replication rates (30%-90%). Consequently, researchers have recognised the imperative to adopt responsible research practices to ensure transparency, rigor, and reproducibility of findings (Gopalakrishna et al., 2021). Large-scale nationwide surveys have been undertaken to gauge engagement with Open Research Practices (ORPs). In the Netherlands, Gopalakrishna and colleagues (2021) conducted a survey of researchers, revealing substantial differences in engagement across various practices and disciplines. Similarly, in the UK, Norris and colleagues (2022) identified a gap between awareness of specific practices and their actual implementation. To date, no study has comprehensively assessed the levels of engagement with ORPs among Irish researchers.

To address this gap, our study introduces a preregistered survey evaluating ORPs across a spectrum of disciplines, ranging from the hard sciences to social sciences, including both qualitative and quantitative research methodologies. Incorporating practices throughout the research lifecyle, from study pre-registration, and open materials to preprints, and open access publications, the survey aimed to gather insights from researchers across Ireland. Furthermore, we aimed to gain an understanding of Irish researchers’ experiences with open research, and their perceived training needs.

The survey is part of a larger project and will inform the development of a tailored open research training programme, acknowledging differences in experiences, needs, and perspectives across diverse disciplines and research methodologies. This inclusive approach ensures a nuanced understanding of ORP awareness and engagement, with specific attention to identifying training disparities among individuals who engage in different research methodologies.

Here, we present preliminary findings for Irish researchers in the social sciences, capturing researcher awareness of specific open research practices, level of engagement in those practices, the impact of preferred methodology (qualitative vs quantitative), and the extent to which researchers perceive barriers to entry.

Method

We collected data from 307 participants from 16 research institutions in Ireland, which includes a total of 97 researchers from the social sciences. The survey probed the awareness and adoption of various practices by individual researchers. The study also explored the associations between researchers' practices, their field of study, their primary methodologies (quantitative, qualitative, or mixed methods), their self-reported capability, opportunities, and motivations regarding engagement in ORPs. Finally, we also gathered data on people’s training experiences, and desires for ORP training. The full list of survey questions can be found here (https://osf.io/chxgd/), on the project’s webpage. The survey received ethical clearance from Maynooth University.

Results

Preliminary findings emphasise a general interest in open research across all fields of social sciences. Despite reported constraints such as limited time (67% considered time to be an issue), lack of financial support (87% reported a lack of financial support), poor incentives from institutions (75% reported a lack of incentives), and minimal recognition (86% reported a lack of recognition), researchers in social sciences generally express intentions to apply ORPs to their research.

Awareness of open research practices ranged from 98% (Open Access) to 44% (Registered Reports), while level of engagement ranged from 77% (Open Access) to 17% (Registered Reports). However, there were disparities depending on whether researchers generally used quantitative or qualitative methodologies. We highlight a significant gap between the awareness and actual implementation of ORPs among Irish researchers, evident across both quantitative and qualitative research methodologies. Interestingly, this gap is more substantial for qualitative research methodologies (46%) compared to quantitative ones (20%) in social sciences.

Overall, quantitative researchers exhibit higher awareness of ORPs (86%) and display a smaller awareness-engagement gap (20%) compared to their qualitative counterparts. Qualitative researchers demonstrate lower awareness of certain ORPs (76%); however, they also tend to show a greater awareness-engagement gap (46%). Why is engagement lagging so far behind awareness? One possible explanation is that engagement efforts may not have been accompanied by targeted training or support specifically tailored for researchers from social sciences, and especially in qualitative methods. Social science research often involves unique challenges and nuances that may require specialised guidance for effective implementation of open practices (e.g., how best to share sensitive or personal data?).

Consequently, researchers can feel such practices are not applicable to their research. This discrepancy highlights a crucial need for targeted training initiatives, and adequate research support, to bridge the gap between awareness, perceived applicability, and actual adoption of ORPs in these fields. This observation is supported by a majority reporting a lack of training in ORPs (63%) despite overwhelmingly expressing a willingness to participate in such training (85%). This suggests that these practices are not commonly integrated into researcher training for social science in Ireland.

Conclusion

The current research aims to identify training needs among Irish researchers in social sciences. Preliminary results reveal researchers' willingness to engage in open research, but significant disparities exist between awareness and engagement, accompanied by a general lack of suitable training. Recognising these nuances is crucial for tailoring interventions and support structures, ensuring the programme addresses unique challenges. The insights gained from our survey will guide our programme‘s ethos, and inform others developing training content for researchers across fields.

Acknowledgement

This project has received funding from Ireland’s National Open Research Forum under the NORF Open Research Fund.

References

Camerer, C. F., Dreber, A., Forsell, E., Ho, T. H., Huber, J., Johannesson, M., ... & Wu, H. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351, 1433-1436.

Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. (2021). Investigating the replicability of preclinical cancer biology. Elife, 10,e71601.

Gopalakrishna, G., Wicherts, J. M., Vink, G., Stoop, I., van den Akker, O. R., Ter Riet, G., & Bouter, L. M. (2022). Prevalence of responsible research practices among academics in The Netherlands. F1000Research, 11.

Hardwicke, T. E., Bohn, M., MacDonald, K., Hembacher, E., Nuijten, M. B., Peloquin, B. N., Yoon, E.J., & Frank, M. C. (2021). Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: An observational study. Royal Society Open Science, 8, 201494.

Norris, E., Prescott, A., Noone, C., Green, J. A., Reynolds, J., Grant, S. P., & Toomey, E. (2022). Establishing open science research priorities in health psychology: a research prioritisation Delphi exercise. Psychology & Health, 1-25.

Tincani, M., & Travers, J. (2019). Replication research, publication bias, and applied behavior analysis. Perspectives on Behavior Science, 42, 59-75.