Identification of concepts of importance for the assessment of internal validity of in vitro toxicology studies using a modified Delphi technique
Creators: name: Mathisen, Gro Haarklou
name: Svendsen, Camilla
name: Vist, Gunn Elisabeth
name: Husøy, Trine
name: ames, heather
name: Bearth, Angela
name: Audebert, Marc
name: Bernhard, Annette
name: Beronius, Anna
name: Bruzell, Ellen M.
name: Di Consiglio, Emma
name: Davenport, Miles
name: Druwe, Ingrid
name: Geci, René
name: Gundert-Remy, Ursula
name: Hartung, Thomas
name: Hoffmann, Sebastian
name: Hogberg, Helena
name: Hooijmans, Carlijn R
name: Lizarraga, Lucina
name: Olker, Jennifer
name: prieto, pilar
name: Robinson, Joshua
name: Rooney, Andrew
name: Sebollela, Adriano
name: Smith, Nicola
name: Spilioti, Eliana
name: Spyropoulou, Anastasia
name: Tcheremenskaia, Olga
name: Testai, Emanuela
name: Wang, Amy
name: Zilliacus, Johanna
name: Whaley, Paul
Publisher: name: Zenodo
Year: 2025

description: Context: In vitro toxicity studies are increasingly being included as evidence in systematic reviews and chemical risk assessments. INVITES-IN, a tool for assessing the internal validity of in vitro studies, is under development. The first step in the creation of INVITES-IN was the development of an “item bank” database of 405 concepts (“items”) of potential relevance for assessing the internal validity of in vitro toxicity studies. The items were gathered from focus groups and a purposive literature sample that included one systematic review and study appraisal tools for human, animal, and in vitro studies. In this paper we present the second step of the creation of INVITES-IN, i.e. the methods and results for identifying items for consideration when assessing the potential for bias in an in vitro study. 

Method: A two-round digital Delphi survey, followed by an online workshop with guided discussions, was performed. The Delphi participants were experienced with both in in vitro models and systematic review methods.

Results: Fifteen experts completed both Delphi rounds, and thirteen participated in the workshops. Of the 405 items in the bank, 372 were included for consideration when assessing potential for bias in an in vitro study. Both the literature sources and the focus group discussions contributed with concepts considered to be important for the assessment of the potential for bias in an in vitro study; 83%-100% of the items collected from the different literature sources were identified to be important and 91% (127) of the new items discovered in the focus group discussions were identified to be important.

Discussion: 

The 372 retained items will be interpreted into a manageable set of study appraisal criteria and a supporting guidance that will constitute the INVITES-IN study appraisal tool. The high retention of items included in tools designed for assessment of human and animal studies to in vitro studies suggests that many items are generally applicable across multiple study designs. Therefore, tool development processes should benefit from drawing on assessment tools outside the immediate domain of interest. Tool development would also likely benefit from supplementing literature reviews with focus group discussions, as our results demonstrate that the use of focus group discussions with domain experts is a pragmatic and valuable approach to increasing coverage of items in a tool development process. 

In conclusion, this study demonstrates the value of using rigorous methods to ensure a comprehensive dataset as the starting point for creation of an assessment tool.