Comparative Analysis of Inclusion Criteria of Journal Blacklists and Whitelists Michaela Strinzel, Anna Severin, Matthias Egger und Katrin Milzow Swiss National Science Foundation Background •Journal Blacklists (BL) and Whitelists (WL) aim to capture the legitimacy, quality and transparency of scientific journals[1]. •With the emergence of Predatory Journals BL and WL are increasingly conculted by scholars in their search for publication outlets [2]. •While there is doubt over the objectivity, reliability and transparency of BL and WL [3], these lists and their inclusion criteria have not yet been analysed systematically. Aim Systematic comparison of inclusion criteria and contents (journals and publishers) of journal BL and WL •Aim: Identify and understand indicators of legitimacy, quality and transparency in scholarly publishing •Aim: Point out the scientific community's expectations towards legitimacy, quality and transparency Methods Qualitative coding and comparative analysis of 198 inclusion criteria of 2 BL (Beall's List and Cabell's International) and 2 WL (DOAJ and Cabell's International) on 3 levels: •Subject Matter: To what aspect of a scientific journal does the inlcusion criterion relate? •Informative Function: Does the criterion relate to (1) Quality, (2) Transparency, (3) Legitimacy, or (4) Establishing Stage of a scientific journal? • Verifiability: How easy is it to verify a criterion? Comparison of BL and WL in terms of their contents (e.g. journals and publishers) making use of the Jaro-Winkler Distance. Illustration of overlaps with Venn diagrams. Results Image 1: Subject Matter Image 1: Informative function Image 3: Overlaps between the BL Beall’s List of Publishers and the WL DOAJ Lists Transparency Quality Degree of Establishment Legitimacy BL 3 2.4 2.5 2.1 WL 3 2.4 2 1.7 Tabelle 1: Verifiability of inclusion criteria by informative function in average. (Rated on a scale from 1=hard bis 3=easy) Contact Information •Swiss National Science Foundation: http://www.snf.ch •E-Mail: michaela.strinzel@snf.ch; anna.severin@snf.ch Explanation of Results •Subject Matter BL put focus on Business Practices and Quality Control of journals, while WL stress Quality Control and Policies •Informative Function BL: Most inclusion criteria give information regarding Integrity and Legitimacy of a journal or publisher. WL: Most inclusion criteria provide information on Transparency. • Verifiability Overall, inclusion criteria can be said to be verifiable. There are, however, significant differences in verifiability across inclusion criteria of different informative functions. • Comparison of BL and WL contents Content overlaps between BL and WL (here: publisher in Beall’s List and in DOAJ) is relatively small. Conclusions Methdologically, it proved feasible to classify and systematically compare inclusion criteria and contents of BL and WL. In terms of their subject matter and informative function, BL and WL seem to complement one another. As such, BL and WL serve different purposes. This is reflected in the small content overlaps between the two types of journal lists. Our systematic classification and comparison of inclusion criteria can help to develop a more holistic understanding of quality, legitimacy and transparency in scholarly publishing. References [1] Vence, T. (2017). On Blacklists and Whitelists. The Scientist. [2] Swauger, S. (2017). Open access, power, and privilege: A response to ’What I learned from predatory publishing.’ College & Research Libraries 78:11. [3] Olivarez et al. (2018). Format Aside: Applying Beall’s Criteria to Assess the Predatory Nature of both OA and Non-OA Library and Information Science Journals. College & Research Libraries, 79:1. This work is accessible and can be shared and adapted under the Creative Commons (CC) license (CC BY 4.0).