Full length article

MATES: A tool for appraising the completeness with which a meta-analysis
has been reported

Kyle Morrison a,*, b, c, Pietro Pollo a, b, Lorenzo Ricolfi a, b, Coralie Williams a, b,
Yefeng Yang a, ao, b, Damien Beillouin d, b, Simone Jaqueline Cardoso e, b, Verónica Ferreira f, b,
Brian Gallagher g, b, Jelaine L. Gan h, i, b, Guang Hao j, b, Mojtaba Keikha k, b,
Betina Kozlowsky-Suzuki l, b, T.M. Kiran Kumara m, b, Francesco Latterini n, b,
Alexandro B. Leverkus o, p, b, Erin L. Macartney a, q, b, Silvina Magdalena Manrique r, b,
April Robin Martinig a, s, b, Ayumi Mizuno a, b, b, Shanika Nanayakkara t, u, b,
Evangelia Ntzani v, w, b, Dakis-Yaoba Ouédraogo x, b,
Edward Pursell y, b, Zachary Simpson z, b, Harriet Sleight aa, b,
Kok Sin Woon ac, ad, b, Ziqian Xia ab, b, Mona Ghannad ae, b, Eliza Grames af, b,
Emily Alden Hennessy ag, ah, b, Joanna IntHout ai, b,
David Moher aj, ak, b, Rose E. O'Dea al, b, Matthew J. Page am, b, Paul Whaley an, b,
Malgorzata Lagisz a, b, 1, b, Shinichi Nakagawa a, b, 1, **

<footnote>ª Evolution & Ecology Research Centre and the School of Biological, Earth and Environmental Sciences at the University of New South Wales, Sydney, Australia</footnote>

<footnote>ᵇDepartment of Biological Sciences, Faculty of Science, University of Alberta, Edmonton, AB, Canada</footnote>

<footnote>Department of Ecology and Evolution, Research School of Biology, The Australian National University, Canberra, Australian Capital Territory, Australia</footnote>

<footnote>d HortSys, Univ Montpellier, CIRAD, Montpellier, France</footnote>

<footnote>e Laboratory of Plankton Ecology, Department of Zoology, Institute of Biology, Federal University of Juiz de Fora, Juiz de Fora 36036-900, Brazil</footnote>

<footnote>f MARE - Marine and Environmental Sciences Centre, ARNET - Aquatic Research Network, Department of Life Sciences, University of Coimbra, Calçada Martim de Freitas, 3000-456 Coimbra, Portugal</footnote>

<footnote>Department of Biology, Concordia University, Montreal, Quebec, Canada</footnote>

<footnote>School of Natural and Environmental Sciences, Newcastle University, Newcastle upon Tyne, United Kingdom</footnote>

<footnote>Institute of Biology, College of Science, University of the Philippines, Diliman, Quezon City, Philippines</footnote>

<footnote>Department of Epidemiology and Statistics, School of Public Health, Guangdong Pharmaceutical University, China</footnote>

<footnote>k Social Determinants of Health Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran</footnote>

<footnote n="1">¹Department of Ecology and Marine Resources, Institute of Biosciences, Federal University of the State of Rio de Janeiro, Rio de Janeiro, RJ 22290-240, Brazi</footnote>

<footnote>m ICAR-National Institute of Agricultural Economics and Policy Research (NIAP), New Delhi 110012, India</footnote>

<footnote>n Institute of Dendrology, Polish Academy of Sciences, Parkowa 5, Kórnik 62-035, Poland</footnote>

<footnote>Department of Ecology, University of Granada 18071 Granada, Spain</footnote>

<footnote>p Laboratory of Ecology (Iecolab), Inter-University Institute for Earth System Research in Andalusia (IISTA), Granada, Spain</footnote>

<footnote>q Charles Perkins Centre, School of Life and Environmental Sciences, The University of Sydney, Sydney, Australia</footnote>

<footnote>Universidad de Alcalá, Departamento de Geología, Geografía y Medio Ambiente, A-II km 33.0, 28805 Alcalá de Henares, Madrid, Spain</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>** Corresponding author at: Evolution & Ecology Research Centre and the School of Biological, Earth and Environmental Sciences at the University of New South Wales, Sydney, Australia.</footnote>

<footnote>E-mail addresses: kyle.morrison@unsw.edu.au, khtmorrison12@gmail.com (K. Morrison), p.pottier@unsw.edu.au (P. Pottier), pietro_pollo@hotmail.com (P. Pollo), l.ricolfi@unsw.edu.au (L. Ricolfi), coralie.williams@unsw.edu.au (C. Williams), yefeng.yang1@unsw.edu.au (Y. Yang), beillouin@cirad.fr (D. Beillouin), simone.jcardoso@gmail.com (S.J. Cardoso), veronica@ci.uc.pt (V. Ferreira), brian.kenneth.gallagher@gmail.com (B. Gallagher), jelainegan21@gmail.com (J.L. Gan), haoguang2015@hotmail.com (G. Hao), mr.mojtabakeikha@gmail.com (M. Keikha), betinaksuzuki@unirio.br (B. Kozlowsky-Suzuki), kiran.tm@icar.gov.in (T.M. Kiran Kumara), latterini@man.poznan.pl (F. Latterini), leverkus@ugr.es (A.B. Leverkus), erin.macartney@sydney.edu.au (E.L. Macartney), silvina.manrique@uah.es (S.M. Manrique), aprilmartinig@hotmail.com (A.R. Martinig), ayumi.mizuno5@gmail.com (A. Mizuno), shanika.nanayakkara@sydney.edu.au (S. Nanayakkara), evangelia_ntzani@brown.edu (E. Ntzani), dakis.ouedraogo@gmail.com (D.-Y. Ouédraogo), edward.purssell@aru.ac.uk (E. Pursell), zpsimpso@gmail.com (Z. Simpson), harriet.sleight@york.ac.uk (H. Sleight), woonkoksin@gmail.com (K.S. Woon), ziqian.research@gmail.com (Z. Xia), mona.ghannad@gmail.com (M. Ghannad), egrames@binghamton.edu (E. Grames), ehennessy@mgh.harvard.edu (E.A. Hennessy), Joanna.intHout@radboudumc.nl (J. IntHout), dmoher@ohri.ca (D. Moher), rose.eleanor.o.dea@gmail.com (R.E. O'Dea), matthew.page@monash.edu (M.J. Page), p.whaley@lancaster.ac.uk (P. Whaley), m.lagisz@unsw.edu.au (M. Lagisz), snakagaw@ualberta.ca (S. Nakagawa).</footnote>
⁵ The Okanagan Institute for Biodiversity, Resilience, and Ecosystem Services, University of British Columbia, Kelowna, British Columbia, Canada

¹ Sydney Dental School, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia

¹ª Institute of Dental Research, Westmead Centre for Oral Health, Westmead Hospital, Westmead, Australia

⁷ Center for Evidence Synthesis in Health, Brown University School of Public Health, RI, USA

⁸ Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Greece

⁹ Independent Researcher, Athens, Greece

y Faculty of Health, Medicine and Social Care, Anglia Ruskin University, Chelmsford United Kingdom

z US Department of Agriculture, Agricultural Research Service, Sustainable Water Management Research Unit, Stoneville, MS, USA

aa Department of Environment and Geography, University of York, York, United Kingdom

ab Department of Environmental Social Sciences, Stanford University, CA, USA

ac Carbon Neutrality and Climate Change Thrust, Hong Kong University of Science and Technology (Guangzhou), China

ad School of Energy and Chemical Engineering, Xiamen University Malaysia, Malaysia

ae Department of Pediatrics, University of British Columbia and British Columbia Children's Hospital, Vancouver, British Columbia, Canada

a<fnref n="1" /> Department of Biological Sciences, Binghamton University, Binghamton, NY, USA

a² Recovery Research Institute, Center for Addiction Medicine, Massachusetts General Hospital, Boston, MA, USA

ab Harvard Medical School, Boston, MA, USA

a<fnref n="1" /> IQ Health, Radboudumc, Nijmegen, the Netherlands

a) Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

ak School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

a<fnref n="1" /> The School of Agriculture, Food and Ecosystem Sciences, University of Melbourne, Parkville, Melbourne, Australia

am Methods in Evidence Synthesis Unit, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia

an Lancaster Environment Centre, Lancaster University, Lancaster, United Kingdom

aº College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou, China

ARTICLE INFO

Keywords:

Transparency
Reproducibility
Triage
Reliability
Risk of bias

A B S T R A C T

Meta-analysis is commonly a core component of systematic reviews and has become an important method to reconcile conflicting findings, increase statistical power, and chart new research directions. However, poor reporting practices make it challenging to evaluate the validity of meta-analyses. Despite the existence of reporting checklists, a specifically designed tool has yet to be developed to appraise the completeness with which a meta-analysis has been reported. To bridge this gap, we introduce the Meta-analysis Appraisal Tool for Environmental Sciences (MATES). To develop MATES, we adapted a Delphi process involving experts in meta-analysis methodologies, researchers with experience in guideline/appraisal tool development, and editors of relevant journals. The Delphi process had five steps, including three workshops (11–16 participants), a survey (193 participants), and a validation task (30 participants). This iterative development process resulted in a 14-item appraisal tool that reflects the environmental science and research syntheses community’s consensus on essential elements to appraise the completeness with which a meta-analysis has been reported. Validation across 50 meta-analyses demonstrated that the tool is repeatable (average agreement rate: 88.97 %) and time-efficient to implement (17.00 ± 12.23 min). We also outline guidance for interpreting MATES results, describe its potential applications, and reflect on the development process. The authors provide practical implementation guidance for each MATES item, illustrated with real examples in the supplementary material. We also report an extended development methodology to support reproducibility. Finally, we built created a ShinyApp that includes both a training module and an application tool to enhance the usability of MATES (https://kylemorrisonishiny99.shinyapps.io/MATES_shiny/). Overall, MATES provides authors, readers, stakeholders, and editors with a reliable and accessible tool for appraising the completeness with which a meta-analysis in environmental sciences has been reported.

# 1. Introduction

Meta-analysis is often a core component of systematic reviews and is commonly defined as the quantitative synthesis of multiple studies addressing a specific research question (Gurevitch et al., 2018). It is widely recognised as a powerful and reliable methodology for reconciling conflicting outcomes, generating robust evidence, and identifying drivers of heterogeneity in effects (Pollo et al., 2025). In an ideal world, the reporting and conduct of meta-analysis methodologies are in harmony. However, that is often not the case. Suboptimal reporting of meta-analysis methodologies can conceal important technical details and potential biases, directly undermining a reader's ability to accurately interpret the validity of a meta-analysis (Chalmers and Glasziou, 2009; Fraser et al., 2018; Moher, 2018; Nakagawa et al., 2023b). Therefore, to allow readers, stakeholders, and editors to accurately appraise the conduct of a meta-analysis, the authors must transparently report their methodological steps (Haddaway et al., 2018; Parker et al., 2016).

In the environmental sciences, the most commonly followed

reporting checklists for researchers to report how they conducted their meta-analysis are PRISMA (Moher et al., 2009; Page et al., 2021) and ROSES (Haddaway et al., 2018). Developed for systematic reviews, these reporting checklists outline a comprehensive set of items that authors should report in their meta-analyses. These reporting checklists have been widely used and endorsed, contributing to an improvement in the completeness of reporting (Morrison et al., 2025a; Page and Moher, 2017). Nevertheless, while such reporting checklists are designed to guide authors, there is currently no equivalent tool to enable editors, reviewers, or other stakeholders to appraise the completeness of reporting of meta-analyses conducted by others.

A tool designed to appraise the completeness with which a meta-analysis has been reported should incorporate elements that are not typically included in reporting checklists (Morrison et al., 2025). Such a need arises from the differing requirements of appraisers, who evaluate meta-analyses, and authors, who conduct them (Fig. 1). We highlight three key considerations that are critical for appraisal tools but absent from reporting checklists. First, each item should be accompanied by explicit criteria specifying what constitutes fulfilment (i.e., the basis for assigning scores), thereby reducing subjective judgement in determining whether an item has been satisfied (Woodcock et al., 2014). Second, the

<footnote n="1">¹ Equal Supervision.</footnote>
repeatability of appraisal outcomes across independent appraisers should be formally evaluated, as this information is essential for users and readers to judge the reliability of the tool (Shea et al., 2017; Whiting et al., 2016). Third, implementation time should be evaluated, as appraisal tools may need to be applied repeatedly across large numbers of meta-analyses, particularly in contexts such as the editorial review process (Shea et al., 2017). Despite their importance for appraisal, these considerations are not currently reflected in existing guidance for the development of reporting checklists (Moher et al., 2010).

To address this gap and ensure that the key developmental aspects are incorporated, we developed the Meta-analysis Appraisal Tool for Environmental Sciences (referred to as MATES) to appraise the completeness with which a meta-analysis has been reported (i.e., the quantitative aspect of systematic reviews with meta-analysis) (Fig. 1). To maximise usability and uptake of MATES, we incorporated community input through an adapted Delphi process — a structured communication technique used to achieve expert consensus on a specific topic (Barrett and Heale, 2020). To enhance the repeatability of appraisal outcomes and reduce potential for subjective interpretations, we designed each item with a simplified binary appraisal system. Although the primary aim of MATES is to evaluate the completeness with which a meta-analysis has been reported in environmental sciences, many of its core items address fundamental aspects of transparent reporting that are broadly applicable across disciplines. Therefore, MATES may be useful in other related disciplines such as medicine, ecology, evolutionary biology, and conservation.

## 2. Methodology

MATES was developed through an adapted Delphi process following recommendations by the EQUATOR Network (Moher et al., 2010). Here, we outline the development process of MATES, which consists of the following sections: Workshop 1, Survey, Workshop 2, Validation task, and Workshop 3 (Fig. 2). The supplementary materials are organised sequentially: Supplementary File 1 contains the stand-alone MATES questionnaire; Supplementary File 2 provides an explanatory guide with item rationale and step-by-step instructions; Supplementary File 3 offers

a detailed development dossier, including feedback from the workshops and stakeholder survey. We also developed a Shiny App that includes both a training module and an application tool (https://kylemorrisonisshiny99.shinyapps.io/MATES_shiny/). We provide all the analysis code in a dedicated GitHub repository (https://github.com/KyleMorrison99/MATES). During the development of MATES, we assembled a team with diverse genders, experiences, and nationalities to reflect the broader environmental science and evidence synthesis community. This development process was conducted following ethics approval from the University of New South Wales, Sydney (reference number: HC230087). The reporting of our methodology follows the MeRIT framework to promote granularity and accountability for each author's contribution to the project (Nakagawa et al., 2023a).

### 2.1. Workshop 1

After the initial conceptualisation, the core development team (KM, CW, LR, YY, DM, EG, EAH, JH, MG, ML, MJP, PW, RO, SNakagawa) drafted a preliminary set of items for inclusion in MATES. These draft items were then evaluated during a virtual workshop, where the expert panel assessed their importance to the overall completeness of reporting in meta-analyses. The panel also provided feedback on the clarity of each item. Following the first workshop, we confirmed a set of 17 preliminary methodological items, which were subsequently evaluated in the following survey.

### 2.2. Survey

The survey questions were developed by KM with support from CW, LR, ML, SNakagawa, and YY. The primary goal of the survey was to provide the opportunity to the environmental science community to evaluate the importance of each proposed appraisal item within MATES. In general, each question gives survey participants the opportunity to rank each item from “Extremely important” to “Not at all important,” and to provide comments or justifications for their response. Participants were also encouraged to comment on the clarity of the wording and suggest any additional appraisal items. A link to the Qualtrics survey

<figure><img src="image_1.png" /><figcaption>**Fig. 1.** A) From the author's perspective, it is important to report each methodological item in a meta-analysis. Thorough reporting helps convey the study's conduct (what was done) and rigour (how well it was done) to readers and appraisers, ensuring transparency and facilitating evaluation. B) From the appraiser's perspective, the MATES tool offers a structured approach to evaluate the completeness with which a meta-analysis has been reported, separately from its conduct and rigour. To appraise the quality of conduct or rigour effectively, appraisers should first consider the completeness of the reporting. This is because clear and detailed reporting provides the necessary context to evaluate how the study was conducted and the level of rigour applied. Reporting completeness serves as a key step in understanding and appraising the overall quality of a meta-analysis.</figcaption></figure>
was emailed to all corresponding authors of meta-analyses in environ-
mental sciences, identified through a targeted search string on Web of
Science Core Collection (see Supplementary File 3 for the full search
string). In total, 3,969 (5,164 before deduplication) participants were
invited, and 193 participants completed the survey (4.86 %), contrib-
uting to refining the initially proposed MATES items. KM processed the
survey results using R v.4.2.1 (R Core Team, 2022) in RStudio (RStudio
Team, 2020) (see GitHub repository: https://github.com/KyleMorriso
n99/MATES for all code). The results of this survey were analysed in a
structured manner and were visualised in a series of bar plots to present
to the core team members in the subsequent workshop. We provide all
survey questions and processed results in Supplementary File 3.

## 2.3. Workshop 2

KM held one-to-one meetings with each of the core team members (DM, EG, EAH, JH, MG, ML, MJP, PW, RO, SNakagawa) to interpret the results of the survey. During these discussions, we reviewed the initially proposed feedback and adjusted the tool accordingly. Through these consultations, KM refined the MATES appraisal items with support from CW, LR, ML, SNakagawa, and YY. As a result, four items were removed, one item was added, and several other items were revised to enhance the

clarity of wording. These improvements to the checklist were imple-
mented based on the survey results and the perspectives provided by
core team members (see Supplementary File 3 for specific adjustments).

## 2.4. Validation task

We conducted a validation task to assess the repeatability (also known as intra-class correlation), inter-rater reliability between appraisers, and the time to implement the MATES tool. KM performed a literature search in the Web of Science Core Collection to identify 50 recently published meta-analyses in environmental sciences that were used for the validation task (see Supplementary File 3). We invited 31 participants who submitted letters of interest following the survey. To ensure that agreement rates and implementation times reflected typical user experiences, participants were not trained in the use of MATES prior to the appraisal. Instead, they were instructed to independently apply the tool using only the guidance provided in Supplementary File 2. Each participant received a link to a Qualtrics survey containing the tool and was directed to review and then appraise the completeness of reporting of 5-7 meta-analyses using the draft MATES tool (except for KM, who appraised all 50 meta-analyses).

The validation task enabled us to evaluate agreement rates and

<figure><img src="image_2.png" /><figcaption>**Fig. 2.** Overview of the modified Delphi process for developing the MATES items with five steps: 1) conducting a workshop to develop the initial MATES items, 2) surveying the environmental science and evidence synthesis community to evaluate the initially proposed MATES items, 3) conducting a second workshop to interpret the survey results and modify the MATES tool and items accordingly, 4) performing a validation task to measure inter-rater reliability and usability, and 5) holding a third workshop to interpret the validation task results and finalise the MATES items.</figcaption></figure>
<table><caption>**Table 1**

**MATES questionnaire including each item with appraisal criteria**</caption><thead><tr><th>MATES item</th><th>Question and criteria</th></tr></thead><tbody><tr><td><strong>MATES item 1: Effect size formula reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report the formula (or reference for the formula) for each effect size statistic/measure used (e.g., Zr, odds ratio, risk ratio, Hedges' g, Cohen's d, or log response ratio) in the manuscript or supplementary material?</li><li><strong>Criteria for 'Yes'</strong><ul><li>The authors have reported the formula (or reference for the formula) for each effect size statistic/measure used in the manuscript or supplementary material.</li><li><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported the formula (or reference for the formula) for each effect size statistic/measure used in the manuscript or supplementary material.</li><li><strong>Criteria for 'Not Applicable'</strong><ul><li>The authors adopted the effect sizes directly from the included primary studies.</li></ul></li></ul></li></ul></td></tr><tr><td><strong>MATES item 2: Model type reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report the type of the primary or first meta-analytic/meta-regression model (e.g., fixed, random, or mixed-effects models), either descriptively or in statistical notation in the manuscript or supplementary material?</li><li><strong>Criteria for 'Yes'</strong><ul><li>The authors have reported the type of the primary or first meta-analytic/meta-regression model, either descriptively or in statistical notation, in the manuscript or supplementary material.</li><li><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported the type of the primary or first meta-analytic/meta-regression model, either descriptively or in statistical notation, in the manuscript or supplementary material.</li><li><strong>Criteria for 'Not Applicable'</strong><ul><li>There is no 'Not Applicable' option for MATES item 2.</li></ul></li></ul></li></ul></td></tr><tr><td><strong>MATES item 3: Effect size dependence (non-independence) reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report the dependency structure (e.g., multiplicity or repeated measure) of effect size estimates (e.g., acknowledge multiple effect sizes were taken from a single study) in the primary or first meta-analytic/meta-regression model in the manuscript or supplementary material?</li><li><strong>Criteria for 'Yes'</strong><ul><li>The authors have reported whether there is dependence of effect size estimates in the primary or first meta-analytic/meta-regression model in the manuscript or supplementary material. Alternatively, the authors have reported that effect size estimates are assumed to be independent.</li><li><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported whether there is dependence of effect size estimates in the primary or first meta-analytic/meta-regression model in the manuscript or supplementary material.</li></ul></li></ul></li><li><strong>Criteria for 'Not Applicable'</strong><ul><li>There is no 'Not Applicable' option for MATES item 3.</li></ul></li></ul></td></tr><tr><td><strong>MATES item 4: Software/programming language reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report the name or a reference to the software or programming language used to conduct the analyses (i.e., used to conduct the computations, statistical models, and inferences comprising the meta-analysis) in the manuscript or supplementary material?</li><li><strong>Criteria for 'Yes'</strong><ul><li>The authors have reported the name or a reference to the software or programming language used to conduct the analyses in the manuscript or supplementary material.</li><li><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported the name or a reference to the software or programming language used to conduct the analyses in the manuscript or supplementary material.</li></ul></li></ul></li><li><strong>Criteria for 'Not Applicable'</strong><ul><li>There is no 'Not Applicable' option for MATES item 4.</li></ul></li></ul></td></tr><tr><td><strong>MATES item 5: Number of included primary studies reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report the total number of unique primary studies (i.e., separate individual articles) included in the primary or first meta-analytic/meta-regression model in the manuscript or supplementary material?</li><li><strong>Criteria for 'Yes'</strong><ul><li>The authors have reported the total number of unique primary studies synthesised in the primary or first meta-analytic/meta-regression model in the manuscript or supplementary material. The authors may have reported the total number of unique primary studies in a screening flowchart (e.g., PRISMA flowchart).</li><li><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported the total number of unique primary studies synthesised in the primary or first meta-analytic/meta-regression model in the manuscript or supplementary material.</li></ul></li></ul></li><li><strong>Criteria for 'Not Applicable'</strong><ul><li>There is no 'Not Applicable' option for MATES item 5.</li></ul></li></ul></td></tr><tr><td><strong>MATES item 6: Number of included effect sizes reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report the total number of effect size estimates synthesised in the primary or first meta-analytic/meta-regression model in the manuscript or supplementary material?</li><li><strong>Criteria for 'Yes'</strong><ul><li>The authors have reported the total number of effect sizes estimates synthesised in the primary or first meta-analytic/meta-regression model in the manuscript or supplementary material. The authors may have reported the total number of effect sizes in a screening flowchart (e.g., PRISMA flowchart).</li></ul></li></ul></td></tr></tbody></table>

(continued on next page)
<table><caption>**Table 1 (continued)**</caption><thead><tr><th>MATES item</th><th>Question and criteria</th></tr></thead><tbody><tr><td></td><td><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported the total number of effect sizes synthesised in the primary or first meta-analytic/meta-regression model in the manuscript or supplementary material.</li></ul><strong>Criteria for 'Not Applicable'</strong><ul><li>There is no 'Not Applicable' option for MATES item 6.</li></ul></td></tr><tr><td><strong>MATES item 7: Pooled mean effect reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report the pooled mean effect estimate from the primary or first meta-analytic/meta-regression model and its associated error or precision (e.g., standard error, confidence intervals, prediction intervals) using a plot, table, or within text in the manuscript or supplementary material?</li></ul><strong>Criteria for 'Yes'</strong><ul><li>The authors reported the pooled mean effect estimate and its associated error or precision from the primary or first meta-analytic/ meta-regression model using a plot, table, or within text in the manuscript or supplementary material.</li></ul><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported the pooled mean effect estimate and its associated error or precision from the primary or first meta- analytic/meta-regression model using a plot, table, or within text in the manuscript or supplementary material.</li></ul><strong>Criteria for 'Not Applicable'</strong><ul><li>There is no 'Not Applicable' option for MATES item 7.</li></ul></td></tr><tr><td><strong>MATES item 8: Effect size weighting strategy reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report the strategy used to assign effect size weights (e.g., the sampling variance, sampling error, or equivalent) used in the primary or first meta-analytic/meta-regression model in the manuscript or supplementary material?</li></ul><strong>Criteria for 'Yes'</strong><ul><li>The authors have reported the strategy used to assign effect size weights of effect size estimates used in the primary or first meta- analytic/meta-regression model in the manuscript or supplementary material. The authors must report the weighting strategy and not solely the weights of each study.</li></ul><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported the strategy used to assign effect size weights of effect size estimates used in the primary or first meta- analytic/meta-regression model in the manuscript or supplementary material.</li></ul><strong>Criteria for 'Not Applicable'</strong><ul><li>The authors have conducted a meta-analysis without weights (i.e., unweighted meta-analysis).</li></ul></td></tr><tr><td><strong>MATES item 9: Heterogeneity reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report an assessment of heterogeneity (e.g., Q-statistic, τ, I²) between effect size estimates for the primary or first meta- analytic/meta-regression model in the manuscript or supplementary material?</li></ul><strong>Criteria for 'Yes'</strong><ul><li>The authors have reported an assessment of heterogeneity between effect size estimates for the primary or first meta-analytic/ meta-regression model in the manuscript or supplementary material.</li></ul><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported an assessment of heterogeneity between effect size estimates for the primary or first meta-analytic/ meta-regression model in the manuscript or supplementary material.</li></ul><strong>Criteria for 'Not Applicable'</strong><ul><li>The authors report that they conducted a meta-analysis using a fixed-effects model.</li></ul></td></tr><tr><td><strong>MATES item 10: Publication bias/small-study effect reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report a statistical or graphical assessment of publication bias/small-study effects in the manuscript or supplementary material?</li></ul><strong>Criteria for 'Yes'</strong><ul><li>The authors have reported a statistical or graphical assessment of publication bias/small-study effects in the manuscript or supplementary material.</li></ul><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported a statistical or graphical assessment of publication bias/small-study effects in the manuscript or supplementary material.</li></ul><strong>Criteria for 'Not Applicable'</strong><ul><li>The authors report that an assessment of publication bias/small-study effects cannot be conducted or the authors reported that the outcome of interest is assumed to not be under any publication bias selection pressure.</li></ul></td></tr><tr><td><strong>MATES item 11: Sensitivity analysis reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report a sensitivity analysis or subset analysis in addition to the publication bias assessment (e.g., leave-one-out analysis, use of an alternative model, or within-study risk of bias assessment) in the manuscript or supplementary material?</li></ul><strong>Criteria for 'Yes'</strong><ul><li>The authors reported a sensitivity analysis in addition to the publication bias assessment in the manuscript or supplementary material.</li></ul><strong>Criteria for 'No'</strong><ul><li>The authors have <strong>not</strong> reported a sensitivity analysis in addition to the publication bias assessment in the manuscript or supplementary material.</li></ul><strong>Criteria for 'Not Applicable'</strong><ul><li>There is no 'Not Applicable' option for MATES item 11.</li></ul></td></tr><tr><td><strong>MATES item 12: Code reporting</strong></td><td><strong>Question</strong><ul><li>Do the authors report any relevant code used to conduct the analyses (i.e., used to conduct the computations, statistical models, and inferences comprising the meta-analysis) in the manuscript, supplementary material, or in a referenced external repository?</li></ul><strong>Criteria for 'Yes'</strong></td></tr></tbody></table>

(continued on next page)
<table><caption>**Table 1 (continued)**</caption><thead><tr><th>MATES item</th><th>Question and criteria</th></tr></thead><tbody><tr><td></td><td><ul><li>The authors have reported any relevant code used to conduct the analyses in the manuscript, supplementary material, or in a referenced external repository. Common examples of software used to perform meta-analysis, which can generate code or record syntax, include R, Python, SPSS, and STATA.</li></ul><p><strong>Criteria for 'No'</strong></p><ul><li>The authors have <strong>not</strong> reported any relevant code used to conduct the analyses in the manuscript, supplementary material, or in a referenced external repository.</li></ul><p><strong>Criteria for 'Not Applicable'</strong></p><ul><li>The authors have used a GUI/click-based software that does not generate code or record syntax (e.g., Comprehensive Meta-analysis software, OpenMEE). Alternatively, the authors have <strong>not</strong> reported the software used to conduct the analysis (i.e., 'No' to MATES item 4).</li></ul></td></tr><tr><td><strong>MATES item 13: Data reporting/archiving</strong></td><td><p><strong>Question</strong></p><ul><li>Do the authors report the original source statistics (i.e., used to calculate effect sizes and conduct the meta-analysis) extracted from each eligible primary study (i.e., research paper) in the manuscript, supplementary material, or in a referenced external repository?</li></ul><p><strong>Criteria for 'Yes'</strong></p><ul><li>The authors have reported the original source statistics extracted from each eligible primary study in the manuscript, supplementary material, or in a referenced external repository. Alternatively, the authors have stated that the raw data is sensitive and cannot be shared.</li></ul><p><strong>Criteria for 'No'</strong></p><ul><li>The authors have <strong>not</strong> reported the raw data extracted from each eligible primary study, the manuscript, supplementary material, or in a referenced external repository. The authors have reported that the raw data is available upon request.</li></ul><p><strong>Criteria for 'Not Applicable'</strong></p><ul><li>There is no 'Not Applicable' option for MATES item 13.</li></ul></td></tr><tr><td><strong>MATES item 14: Variable description reporting</strong></td><td><p><strong>Question</strong></p><ul><li>Do the authors report descriptions of the variables extracted from each eligible primary study (i.e., research paper) in the manuscript, supplementary material, or in a referenced external repository?</li></ul><p><strong>Criteria for 'Yes'</strong></p><ul><li>The authors have reported descriptions of the variables extracted from each eligible primary study in the manuscript, supplementary material, or in a referenced external repository.</li></ul><p><strong>Criteria for 'No'</strong></p><ul><li>The authors have <strong>not</strong> reported descriptions of the variables extracted from each eligible primary study in the manuscript, supplementary material, or in a referenced external repository.</li></ul><p><strong>Criteria for 'Not Applicable'</strong></p><ul><li>The authors have reported that the data cannot be provided because the data is sensitive.</li></ul></td></tr></tbody></table>

implementation time as each meta-analysis was appraised by at least three independent reviewers. We calculated the percentage of agreement and intra-class correlation (ICC) – for binary data (i.e., the Yes / No format). ICC can be defined as the proportion of total variance accounted for by differences among appraisers – for the results of each item. We then estimated a study-level Cohen's Kappa statistic to show within-study agreement (i.e., show that repeatability may vary between appraised meta-analyses). Finally, each participant provided the start and end time for their appraisal of each study, which was then used to estimate the implementation time of MATES. All analyses were conducted in the R Statistical Environment version 4.2.1 (R Core Team, 2022) using RStudio build 576 (R Studio Team, 2020). The rptR v0.9.22 (Stoffel et al., 2017) and irr v0.84.1 (Gamer and Lemon, 2019) packages were used to calculate the ICC and Kappa statistics, respectively. All data visualisations were created using ggplot2 v3.5.1 (Wickham, 2016).

## 2.5. Workshop 3

To interpret the results and feedback from the validation task, KM led a discussion group with several MATES authors and held individual meetings with core team members (CW, DM, EG, EAH, JH, LR, MG, ML, MP, PW, PPollo, PPottier, RO, SNakagawa, and YY). The agreement rate, implementation time, and additional feedback from the validation task were reviewed during these discussions. As a result, KM made adjustments, such as refining the wording for clarity to enhance agreement rates and address any remaining concerns (see Supplementary File 3 for details on these adjustments).

## 3. Results

The MATES questionnaire consists of 14 items that reflect key methodological elements that can be used to appraise the completeness

with which a meta-analysis in the environmental sciences has been reported (Table 1). The MATES appraisal items are primarily assessed with a straightforward binary approach: 'Yes' if the item is reported and 'No' if it is not. For some items, a 'Not Applicable' option is allowed. A full checklist is provided in Supplementary File 1, and a detailed explanation and elaboration of each MATES item is provided in Supplementary File 2.

### 3.1. Validation task

Assessor agreement rates are presented in Table 2. Overall, 11 of the 14 MATES items achieved agreement rates above 80 %, with particularly high levels observed for reporting publication bias (item 10), heterogeneity (item 9), and sensitivity analysis (item 11), each exceeding an average of 90 % agreement. In contrast, lower levels of agreement were found for the effect size weighting strategy (item 8), variable description reporting (item 14), and data reporting/archiving (item 13). No item fell below a 75 % agreement rate. The lower agreement may stem from subjective interpretations by the assessors or slight wording ambiguities in the previous phrasing of the items. In response to community feedback, we made adjustments to these items to enhance the clarity of each item and reduce subjective interpretations. Across studies, the mean Cohen's Kappa for appraisals was 0.57 (95 % CI = [0.519, 0.624]), indicating moderate agreement. The median time required to complete an appraisal was 14.00 ± 11.77 min (including KM) and 17 ± 12.23 min (excluding KM).

## 4. Discussion

The Meta-analysis Appraisal Tool for Environmental Sciences (MATES) is the first community-driven tool designed to appraise the completeness with which a meta-analysis in the environmental sciences
<caption>**Table 2**</caption>

Rates of agreement among appraisers and inter-class correlations for each MATES item. The intra-class correlation (ICC) for binary data can be defined as the proportion of total variance accounted for by differences among appraisers (Nakagawa and Schielzeth, 2010).

<table><thead><tr><td>MATES Item</td><td>Agreement rate (%)</td><td>ICC [95 % CI]</td></tr></thead><tbody><tr><td>MATES item 1: Effect size statistic formula reporting</td><td>86.2</td><td>0.947 [0.851, 0.991]</td></tr><tr><td>MATES item 2: Model type reporting</td><td>90.8</td><td>0.939 [0.802, 0.991]</td></tr><tr><td>MATES item 3: Effect size dependence (non-independence) reporting</td><td>86.3</td><td>0.918 [0.725, 0.987]</td></tr><tr><td>MATES item 4: Software/programming language reporting</td><td>96.0</td><td>0.318 [0.134, 0.504]</td></tr><tr><td>MATES item 5: Number of included primary studies reporting</td><td>96.6</td><td>0.541 [0.345, 0.724]</td></tr><tr><td>MATES item 6: Number of included effect sizes reporting</td><td>80.4</td><td>0.499 [0.159,0.895]</td></tr><tr><td>MATES item 7: Pooled mean effect reporting</td><td>94.0</td><td>0.167 [0.032,0.333]</td></tr><tr><td>MATES item 8: Effect size weighting strategy reporting</td><td>76.4</td><td>0.235 [0.067, 0.459]</td></tr><tr><td>MATES item 9: Heterogeneity reporting</td><td>97.8</td><td>0.932 [0.800, 0.986]</td></tr><tr><td>MATES item 10: Publication bias/small-study effect reporting</td><td>98.8</td><td>0.986 [0.961, 0.997]</td></tr><tr><td>MATES item 11: Sensitivity analysis reporting</td><td>90.3</td><td>0.993 [0.982, 0.999]</td></tr><tr><td>MATES item 12: Code reporting</td><td>96.1</td><td>0.740 [0.561, 0.963]</td></tr><tr><td>MATES item 13: Data reporting/archiving</td><td>78.6</td><td>0.221 [0.069, 0.357]</td></tr><tr><td>MATES item 14: Variable description reporting</td><td>77.3</td><td>0.057 [0.000, 0.220]</td></tr></tbody></table>

has been reported. Developed through an adapted Delphi process, MATES is representative of the current opinions of the environmental science and research synthesis community. The validation task confirmed that the tool produces repeatable outcomes and is time-efficient to implement. To further support uptake, we developed a user-friendly training tool and translated the checklist into multiple languages to increase accessibility (see https://kylemorrisonisshiny99.shinyapps.io/MATES_shiny/).

## 4.1. Application of the checklist and interpretation of outcomes

Here, we outline how to apply the MATES questionnaire and how the outcomes can be interpreted (Fig. 3). To accurately appraise the completeness with which a meta-analysis has been reported using MATES, we recommend that users first familiarise themselves with each MATES item. Reviewing the items in advance helps minimise errors and improves the accuracy of appraisal outcomes. Once users are

comfortable with the checklist, they should then familiarise themselves with the meta-analysis under review, as some MATES items are often reported in supplementary files or in external repositories. Furthermore, involving a second appraiser can improve the reliability of appraisal outcomes. Working collaboratively allows disagreements to be resolved through discussion and helps minimise errors, as shown in abstract screening studies (Wang et al., 2023).

To help readers interpret MATES outcomes, we suggest that users record the location or quote each item reported. We propose that citing the location or providing quotations is the most transparent approach to appraising completeness of reporting using MATES, as this enables others to easily verify the information being appraised. While this represents best practice, we acknowledge that it may also increase the time required to carry out the appraisal. Next, we encourage users to interpret the reporting of each item individually rather than relying on a total score, as appraising items collectively may reduce the users' ability to identify the strengths or weaknesses of specific elements with which a

<figure><img src="image_3.png" /><figcaption>**Fig. 3.** The process of appraising meta-analyses using the MATES tools. Start by reading the MATES resources, which include the statement, elaboration, and item checklist. Then, use the MATES training tool on the ShinyApp platform to familiarise yourself with the appraisal process. Once trained, read the meta-analysis you wish to evaluate. Next, apply the MATES tool using the ShinyApp platform for your appraisal. After completing the appraisal, interpret the outcomes for each item. If one needs to appraise another meta-analysis, repeat the process by reading the next paper and evaluating the completeness of reporting using MATES.</figcaption></figure>
meta-analysis has been reported (Greenland, 2001; Shea et al., 2017).

We developed and implemented MATES using the primary or first meta-analytic/meta-regression model reported in each study to ensure consistency of appraisal. This criterion was applied only to items explicitly referring to the “primary or first meta-analytic/meta-regression model”. However, if a study includes multiple meta-analytic/meta-regression models, the checklist can also be applied to secondary models, if deemed appropriate by the user. This should be done with caution, as it is plausible that some items are less likely to be reported for additional analyses within the same study, given that they are often not conducted.

As discussed, MATES is designed to appraise the completeness with which a meta-analysis has been reported. Therefore, its results should not be interpreted as appraising the methodological quality or rigour. A meta-analysis may demonstrate excellent reporting of poorly conducted methods, or conversely, poor reporting of well-conducted methods (Schulz et al., 2014). For instance, authors may report that they assessed publication bias, but the assessment itself may have been conducted incorrectly. While completeness of reporting, methodological quality, and rigour are related concepts, they are not synonymous.

## 4.2. Potential uses of MATES

We have highlighted below potential situations where we believe MATES would be particularly useful.

First, journal editors could adopt MATES as a triage tool at the submission stage to systematically appraise the completeness of reporting, including the provision of data, code, and other essential materials, before initiating peer review. Employing MATES early in the submission process can therefore ensure that there is adequate reporting of the methodology, freeing peer reviewers to focus their efforts on evaluating the methodological quality and rigor. Insufficient reporting at the time of submission often leads to delays in the peer review process, necessitating multiple rounds of revision—initially to address deficiencies in reporting and subsequently to appraise methodological quality/rigor (Lotriet, 2012). As an early time-efficient screening tool, MATES can support the systematic integration of reporting standards into the editorial workflow (Whaley and Roth, 2022).

Second, MATES can be used in meta-reviews by providing a repeatable and timely tool to appraise the completeness of reporting. This appraisal of reporting is distinct from risk of bias or methodological quality critical appraisal (Higgins et al., 2011) and is better suited to reflect the transparency of evidence (Nosek et al., 2015). This interest in the completeness of reporting and hence transparency of meta-analysis methodologies is of growing interest due to the role of transparency in reproducibility and FAIR principles (Goodman et al., 2016; Kambouris et al., 2024; Wilkinson et al., 2016). Moreover, MATES can be incorporated into higher-level syntheses (e.g., meta-reviews and second-order meta-analyses) as a form of sensitivity analysis, allowing authors to examine how variations in reporting might influence their conclusions.

Third, recent innovations have explored the use of large language models (LLMs) to support research reporting, such as the FAIR-checker (Gaignard et al., 2023). These approaches typically employ checklist-based appraisal methods within an LLM interface but depend on a well-defined appraisal framework developed by domain experts. MATES offers such a framework and thus may also provide a foundation for future machine-assisted approaches aimed at improving the completeness with which meta-analyses are reported (Whaley et al., 2023).

We propose that MATES can be utilised in various other contexts—by readers of meta-analyses, policymakers, or stakeholders interested in the completeness of reporting.

## 4.3. Repeatability of MATES compared to other appraisal tools

Our findings demonstrate that, overall MATES outcomes are highly repeatable. The lowest observed agreement rate was 76.4 %, while 8

items yielded repeatability estimates exceeding 90 %. This finding is competitive and, in many cases, suggests more repeatable than items in other appraisal tools such as CEESAT (Woodcock et al., 2014) and AMSTAR2 (Shea et al., 2017). Notably, MATES achieved these estimates using appraisers without prior training on the tool, distinguishing it from the approach employed by CEESAT, which reported an average agreement rate of 79 % after training (Woodcock et al., 2014). The improved repeatability is likely due to the simplified binary system used in MATES, making its items easier to appraise compared to other appraisal tools that employ scaled rating systems (e.g., red, amber, green). Similarly, we found that MATES was time-efficient to use (17.00 min ± 12.23 min) and competitive against existing tools (AMSTAR = 15–32 min). We appreciate that other appraisal tools that appraise the methodological quality/rigour require such scaled systems which, in turn, may reduce repeatability and require more time to implement. Overall, the validation task revealed that MATES is repeatable and time-efficient and highly competitive against currently existing appraisal tools.

## 4.4. Strengths and challenges of the development methodology

Through the development of MATES, we gained valuable insights into the strengths and challenges of using the Delphi process for creating appraisal tools.

A key strength of our approach was its online hybrid format, which enabled us to gather contributions from 16 countries (Fig. S1). The workshops provided broad guidance on the tool’s framework, potential applications, and assessment areas, while the surveys offered detailed insights on specific items and phrasing. These methods enriched our understanding and contributed to the development of a more robust tool.

Despite these strengths, we also encountered some challenges that provide scope for future studies. The methodology was time-intensive and required careful planning for multiple workshops, a survey, and a validation task. While fruitful for gathering diverse perspectives, the workshops and survey generated a large amount of qualitative evidence, which was occasionally conflicting. Interpreting this extensive feedback fairly and making appropriate adjustments to accommodate all viewpoints was a complex task. Additionally, the methodology carried some risk because the project’s success relied heavily on the active participation of the environmental science and evidence synthesis community. However, their engagement significantly enhanced the tool’s relevance and effectiveness. Although we successfully gathered contributions from 16 countries, future efforts could aim to capture an even broader range of perspectives and expertise across different aspects of meta-analysis methodology. Furthermore, the validation task was conducted on a sample of 50 recently published meta-analyses in environmental sciences. As MATES is applied to a broader range of studies across different time periods and disciplines, additional implementation challenges may emerge.

In summary, while the Delphi-like process is valuable for developing appraisal tools and guidelines and gathering diverse perspectives, it is time-intensive, and incorporating large amounts of qualitative feedback can be challenging. Nonetheless, we feel that the benefits outweigh these challenges, ultimately strengthening the quality and applicability of MATES.

## 5. Concluding remarks

The Meta-Analysis Appraisal Tool for Environmental Sciences (MATES) provides a consistent and practical tool to appraise the completeness with which a meta-analysis is reported. Developed through a community-driven, iterative Delphi process with input from environmental scientists, MATES focuses specifically on the transparent reporting of methodological aspects. By supporting readers, reviewers, stakeholders, and editors, the tool has the potential to raise reporting
standards across the field. Broad adoption of MATES will not only expose key weaknesses in current reporting practices but also drive greater transparency and reproducibility in meta-analyses.

## Declaration of generative AI and AI-assisted technologies in the writing process

While preparing this work, the authors used GPT 4.0, GPT 4.5, GPT o3, GPT o1, and GPT 5, created by OpenAI, and Gemini by Google to enhance clarity, readability, and writing flow. Generative AI was also used to aid in annotating code. After using the tools, the authors reviewed and edited the content as needed and take full responsibility for the publication's content.

## CRediT authorship contribution statement

**Kyle Morrison:** Writing – original draft, Visualization, Validation, Software, Resources, Project administration, Methodology, Investigation, Formal analysis, Data curation, Conceptualization. **Patrice Pottier:** Writing – review & editing, Investigation. **Pietro Pollo:** Writing – review & editing, Investigation. **Lorenzo Ricolfi:** Writing – review & editing, Investigation. **Coralie Williams:** Writing – review & editing, Investigation. **Yefeng Yang:** Writing – review & editing, Investigation. **Damien Beillouin:** Writing – review & editing, Investigation. **Simone Jaqueline Cardoso:** Writing – review & editing, Investigation. **Brian Veronica Ferreira:** Writing – review & editing, Investigation. **Brian Galager:** Writing – review & editing, Investigation. **Jelaine L. Gan:** Writing – review & editing, Investigation. **Guang Hao:** Writing – review & editing, Investigation. **Mojtaba Keikha:** Writing – review & editing, Investigation. **Betina Kozlowsky-Suzuki:** Writing – review & editing, Investigation. **T.M. Kiran Kumara:** Writing – review & editing, Investigation. **Francesco Latterini:** Writing – review & editing, Investigation. **Alexandro B. Leverkus:** Writing – review & editing, Investigation. **Erin L. Macartney:** Writing – review & editing, Investigation. **Silvina Magdalena Manrique:** Writing – review & editing, Investigation. **April Robin Martining:** Writing – review & editing, Investigation. **Ayumi Mizuno:** Writing – review & editing, Investigation. **Shanika Nanayakkara:** Writing – review & editing, Investigation. **Evangelia Ntzani:** Writing – review & editing, Investigation. **Dakis-Yaoba Ouédraogo:** Writing – review & editing, Investigation. **Edward Pursell:** Writing – review & editing, Investigation. **Zachary Simpson:** Writing – review & editing, Investigation. **Harriet Sleight:** Writing – review & editing, Investigation. **Kok Sin Woon:** Writing – review & editing, Investigation. **Ziqian Xia:** Writing – review & editing, Investigation. **Mona Ghannad:** Writing – review & editing, Conceptualization. **Eliza Grames:** Writing – review & editing, Conceptualization. **Emily Alden Hennessy:** Writing – review & editing, Conceptualization. **Joanna IntHout:** Writing – review & editing, Conceptualization. **David Moher:** Writing – review & editing, Conceptualization. **Matthew J. Page:** Writing – review & editing, Conceptualization. **Paul Whaley:** Writing – review & editing, Conceptualization. **Malgorzata Lagisz:** Writing – review & editing, Visualization, Supervision, Methodology, Investigation, Conceptualization. **Shinichi Nakagawa:** Writing – review & editing, Supervision, Methodology, Investigation, Formal analysis, Conceptualization.

## Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: YY became Associate Editor of Environmental International after submitting this article. Given role as Associate Editor, YY has no involvement in the peer-review of this article and has no access to information regarding its peer-review. We confirm there are no other conflicts of interest to declare.

## Acknowledgements

We would like to thank all the anonymous survey participants who contributed to the development of MATES. We would like to extend our thanks to all the authors of the meta-analyses used during the implementation task. We would like to thank the financial support provided, which facilitated the completion of this work: DP230101248 (Malgorzata Lagisz), TED2021-130976B-100, BIOD22_002 (Alexandro B. Leverkus), RGPIN-2019-05520, RN513790-510021 (April Robin Martining), K01AA028536 (Emily Alden Hennessy), NE/S00713X/1 (Harriet Sleight), CEECIND/02484/2018, UIDP/04292/2020, UIDB/04292/2020, LA/P/0069/2020 (Verónica Ferreira), and CERC-2022-00074 (Shinichi Nakagawa). We also acknowledge the structural support provided by our colleagues at the Evolution & Ecology Research Centre and the School of Biological, Earth and Environmental Sciences at the University of New South Wales, Sydney, Australia.

## Appendix A. Supplementary material

Supplementary data to this article can be found online at https://doi.org/10.1016/j.envint.2025.109935.

## Data availability

All data and code are provided in: https://github.com/KyleMorrison99/MATES.

## References

* Chalmers, I., Glasziou, P., 2009. Avoidable waste in the production and reporting of research evidence. *Lancet* 374.

* Fraser, H., Parker, T., Nakagawa, S., Barnett, A., Fidler, F., 2018. Questionable research practices in ecology and evolution. *PLoS One* 13 (7), e0200303. https://doi.org/10.1371/journal.pone.0200303.

* Gaignard, A., Rosnet, T., De Lamotte, F., Lefort, V., Devignes, M.-D., 2023. FAIR-Checker: supporting digital resource findability and reuse with knowledge graphs and semantic web standards. *J Biomed Semant* 14, 7. https://doi.org/10.1186/s13326-023-00289-5.

* Greenland, S., 2001. On the bias produced by quality scores in meta-analysis, and a hierarchical view of proposed solutions. *Biostatistics* 2 (4), 463–471. https://doi.org/10.1093/biostatistics/2.4.463.

* Gurevitch, J., Koricheva, J., Nakagawa, S., Stewart, G., 2018. Meta-analysis and the science of research synthesis. *Nature* 555, 175–182. https://doi.org/10.1038/nature25753.

* Haddaway, N.R., Macura, B., Whaley, P., Pullin, A.S., 2018. ROSES Reporting standards for systematic evidence syntheses: pro forma, flow-diagram and descriptive summary of the plan and conduct of environmental systematic reviews and systematic maps. *Environ. Evidence* 7, 7. https://doi.org/10.1186/s13750-018-0121-7.

* Moher, D., 2018. Reporting guidelines: doing better for readers. *BMC Med.* 16, 233. https://doi.org/10.1186/s12916-018-1226-0.

* Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., for the PRISMA Group, 2009. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. *BMJ* 339, b2535. https://doi.org/10.1136/bmj.b2535.

* Moher, D., Schulz, K.F., Simera, I., Altman, D.G., 2010. Guidance for developers of health research reporting guidelines. *PLoS Med.* 7 (2), e1000217. https://doi.org/10.1371/journal.pmed.1000217.

* Morrison, K., Yang, Y., Williams, C., Ricolfi, L., Lagisz, M., Nakagawa, S., 2025. Mapping meta-analyses on organochlorine pesticides reveals low methodological quality. *Nat. Sustainability*. https://doi.org/10.1038/s41893-025-01634-5.

* Nakagawa, S., Schielzeth, H., 2010. Repeatability for Gaussian and non-Gaussian data: a practical guide for biologists. *Biol. Rev.* 85, 935–956. https://doi.org/10.1111/j.1469-185X.2010.00141.x.

* Nakagawa, S., Yang, Y., Macartney, E.L., Spake, R., Lagisz, M., 2023a. Quantitative evidence synthesis: a practical guide on meta-analysis, meta-regression, and publication bias tests for environmental sciences. *Environ. Evidence* 12, 8. https://doi.org/10.1186/s13750-023-00301-6.

* Nakagawa, S., Ivimey-Cook, E.R., Grainger, M.J., et al., 2023b. Method Reporting with Initials for Transparency (MeRIT) promotes more granularity and accountability for author contributions. *Nat. Commun.* 14, 1788. https://doi.org/10.1038/s41467-023-37039-1.

* Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L., Tetzlaff, J.M., Akl, E.A., Brennan, S.E., Chou, R., Glanville, J., Grimshaw, J.M., Hróbjartsson, A., Lalu, M.M., Li, T., Loder, E.W., Mayo-Wilson, E., McDonald, S., McGuinness, L.A., Stewart, L.A., Thomas, J., Tricco, A.C., Welch, V.A., Whiting, P., Moher, D., 2021. The PRISMA 2020 statement: an updated guideline for
reporting systematic reviews. Int. J. Surg. 88, 105906. https://doi.org/10.1016/j.ijsu.2021.105906.

Parker, T.H., Forstmeier, W., Koricheva, J., Fidler, F., Hadfield, J.D., Chee, Y.E., Kelly, C. D., Gurevitch, J., Nakagawa, S., 2016. Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31 (9), 711–719. https://doi.org/10.1016/j.tree.2016.07.002.

Shea, B.J., Reeves, B.C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, R., Tugwell, P., Welch, V., Kristjansson, E., Henry, D.A., 2017. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 358, j4008. https://doi.org/10.1136/bmj.j4008.

Stoffel, M.A., Nakagawa, S., Schielzeth, H., 2017a. rptR: Repeatability estimation and variance decomposition by generalized linear mixed-effects models. Methods Ecol. Evol. 8, 1639–1644. https://doi.org/10.1111/2041-210X.12797.

Whaley, P., Wattam, S., Scott, A.M., Based Toxicology Collaboration, 2023. Using bench protocol platforms to improve compliance with systematic review guidance documents and reporting checklists: proof of concept. Evidence-Based Toxicol. 1, 2259938. https://doi.org/10.1080/2833373X.2023.2259938.

Whiting, P., Savović, J., Higgins, J.P.T., Caldwell, D.M., Reeves, B.C., Shea, B., Davies, P., Kleijnen, J., Churchill, R., 2016. ROBIS: A new tool to assess risk of bias in systematic reviews was developed. J. Clin. Epidemiol. 69, 225–234. https://doi.org/10.1016/j.jclinepi.2015.06.005.

Woodcock, P., Pullin, A.S., Kaiser, M.J., 2014. Evaluating and improving the reliability of evidence syntheses in conservation and environmental science: a methodology. Biol. Conserv. 176, 54–62. https://doi.org/10.1016/j.biocon.2014.04.020.

Barrett, D., Heale, R., 2020. What are Delphi studies? Evid Based Nurs 23, 68–69. https://doi.org/10.1136/ebnurs-2020-130330.

Goodman, S.N., Fanelli, D., Ioannidis, J.P.A., 2016. What does research reproducibility mean? Sci. Transl. Med. 8. https://doi.org/10.1126/scitranslmed.aaf5027.

Higgins, J.P.T., Altman, D.G., Gotzsche, P.C., Juni, P., Moher, D., Oxman, A.D., Savovic, J., Schulz, K.F., Weeks, L., Sterne, J.A.C., Cochrane Bias Methods Group, Cochrane Statistical Methods Group, 2011. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343, d5928. https://doi.org/10.1136/bmj.d5928.

Kambouris, S., Wilkinson, D.P., Smith, E.T., Fidler, F., 2024. Computationally reproducing results from meta-analyses in ecology and evolutionary biology using shared code and data. PLoS ONE 19, e0300333.

Lotriet, C., 2012. Reviewing the review process: Identifying sources of delay. AMJ 5, 26–29. https://doi.org/10.4066/AMJ.2012.1165.

Morrison, K., Lagisz, M., Liu, M., Santana, M.S., Nakagawa, S., Yang, Y., 2025. A Meta-epidemiological study of the Diversity. Transparency and Accessibility of PRISMA extension development. https://doi.org/10.31222/osf.io/z2wpv_v1.

Nosek, B.A., Alter, G., Banks, G.C., Borsboom, D., Bowman, S.D., Breckler, S.J., Buck, S., Chambers, C.D., Chin, G., Christensen, G., Contestable, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D.P., Hesse, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Mabry, P., Madon, T., Malhotra, N., Mayo-Wilson, E., McNutt, M., Miguel, E., Paluck, E.L., Simonsohn, U., Soderberg, C., Spellman, B.A., Turitto, J., VandenBos, G., Vazire, S., Wagenmakers, E.J., Wilson, R., Yarkoni, T., 2015. Promoting an open research culture. Science 348, 1422–1425. https://doi.org/10.1126/science.aab2374.

Page, M.J., Moher, D., 2017. Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review. Syst Rev 6, 263. https://doi.org/10.1186/s13643-017-0663-8.

Pollo, P., Martinig, A., Mizuno, A., Morrison, K., Pottier, P., Ricolfi, L., Tam, J., Williams, C., Yang, Y., Drobniak, S., Lagisz, M., Nakagawa, S., 2025. Harnessing meta-analyses' insights in ecology and evolution research. https://doi.org/10.32942/X2WP5P.

Schulz, K.F., Moher, D., Altman, D.G., 2014. Ambiguities and Confusions Between Reporting and Conduct, in: Moher, D., Altman, D.G., Schulz, K.F., Simera, I., Wager, E. (Eds.), Guidelines for Reporting Health Research: A User's Manual. Wiley, pp. 41–47. https://doi.org/10.1002/9781118715598.ch5.

Whaley, P., Roth, N., 2022. How we promote rigour in systematic reviews and evidence maps at Environment International. Environment International 170, 107543. https://doi.org/10.1016/j.envint.2022.107543.

## Software references

Gamer, M., Lemon, J., IFPS, 2019. irr: various coefficients of interrater reliability and agreement. R Package Version (84), 1. https://CRAN.R-project.org/package=irr.

R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.

R Studio Team (2020). RStudio: Integrated Development for R. RStudio, PBC, Boston, MA URL http://www.rstudio.com/.

Stoffel, M.A., Nakagawa, S., Schielzeth, H., 2017b. rptR: repeatability estimation and variance decomposition by generalized linear mixed-effects models. Methods Ecol. Evol. 8, 1639–1644. https://doi.org/10.1111/2041-210X.12797.

Wickham, H., 2016. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag, New York https://ggplot2.tidyverse.org.

## Validation task references

Boissiere-O'Neill, T., Lee, W.R., Blake, T.L., Sly, P.D., Vilcins, D., 2024. Exposure to endocrine-disrupting plasticisers and lung function in children and adolescents: a systematic review and meta-analysis. Environ. Res. 243. https://doi.org/10.1016/j.envres.2023.117751.

Cao, Y., Chen, J., Bian, Q., Ning, J., Yong, L., Ou, T., Song, Y., Wei, S., 2023. Genotoxicity evaluation of titanium dioxide nanoparticles in vivo and in vitro: a meta-analysis. Toxics 11. https://doi.org/10.3390/toxics11110882.

Cong, X., Liu, Q., Li, W., Wang, L., Feng, Y., Liu, C., Guo, L.-Q., Wang, L., Shi, C., Li, P., 2023. Systematic review and meta-analysis of breast cancer risks in relation to 2,3,7,8-tetrachlorodibenzo-p-dioxin and per- and polyfluoroalkyl substances. Environ. Sci. Pollut. Res. 30, 86540–86555. https://doi.org/10.1007/s11356-023-28592-9.

Darusman, T., Murdiyarso, D., Anas, I., 2023. Effect of rewetting degraded peatlands on carbon fluxes: a meta-analysis. Mitigation Adapt. Strateg. Global Change 28. https://doi.org/10.1007/s11027-023-10046-9.

Dassou, A.G., Tovignan, S., Vodouhè, F., Vodouhè, S.D., 2024. Meta-analysis of agroecological technologies and practices in the sustainable management of banana pests and diseases. Environ. Dev. Sustain. 26, 21937–21954. https://doi.org/10.1007/s10668-023-03570-w.

Dong, X., Zou, M., Li, C., Luo, H., Zhu, S., Gong, Z., 2024. Leptin receptor Gln223Arg and Lys109Arg polymorphisms may be associated with HBV-related hepatocellular carcinoma risk: a system review and meta-analysis. Environ. Toxicol. https://doi.org/10.1002/tox.24286.

Duque-Cartagena, T., Mundstock, E., Dalla, M.D.B., Padoin, A.V., Canon-Montanez, W., Mattiello, R., 2023. The role of environmental pollutants in body composition: systematic review and meta-analysis. Environ. Res. 228. https://doi.org/10.1016/j.envres.2023.115840.

Dzhambov, A.M., 2023. Comments on “the association between road traffic noise and type 2 diabetes: a systematic review and meta-analysis of cohort studies” by Wu, Shan et al., DOI doi: 10.1007/s11356-023-25926-5. Environ. Sci. Pollut. Res. 30, 88235–88237. https://doi.org/10.1007/s11356-023-28830-0.

Ellis, L.B., Molina, K., Robbins, C.R., Freisthler, M., Sgargi, D., Mandrioli, D., Perry, M.J., 2023. Adult organophosphate and carbamate insecticide exposure and sperm concentration: a systematic review and meta-analysis of the epidemiological evidence. Environ. Health Perspect. 131. https://doi.org/10.1289/EHP12678.

Fu, M., Ren, Z., Liu, J., He, Y., Liu, C., Li, Y., Cao, X., 2023. Association between cadmium exposure and the risk of cardiovascular outcomes: a meta-analysis. Hum. Ecol. Risk Assess. 29, 1091–1108. https://doi.org/10.1080/10807039.2023.2236725.

Ghorbani Nejad, B., Rahimi Kakavandi, N., Mirzaei, S., Rastegar Pouyani, N., Habibian Sezavar, A., 2024. Exposure to atrazine by drinking water and the increased risk of neonatal complications in consequence: a meta-analysis. Int. J. Environ. Health Res. 34, 1443–1452. https://doi.org/10.1080/09603123.2023.2219980.

Gilano, G., Sako, S., Dileba, T., Dekker, A., Fijten, R., 2023. Assessing the effect of mHealth on child feeding practice in African countries: systematic and meta-analysis. J. Health Popul. Nutr. 42. https://doi.org/10.1186/s41043-023-00487-y.

Goodman, C.V., Green, R., DaCosta, A., Flora, D., Lanphear, B., Till, C., 2023. Sex difference of pre- and post-natal exposure to six developmental neurotoxicants on intellectual abilities: a systematic review and meta-analysis of human studies. Environ. Health Global Access Sci. Sour. 22. https://doi.org/10.1186/s12940-023-01029-z.

Gui, D., Zhang, Y., Lv, J., Guo, J., Sha, Z., 2024. Effects of intercropping on soil greenhouse gas emissions-a global meta-analysis. Sci. Total Environ. 918. https://doi.org/10.1016/j.scitotenv.2024.170632.

Guimarães, L.S.F., de Carvalho-Junior, L., Façanha, G.L., Resende, N.D.S., Neves, L.M., Cardoso, S.J., 2023. Meta-analysis of the thermal pollution caused by coastal nuclear power plants and its effects on marine biodiversity. Mar. Pollut. Bull. 195. https://doi.org/10.1016/j.marpolbul.2023.115452.

Han, Z., Zhao, C., Li, Y., Xiao, M., Yang, Y., Zhao, Y., Liu, C., Liu, J., Li, P., 2024. Ambient air pollution and vision disorder: a systematic review and meta-analysis. Toxics 12. https://doi.org/10.3390/toxics12030209.

Hao, X., Yang, J., Dong, S., He, F., Zhang, Y., 2024. The influence of grazing intensity on soil organic carbon storage in grassland of China: a meta-analysis. Sci. Total Environ. 924. https://doi.org/10.1016/j.scitotenv.2024.171439.

Hou, D., Cui, X., Liu, M., Qie, H., Tang, Y., Xu, R., Zhao, P., Leng, W., Luo, N., Luo, H., Lin, A., Wei, W., Yang, W., Zheng, T., 2024. The effects of iron-based nanomaterials (Fe NMs) on plants under stressful environments: machine learning-assisted meta-analysis. J. Environ. Manage. 354. https://doi.org/10.1016/j.jenvman.2024.120406.

Husaini, D.C., Reneau, K., Balam, D., 2022. Air pollution and public health in Latin America and the Caribbean (LAC): a systematic review with meta-analysis. Beni-Suef Univ. J. Basic. Appl. Sci. 11. https://doi.org/10.1186/s43088-022-00305-0.

Jin, S., Cui, S., Xu, J., Zhang, X., 2023. Associations between prenatal exposure to phthalates and birth weight: a meta-analysis study. Ecotoxicol. Environ. Saf. 262. https://doi.org/10.1016/j.ecoenv.2023.115207.

Kim, Y., Lee, S., 2023. A systematic review and meta-analysis of the effectiveness of non-face-to-face coaching. Sustainability 15. https://doi.org/10.3390/su15129727.

Lan, H., Hu, Z., Gan, H., Wu, L., Xie, S., Jiang, Y., Ye, D., Ye, X., 2023a. Association between exposure to persistent organic pollutants and pubertal timing in boys and girls: a systematic review and meta-analysis. Ecotoxicol. Environ. Saf. 265. https://doi.org/10.1016/j.ecoenv.2023.115540.

Lan, L., Wei, H., Chen, D., Pang, L., Xu, Y., Tang, Q., Li, J., Xu, Q., Li, H., Lu, C., Wu, W., 2023b. Associations between maternal exposure to perfluoroalkylated substances (PFASs) and infant birth weight: a meta-analysis. Environ. Sci. Pollut. Res. 30, 89805–89822. https://doi.org/10.1007/s11356-023-28458-0.
Lee, J.-M., Jeong, H.-C., Gwon, H.-S., Lee, H.-S., Park, H.-R., Kim, G.-S., Park, D.-G., Lee, S.-I., 2023. Effects of biochar on methane emissions and crop yields in east Asian paddy fields: a regional scale meta-analysis. Sustainability 15. https://doi.org/10.3390/su15129200.

Lefta, R.M., Hjazi, A., Skakodub, A.A., Dham, A., Sapaev, I.B., Khlewee, I.H., Alawadi, A. H., Alsaalamy, A., Habash, R.T., Lari Najafi, M., 2024. A systematic review and meta-analysis of the association between exposure to potentially toxic elements and gestational diabetes mellitus. Env. Sci. Eur. 36. https://doi.org/10.1186/s12302-024-00878-w.

Li, S., Zhao, L., Wang, C., Huang, H., Zhuang, M., 2023. Synergistic improvement of carbon sequestration and crop yield by organic material addition in saline soil: a global meta-analysis. Sci. Total Environ. 891. https://doi.org/10.1016/j.scitotenv.2023.164530.

Liu, H., Ren, W., Ciric, L., Bhatti, M., 2024. Phytotoxicological effects of phytosynthesized nanoparticles: a systematic review and meta-analysis. Crit. Rev. Environ. Sci. Technol. 54, 1237-1257. https://doi.org/10.1080/10643389.2024.2303299.

Liu, L., Wang, Z., Ye, Y., Qi, K., 2023a. Effects of agricultural land types on microplastic abundance: a nationwide meta-analysis in China. Sci. Total Environ. 892. https://doi.org/10.1016/j.scitotenv.2023.164400.

Liu, W., Xu, C., Peng, Y., Xu, X., 2023b. Evolution of tourism risk communication: a bibliometric analysis and meta-analysis of the antecedents of communicating risk to tourists. Sustainability 15. https://doi.org/10.3390/su15129693.

Ma, N., Liu, X., Wang, L., Liu, G., 2024. A meta-analysis on the mitigation measures of methane emissions in Chinese rice paddy. Resour. Conserv. Recycl. 202. https://doi.org/10.1016/j.resconrec.2023.107379.

Margiana, R., Hamoud Alshahrani, S., Kayumova, D., Hussien Radie Alawadi, A., Hjazi, A., Alsalamy, A., Qasim, Q.A., Juyal, A., Garousi, N., 2024. Association between maternal exposure to arsenic by drinking water during pregnancy and risk of preterm birth: a systematic review and meta-analysis. Int. J. Environ. Health Res. 34, 2947-2956. https://doi.org/10.1080/09603123.2023.2280155.

Matias, G., Cagnacci, F., Rosalino, L.M., 2024. FSC forest certification effects on biodiversity: a global review and meta-analysis. Sci. Total Environ. 908. https://doi.org/10.1016/j.scitotenv.2023.168296.

MoghaddamHosseini, V., Ebrahimi Aval, H., Lari Najafi, M., Lotfi, H., Heydari, H., Miri, M., Dadvand, P., 2023. The association between exposure to polycyclic aromatic hydrocarbons and birth outcomes: a systematic review and meta-analysis of observational studies. Sci. Total Environ. 905. https://doi.org/10.1016/j.scitotenv.2023.166922.

Ou, X.-X., Wang, X., Zhan, X.-L., Shen, S.-L., Karatela, S., Jing, J., Cai, L., Liu, R.-Q., Lin, L.-Z., Dong, G.-H., 2024. The associations of secondhand smoke exposure with neurodevelopmental disorders and critical time window identification: a systematic review and meta-analysis. Sci. Total Environ. 913. https://doi.org/10.1016/j.scitotenv.2023.169649.

Puzzo, G., Prati, G., 2024. Psychological correlates of e-waste recycling intentions and behaviors: a meta-analysis. Resour. Conserv. Recycl. 204. https://doi.org/10.1016/j.resconrec.2024.107462.

Rezaeian, S., Taghipour, A., Bahadory, S., Mogharab, V., Abdoli, A., 2023. Global prevalence and genotype distribution of *Microsporidia* spp. in various consumables: a systematic review and meta-analysis. J. Water Health 21, 895-914. https://doi.org/10.2166/wh.2023.042.

Safaei, P., Sleman Ali, H., Mohajer, A., Sarwar Karim, H., Salamzadeh, J., Sadighara, P., Ghanati, K., 2023. Prevalence and concentration of ochratoxin a in spices: a global systematic review and meta-analysis study. Int. J. Environ. Anal. Chem. https://doi.org/10.1080/03067319.2023.2194021.

Shahid, M., 2024. Response to comments on "effect of soil amendments on trace element-mediated oxidative stress in plants: meta-analysis and mechanistic interpretations"

by Shahid (2021). J. Hazard. Mater. 465. https://doi.org/10.1016/j.jhazmat.2023.133043.

Shahid, M., 2021. Effect of soil amendments on trace element-mediated oxidative stress in plants: meta-analysis and mechanistic interpretations. J. Hazard. Mater. 407. https://doi.org/10.1016/j.jhazmat.2020.124881.

Shi, B., Yang, R., Tian, W., Lu, M., Wang, X., 2024a. Factors influencing cadmium accumulation in plants after inoculation with rhizobacteria: a meta-analysis. Sci. Total Environ. 916. https://doi.org/10.1016/j.scitotenv.2024.170277.

Shi, S., Cao, C., Jiang, Y., 2024b. Letter to the editor: comments on "effect of soil amendments on trace element-mediated oxidative stress in plants: meta-analysis and mechanistic interpretations" by Shahid (2021). J. Hazard. Mater. 465. https://doi.org/10.1016/j.jhazmat.2023.133042.

Srivastava, S.K., Mishra, A., Singh, S., Jaiswal, D., 2023. Household food waste and theory of planned behavior: a systematic review and meta-analysis. Environ. Sci. Pollut. Res. 30, 97645-97659. https://doi.org/10.1007/s11356-023-29141-0.

Sun, J., Zhou, T., Wang, D., 2023. Effects of urbanisation on PM2.5 concentrations: a systematic review and meta-analysis. Sci. Total Environ. 900. https://doi.org/10.1016/j.scitotenv.2023.166493.

Tan, X., Qi, F., Liu, Q., Qie, H., Duan, G., Lin, A., Liu, M., Xiao, Y., 2024. Is Cr(III) re-oxidation occurring in Cr-contaminated soils after remediation: meta-analysis and machine learning prediction. J. Hazard. Mater. 465. https://doi.org/10.1016/j.jhazmat.2023.133342.

Veneri, F., Vinceti, M., Generali, L., Giannone, M.E., Mazzoleni, E., Birnbaum, L.S., Consolo, U., Filippini, T., 2023. Fluoride exposure and cognitive neurodevelopment: systematic review and dose-response meta-analysis. Environ. Res. 221. https://doi.org/10.1016/j.envres.2023.115239.

Wang, K., Sang, G.-Y., Huang, L.-Z., Li, S.-H., Guo, J.-W., 2023. The effectiveness of educational robots in improving learning outcomes: a meta-analysis. Sustainability 15. https://doi.org/10.3390/su15054637.

Wilkinson, M.D., Dumontier, M., Aalbersberg, Ij, J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., Da Silva Santos, L.B., Bourne, P.E., Bouwman, J., Brookes, A.J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C.T., Finkers, R., Gonzalez-Beltran, A., Gray, A.J.G., Groth, P., Goble, C., Grethe, J.S., Heringa, J., 'T Hoen, P.A.C., Hooft, R., Kuhn, T., Kok, R., Kok, J., Lusher, S.J., Martone, M.E., Mons, A., Packer, A.L., Persson, B., Rocca-Serra, P., Roos, M., Van Schaik, R., Sansone, S.-A., Schultes, E., Sengstag, T., Slater, T., Strawn, G., Swertz, M. A., Thompson, M., Van Der Lei, J., Van Mulligen, E., Velterop, J., Waagmeester, A., Wittenburg, P., Wolstencroft, K., Zhao, J., Mons, B., 2016. The FAIR guiding principles for scientific data management and stewardship. Sci. Data 3, 160018. https://doi.org/10.1038/sdata.2016.18.

Wu, Q., Li, G., Zhao, C.-Y., Na, X.-L., Zhang, Y.-B., 2023a. Association between phthalate exposure and obesity risk: a meta-analysis of observational studies. Environ. Toxicol. Pharmacol. 102. https://doi.org/10.1016/j.etap.2023.104240.

Wu, S., Du, W., Zhong, X., Lu, J., Wen, F., 2023b. The association between road traffic noise and type 2 diabetes: a systematic review and meta-analysis of cohort studies. Environ. Sci. Pollut. Res. 30, 39568-39585. https://doi.org/10.1007/s11356-023-25926-5.

Zakrzewski, A.J., Gajewska, J., Chajęcka-Wierzchowska, W., Zaluski, D., Zadernowska, A., 2024. Prevalence of *Listeria monocytogenes* and other *Listeria* species in fish, fish products and fish processing environment: a systematic review and meta-analysis. Sci. Total Environ. 907. https://doi.org/10.1016/j.scitotenv.2023.167912.

Zhao, Y., Zhang, S., Guo, L., Xiao, M., Han, Z., Yang, Y., Wang, B., Li, P., 2024. Association between air pollutants and the risk of sleep disorders: a systematic review and meta-analysis. Aerosol Air Qual. Res. 24. https://doi.org/10.4209/aaqr.230197.