# NMAstudio 2.0: An interactive tool for network meta-analysis to enhance understanding, interpretation, and communication of the findings

Tianqi Yu, Silvia Metelli, Theodoros Papakonstantinou, Anna Chaimani

<figure><img src="image_1.png" /></figure>

## To cite this version:

Tianqi Yu, Silvia Metelli, Theodoros Papakonstantinou, Anna Chaimani. NMAstudio 2.0: An interactive tool for network meta-analysis to enhance understanding, interpretation, and communication of the findings. Research Synthesis Methods, In press, pp.1-14. 10.1017/rsm.2026.10074. hal-05490962v1

HAL Id: hal-05490962

https://hal.science/hal-05490962v1

Submitted on 3 Feb 2026 (v1), last revised 2 Mar 2026 (v2)

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# NMAstudio: An interactive tool for network meta-analysis to enhance understanding, interpretation, and communication of the findings

Tianqi Yu¹, Silvia Metelli¹, Theodoros Papakonstantinou², Anna Chaimani³,¹,*

¹Université Paris Cité, Center of Research in Epidemiology and Statistics, Inserm, Paris, France

²Laboratory of Hygiene, Social and Preventive Medicine and Medical Statistics, School of Medicine, Aristotle University of Thessaloniki, Greece

³Oslo Center for Biostatistics and Epidemiology, Department of Biostatistics, University of Oslo, Oslo, Norway

*Correspondence: anna.chaimani@medisin.uio.no

Word Count: 3933

## Abstract

Network meta-analysis (NMA) is a vital methodology for synthesizing evidence across multiple treatments and informing medical decision-making. However, effective visualization and interpretation of results from large networks of interventions remain challenging, particularly for non-specialists. NMAstudio is an innovative, interactive web application designed to address these difficulties by streamlining NMA workflows and enhancing result visualization. Developed using Python and R, NMAstudio seamlessly integrates with established NMA frameworks. Our exemplar application of NMAstudio using a Cochrane Review comparing several treatments for chronic plaque psoriasis demonstrates its capacity to facilitate all crucial steps of an NMA. The application features an intuitive interface for uploading data, automating analyses, generating interactive visualizations such as network diagrams, forest plots, ranking plots, and producing unique outputs like boxplots for transitivity checks and bi-dimensional forest plots. Most outputs are dynamically linked with the network diagram, enabling users to interactively explore evidence networks, apply advanced filtering, and highlight specific features by selecting nodes or edges within the diagram. While NMAstudio aims to simplify NMAs, it also incorporates steps during the data upload process to mitigate the risk of producing poorly reported NMAs. NMAstudio represents a significant step forward in improving the usability and accessibility of NMA, offering researchers a robust, versatile platform for evidence synthesis. Its integration of advanced features with an emphasis on user experience positions it as a valuable resource for enhancing decision-making and promoting evidence-based practice across diverse contexts.
**Keywords:** comparative effectiveness research; open-access software; multiple treatments; interactive visualizations; data synthesis

## Highlights

### What is already known

* Network meta-analysis (NMA) is a crucial method for synthesizing evidence across multiple treatments and guiding medical decision-making.

* Visualizing and interpreting results from large networks of interventions remain challenging, especially for non-specialists.

### What is new

* NMA studio is an interactive web application designed to simplify NMA workflows and improve result visualization.

* It features a user-friendly interface for uploading data, automating analyses, and creating interactive visualizations.

* Outputs are dynamically linked to the network diagram, enabling users to explore evidence networks, apply filters, and highlight key features interactively.

* To support open research, NMA studio includes functions for saving and sharing uploaded data, making it easier for users to manage and share research projects.

### Potential impact for Research Synthesis Methods readers

* NMA studio enhances understanding and interpretation of results, accounting for risks of bias and data limitations, while fostering data sharing and open science.

* Its advanced features and focus on usability make it a valuable tool for evidence-based decision-making across various research contexts.
## Background

Network meta-analysis (NMA) is a powerful tool for synthesizing evidence on multiple healthcare interventions and is widely used to support clinical recommendations and guidelines. By combining for every comparison evidence that comes directly and indirectly from the available studies within a connected network of interventions, it usually offers more comprehensive and precise findings than individual studies or pairwise meta-analysis do [1]. Beyond estimating relative effects, NMA offers the ability to rank the competing interventions based on specific outcomes and facilitate decision-making, particularly in the presence of many alternative interventions for the condition of interest.

Despite the growing popularity of NMAs and the availability of extensive guidance, criticisms persist mainly due to their complex methodology and assumptions which are not always examined carefully [2-5]. Several empirical studies have identified deficiencies in published NMAs, although an improvement of the overall quality has been observed over time [6, 7]. The significant level of statistical expertise required and the long outputs that NMA typically produces pose a major barrier in understanding and properly considering strengths and limitations of the findings [2]. Additionally, even though several software tools are available for conducting NMAs [8, 9], they do not offer visualizations that allow to easily make the connection between data and results.

Complete reporting is a crucial aspect of every NMA to avoid misinterpretations or over-confidence on uncertain findings. The PRISMA-NMA checklist [10] assists researchers towards this direction, but often it is only partially followed by published NMAs. On the other hand, comprehensive NMAs usually put a lot of the required information into supplementary materials, which can extend to hundreds of pages [11, 12]. Navigating through these extensive and complex outputs can be both time-consuming and confusing, thereby impeding effective communication and decision-making. Typically, the volume of information in an NMA is significantly greater than a pairwise meta-analysis. This presents significant challenges to the principles of open science, which emphasize transparency, accessibility, and reproducibility [13, 14] since it makes it difficult to share data and findings in a user-friendly manner.

In recent years, several new graphical and tabular visualizations for NMA have been introduced [15, 16]. For example, contribution plots highlight the balance between direct and indirect information in the results [17]; radial ranking plots integrate network graphs with the relative ranking of the interventions [18]; the Kilim plot depicts absolute risk for each outcome and treatment alongside the strength of statistical evidence [19]. However, all these visualizations typically need to be run separately in a scripting environment such as R or STATA and thus lack interactivity and interoperability.

In this paper, we introduce NMAstudio, an online interactive user interface for conducting, reviewing and managing network meta-analyses. Compared to existing applications,
NMAstudio integrates several innovative features such as transitivity assessment via box plots and scatter plots, analysis and visualizations for multiple outcomes, and interactive graphics that make the complexity of NMAs more accessible and transparent. Through novel visualizations, our tool enhances understanding and interpretation of findings in light of risk of bias and other potential limitations of the data, while promoting data sharing and open science. At the same time, it supports transparency by guiding researchers to go through all required steps for a complete report. NMAstudio also simplifies the full NMA process by offering a ‘point-and-click’ environment that completely removes the need for scripting. In the rest of the article, we first describe the main characteristics of the tool and we introduce our illustrative example dataset. Then, we go through the different steps of a typical NMA and show how NMAstudio can address existing challenges in the entire process. A brief description of the key terms used throughout the paper is available in **Box 1**.

Implementation

NMAstudio is a Python web application integrated into the Python “Dash” [20] environment and further connected to the R packages “netmeta” [21] for network meta-analysis and “meta” [22] for pairwise meta-analysis. NMA results are imported from R into Python, where interactive and downloadable visualizations are created using “Plotly” modules. Users of NMAstudio do not need any prior familiarity with these software packages. The application is open-source and accessible via a web browser at https://nmastudio.uiocloud.no/. The source code and data are available in the following GitHub repository (https://github.com/CER-METHODS/NMAstudio-2.0) under the MIT license. We also include instructions on how to self-host the application for users with data privacy concerns, although the process is not trivial. Currently, NMAstudio consists of three pages (see screenshots in **Supplementary file**): the homepage, the analysis setup page and the results page. In the analysis setup page, users upload their data and specify the variables that represent outcomes, risk of bias, and potential effect modifiers. Once these steps are completed, the analysis is performed using the R netmeta package [21], and additional customizable visualizations are generated automatically. The results page then displays all produced plots and tables in an interactive format. Users can explore data and results by interacting with all the outputs or can export them and prepare their NMA report.

Results

Illustrative example
of high-quality systematic reviews is the development of a protocol that outlines the main objectives, study design, and planned analyses of the review [24]. Registering the protocol is important for ensuring transparency, reducing reporting bias, avoiding data-driven decisions and preventing the duplication of reviews [25]. Therefore, NMAstudio asks users to provide the link to their protocol, which will then appear on the results page. This allows all end-users as well as peer-reviewers to easily access and compare the final review with the original protocol [26]. If information on study risk of bias, potential effect modifiers, or the protocol link is not provided, warnings appear in the results page (Figure S1). All settings for the statistical analysis can be found in the results page by clicking the 'Download settings of statistical analysis' button. Additional warnings related to the dataset may appear during the data checks, but these do not appear in the results page (see Figure S2 and the associated explanations in the Supplementary File).

## Exploring the structure of the data

A powerful feature of the application is the interactive network diagram. The function of the graph is to give an outline of the structure of the data included in the studies included. It also acts as a global filter and selector for the rest of the application. The graph is linked with an adjacent table with the selected studies. In the graph, each node represents an intervention, while each edge represents a direct comparison. Users can select interventions or comparisons by clicking on the corresponding nodes or edges. When a node is selected, the table highlights only the treatments that are directly compared with the selected intervention, helping users quickly identify comparisons supported by direct evidence from the available studies. Once nodes or edges are selected, the rest of the visualizations will display information specific to the corresponding interventions and comparisons.

Users can customize the network diagram to display distinct types of information using the provided options. **Figure 1** presents network graphs for the four outcomes of our example, each illustrating specific features. In **Figure 1A**, the network diagram for PASI90 uses edge size to represent the number of trials for each comparison. In **Figure 1B**, the network diagram for SAE demonstrates additional customization options. Users can also adjust the node size to reflect the number of trials for both interventions and comparisons and include labels on the edges to indicate the number of trials. Node colors can also be customized by specifying a color name or code (e.g., "pink" or "#FFC0CB"). Further, node colors can be modified to represent the risk of bias. For example, in the graph of DLQI (**Figure 1C**), each node is displayed as a pie chart, showing the proportion of trials with low (green), moderate (yellow), and high (red) risk of bias for the corresponding intervention. Additionally, if treatment class information is included in the data, the diagram can group interventions by color to indicate they belong to the same treatment class, as shown in **Figure 1D** for AE.
At the top of the results page, users can choose from a dropdown list to display different outputs. The first option, “Data & Transitivity”, presents data tables, box plots and scatter plots that are typically used in NMA to evaluate the *transitivity* assumption which is explained in detail in the **Assessing transitivity** section. Both raw data and data converted to the contrast-based format are shown prior to the analysis. Additionally, on the setup analysis page, if users upload time-related information such as publication year or study start and end dates, a timeline bar will be displayed on the results page, allowing users to visualize how the evidence has evolved over time.

## Assessing transitivity

Before conducting the analysis, it is crucial to assess the validity of the *transitivity* assumption. For an indirect comparison (e.g., A vs. B) to be considered valid, it is necessary for the sets of direct comparisons (e.g., A vs. C and B vs. C) to exhibit similar distributions of key clinical and methodological characteristics (i.e., effect modifiers) [27]. However, substantial empirical evidence suggests that the evaluation of this assumption is often inadequately reported, or not performed, in published NMAs [11, 12, 28].

While the *transitivity* assumption cannot be statistically tested, it can be assessed clinically and epidemiologically [2, 5, 29, 30]. During the data upload process, users are prompted to select potential effect modifiers from their dataset. Currently, effect modifiers must be in continuous format. For dichotomous variables, users should convert them into proportions (e.g., percentage of participants with a given characteristic). For categorical variables, users can assign numerical codes to represent categories (e.g., 1, 2, 3). At the results page, box plots and scatter plots are generated for each selected effect modifier in the dropdown list, allowing users to compare their distribution across the available direct comparisons. In scatter plots, point sizes are proportional to the sample size when that information is provided for each. For example, **Figure 2** shows for the psoriasis network box plots for age and percentage of male participants and scatter plots for BMI and mean weight. Users can highlight specific box plots or scatter plots by selecting the corresponding comparisons in the network diagram. Overall, the distributions of these characteristics appear relatively similar across the comparisons raising no concerns with respect to *transitivity* for this dataset. However, for some comparisons the number of available studies is very small (one or two studies only) and no distributions can be obtained.

## Visualizing and exploring results

*Forest plots for direct and network summary effects*
The psoriasis example includes 20 distinct treatments which can provide 20 different NMA forest plots, each with a different reference treatment. In the “Forest plots” tab of NMAstudio, users can click on any node in the diagram, such as “placebo” to set it as the reference treatment. This generates the corresponding ‘standard’ NMA forest plot (Figure 3) and a bi-dimensional forest plot showing simultaneously the relative effects for two outcomes with the same reference treatment (Figure S3). The standard forest plot shows all effect sizes along with confidence intervals, *prediction intervals* (optional), heterogeneity estimates for each comparison (optional) and the common heterogeneity estimate of the NMA, preventing from over-emphasizing highly uncertain findings. The bi-dimensional forest plot enables users to have a more complete view of the results and identify treatments that perform well on two outcomes simultaneously (e.g. efficacy and safety). Given that this graph can become busy and less visible for large networks, users may remove treatments by clicking the respective point in the legend of the graph. For example, in the psoriasis example, we kept the ten most effective treatments using placebo as reference; those that appear in the upper-right corner are those that perform best both for efficacy and safety (Figure S4).

Although summary effects involving both direct and indirect evidence are the main output from NMA, direct evidence and study-level effects also provide important information and need to be easily accessible. For example, high heterogeneity within a certain comparison or the presence of outlying studies may be the cause for increased heterogeneity or inconsistency in the entire network [30, 32]. The psoriasis example includes 58 comparisons with direct evidence, which is challenging to present clearly using traditional formats. NMAstudio addresses this problem by allowing users to click on any edge in the diagram, such as the one connecting “PBO” and “ETA” to generate the corresponding pairwise forest plot (Figure S5). This feature makes it easy and fast to navigate and review the pairwise forest plots for all or selected direct comparisons.

## League tables presenting all NMA relative effects

League tables typically display the summary relative effects along with the respective confidence intervals for every treatment contrast in the network. They often present the results of two outcomes, one in the upper triangle and the second in the lower triangle. In NMAstudio, two types of league tables are offered. The first is for a single outcome, with the upper triangle showing direct estimates and the lower triangle showing mixed (NMA) estimates. The second is for two outcomes, where both the upper and lower triangles display the mixed effects for each outcome. Users can enhance league tables by coloring the cells based on either the overall risk of bias of the respective contrast or the overall confidence rating. The latter is obtained using the CINEMA framework [34]; users need first to upload the final report as obtained from the CINEMA software (cinema.med.auth.gr). Users can also hover over the colored cells of the
table to see reasons for downgrading the evidence as these appear in the CINEMA report (Figure 4). This makes it straightforward to identify interventions and comparisons with higher and lower confidence in the evidence. The league table is also connected with the network diagram, allowing users to select specific interventions in the diagram and display the relevant comparisons in the table in the desired order (Figure S6).

## Ranking of the interventions

NMAstudio generates two ranking visualizations: a *P-score* heatmap for all the outcomes analyzed and a two-dimensional scatter plot displaying the *P-scores* for two selected outcomes (Figure S7-8). *P-scores* rank treatments based on all relative effects in the league table while also considering their precision, reflecting the average certainty that a treatment is better than its competitors [37]. Consideration of ranking results for different outcomes jointly is crucial for proper decision-making. In the two-dimensional scatter plot, the size of the points is proportional to the number of studies that include the corresponding interventions and the different colors represent different clusters of interventions with similar performance based on the *p*-score values. In our example, IFX demonstrates the highest score for the efficacy outcome (PASI90) but has a notably low *P-score* for the safety outcome (SAE). Also, ranking should always be considered along with the NMA relative effects and other characteristics, such as risk of bias or amount of evidence [38]. Examining the ranking alongside the network diagram, with risk of bias indicated by color and node size reflecting the number of trials for each intervention (Figure S9), reveals that IFX has substantial information coming from low risk of bias studies, while it is mainly compared with placebo and only one study connects it directly with ETA. The two-dimensional scatter plot suggests that BIME might be the best choice, as it is positioned in the top-right corner of the graph (Figure S8).

## Consistency and reporting bias

*Consistency* refers to the statistical agreement between direct and indirect evidence for a given comparison [2, 5]. There are two types of tests for inconsistency: local and global tests. In contrast to global tests, a local test can only be performed when a certain intervention comparison is included in one or more closed loops in the network [39]. If direct and indirect evidence are inconsistent, there are concerns with respect to the validity of the NMA results [2, 30]. NMAstudio offers two tests for inconsistency: the side-splitting approach that evaluates inconsistency for every comparison informed both directly and indirectly [40] and the design-by-treatment interaction model that provides a global test for the entire network [41]. For the outcome PASI90 of the psoriasis example, the global test indicates no significant inconsistency. However, the local test highlights potential issues for the IXE vs. USK comparison (p-
value=0.04), suggesting that combining direct and indirect effects for this comparison might be problematic and requires further investigation [2, 5]. The local consistency results table is also connected to the network diagram, allowing users to select specific comparisons displayed in the table (Figure S10). Here, the suspicious comparison is only informed by one study which might explain the difference between direct and indirect effects.

Reporting bias poses a significant threat to the validity of results in systematic reviews. Small-study effects are a common issue and refer to the systematic differences in the results between smaller and larger studies [42]. Small-study effects can be visually assessed using funnel plots, where asymmetry may indicate the presence of small-study effects and requires further investigation [43]. NMAstudio provides *comparison-adjusted funnel plots* to examine small-study effects across the entire network [44] and standard funnel plots for specific comparisons. For comparison-adjusted funnel plots, users should select one treatment in the network diagram to be used as reference for all comparisons. This preserves misleading interpretations that may arise when there is not a consistent direction for all comparisons in the graph. In the psoriasis example, we chose PBO as reference treatment; in this way the produced funnel plot assesses whether smaller studies tend to favor active interventions against PBO more than larger studies do. For the four outcomes of the psoriasis network, the funnel plots appear symmetrical raising no concerns with respect to the presence of small-study effects (Figure S11-14). For standard funnel plots, users can select a comparison that includes at least two studies in the network diagram to generate the plot. For example, Figure S15 presents the standard funnel plot for the comparison "PBO vs. ETA" for the PASI90 outcome, showing no visible asymmetry, which suggests no evidence of small-study effects for this comparison.

## Data sharing and open science

Transparency, openness, and reproducibility are essential features of science that should be standard practice [13]. In a typical NMA, authors present key findings in the main text while additional results, raw data, code, and other materials are placed in supplementary materials. However, supplementary materials often receive less attention due to their large volume, requiring substantial time for thorough exploration. The fixed format of supplementary materials also limits readers' flexibility in exploring results. These challenges hinder efficient sharing of data and results and prevent readers from deeply delving into the findings.

To promote an open research culture, NMAstudio provides easy functions for saving and loading uploaded data, allowing users to store and share their projects. On the results page, users can click the *Save/Load Project* button to generate a unique token for data sharing (Figure S16). This token can be shared by authors alongside their published article. Readers, peer-reviewers, or other stakeholders can then use the "Load Project" function in NMAstudio to enter the token, reload all results of the previously saved project to NMAstudio and explore the
results interactively (Figure S17). With its interactive and user-friendly features, NMAstudio gives access to original data and results, supporting independent verification and offering reviewers flexibility to explore findings and draw evidence-based conclusions. Of course, this feature is not intended to replace traditional supplementary materials entirely, but rather to complement them by offering an interactive and user-friendly way to explore data, results, and assumptions.

## Discussion

In this paper, we introduce NMAstudio, a web-application that aims to facilitate the production, visualization, and communication of NMAs. Its underlying motivation is to continuously address existing and new challenges that most investigators encounter when conducting or using NMAs. We illustrate how NMAstudio can simplify the entire analytical process and enhance interpretation and understanding of NMA findings using a Cochrane Review that involves data on multiple treatments and multiple outcomes [23]. NMAstudio's unique design combines a user-friendly 'point-and-click' environment with extensive interactivity, and a direct connection between the network diagram and all outputs. Other web-based applications for NMA are also available [45-48]. However, to our knowledge, NMAstudio is the only tool, to date, that allows analyzing multiple outcomes simultaneously, visualizing the evolution of evidence over time, evaluating *transitivity*, facilitating data sharing, and offering such a high level of interaction with data and results. Moreover, NMAstudio allows flexibility at the preparation of the data file with no restrictions on variable names or number of study arms. Users benefit from a wide range of interactive, high-quality plots that are unique to NMAstudio and not derived from other packages. These plots are generated in a single analysis run, making the process efficient, fast, and user-friendly.

A usual concern for tools that aim to simplify NMAs and statistical analyses, in general, is the risk for an increased number of poorly conducted studies and invalid results. NMAstudio involves steps during the data upload process that aim to mitigate, to some degree, this risk. First, users are asked to provide the link of their protocol to prevent data-driven decisions. Second, they need to upload risk of bias information for the included studies which then appears automatically in the league table. Third, users are asked to upload data on the potential effect modifiers which are used for the evaluation of *transitivity*. If any of these steps is skipped, warnings appear on the results page to inform end-users about the omission of these steps and the associated risks. Despite these steps, it is essential to emphasize, though, that involvement of statisticians and methodologists in NMA projects remain indispensable. One limitation of the current version of the tool is that it does not allow viewing the actual R functions executed during analysis. Additionally, software sustainability is an important consideration. It refers to
the capacity of a software tool to remain reliable, maintainable, and accessible over time,
independent of changes in technology or the duration of specific funding sources. NMAstudio
will address this by offering self-hosting options, which allow users to run the software
independently of external servers, and by ensuring that users retain ownership of their data.

Further updates are under development. NMAstudio currently offers only performing NMA in
the frequentist framework using the netmeta package in R [21] and the inverse-variance model.
Additional models available in netmeta, such as the Mantel-Haenszel or the penalized
likelihood models [49, 50], as well as Bayesian NMA will be implemented in upcoming
versions of the application. We are also working on allowing the conduct of sensitivity and
subgroup analyses based on pre-defined (during data uploading) study and intervention
characteristics. Comparison of results between primary and exploratory analyses will be
enhanced through the interactive features that NMAstudio offers. Additionally, NMAstudio
will allow for a full order of the treatments in the comparison-adjusted funnel plots. Finally, we
plan to allow users to upload NMA results obtained from external analyses and use NMAstudio
for visualization purposes. To further enhance the comprehensiveness and interpretation of
NMA findings for end-users, a knowledge translation tool is being developed for integration
into NMAstudio. This tool is designed to provide an interactive, flexible, and user-friendly
approach to summarizing and presenting NMA results within a single page. By featuring two
versions – a standard version and an advanced version – it aims to improve the accessibility
and usability of NMA findings for a broad range of stakeholders.

Conclusions

We believe NMAstudio serves as a valuable resource for researchers and clinical practitioners embarking on NMA. Its unique interactive features facilitate the communication of NMA outputs and enhance the clarity of results for diverse audiences. At the same time, it offers increased flexibility and interactivity, while prompting users to adhere to high standards for conducting and reporting NMAs. Future versions of NMAstudio together with the upcoming knowledge translation tool have the potential to further promote transparency, reproducibility, data sharing, open science, and broad research collaboration.

Declarations

Data availability

All data and code can be obtained via https://doi.org/10.5281/zenodo.18368497 or
https://github.com/CER-METHODS/NMAstudio-2.0

Funding
Tianqi Yu received funding from the French National Research Agency under the project ANR-22-CE36-0013-01.

**Competing interests**

The authors declare that they have no competing interests.

**Authors' contributions**

TY, SM, and AC conceptualized and designed the project. TY, SM, and TP wrote the code and developed the software. TY performed all analyses. TY and AC drafted the manuscript. All authors revised the manuscript and accepted the final version.

**Acknowledgments**

We sincerely thank Isabelle Boutron, Toshi Furukawa, Emily Karahalios, Tianjing Li, Michael McCaul, Adriani Nikolakopoulou, Haliton Oliveira, Georgia Salanti, Guido Schwarzer, Ian Saldanha, Nicky Welton, and Sally Yaacoub for their valuable suggestions and feedback on this project.

**References**

1. Nikolakopoulou A, Mavridis D, Salanti G, et al. Nikolakopoulou A, Mavridis D, Furukawa T A, Cipriani A, Tricco A C, Straus S E et al. BMJ. 2018;360:k585.

2. Cipriani A, Higgins JP, Geddes JR, et al. Conceptual and technical challenges in network meta-analysis. Ann Intern Med. 2013;159(2):130-137.

3. Dias S, Ades AE, Welton NJ, et al. Network meta-analysis for decision-making. Hoboken (NJ): John Wiley & Sons; 2018.

4. Dias S, Sutton AJ, Ades AE, et al. Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials. Med Decis Making. 2013;33(5):607-617.

5. Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods. 2012;3(2):80-97.

6. Petropoulou, M, Nikolakopoulou, A, Veroniki, A, et al. Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015. J Clin Epidemiol. 2017;82:20-28.

7. Veroniki AA, Florez I, Hutton B, et al. Two decades of network meta-analysis: Roadmap to their applications and challenges. Res Synth Methods. 2024;15(5):741-746.

8. Neupane B, Richer D, Bonner AJ, et al. Network meta-analysis using R: a review of currently available automated packages. PLoS One. 2014;9(12):e115065.
9. Shim S, Yoon BH, Shin IS, et al. Network meta-analysis: application and practice using Stata. Epidemiol Health. 2017;39:e2017047.

10. Hutton B, Catala-Lopez F, Moher D. The PRISMA statement extension for systematic reviews incorporating network meta-analysis: PRISMA-NMA. Med Clin (Barc). 2016;147(6):262-266.

11. Hutton B, Salanti G, Chaimani A, et al. The quality of reporting methods and results in network meta-analyses: an overview of reviews and suggestions for improvement. PLoS One. 2014;9(3):e92508.

12. Bafeta A, Trinquart L, Seror R, et al. Reporting of results from network meta-analyses: methodological systematic review. BMJ. 2014;348:g1741.

13. Foster ED, Deardorff A. Open science framework (OSF). J Med Libr Assoc. 2017;105(2):203.

14. Allen C, Mehler DM. Open science challenges, benefits and tips in early career and beyond. PLoS Biol. 2019;17(5):e3000246.

15. Chaimani A, Higgins JP, Mavridis D, et al. Graphical tools for network meta-analysis in STATA. PLoS One. 2013;8(10):e76654.

16. Salanti G, Ades AE, Ioannidis JP. Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial. J Clin Epidemiol. 2011;64(2):163-171.

17. Krahn U, Binder H, König J. A graphical tool for locating inconsistency in network meta-analyses. BMC Med Res Methodol. 2013;13:35.

18. Nevill CR, Cooper NJ, Sutton AJ. A multifaceted graphical display, including treatment ranking, was developed to aid interpretation of network meta-analysis. J Clin Epidemiol. 2023;157:83-91.

19. Seo M, Furukawa TA, Veroniki AA, et al. The Kilim plot: A tool for visualizing network meta-analysis results for multiple outcomes. Res Synth Methods. 2021;12(1):86-95.

20. Dabbas E. Interactive Dashboards and Data Apps with Plotly and Dash: Harness the power of a fully fledged frontend web framework in Python – no JavaScript required. Birmingham (UK): Packt Publishing Ltd; 2021.

21. Balduzzi S, Rücker G, Nikolakopoulou A, et al. netmeta: an R package for network meta-analysis using frequentist methods. J Stat Softw. 2023;106:1-40.

22. Schwarzer G. meta: An R package for meta-analysis. R News. 2007;7(3):40-45.

23. Sbidian E, Chaimani A, Guelimi R, et al. Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis. Cochrane Database Syst Rev. 2023;(7):CD011535.
24. Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1-9.

25. Stewart L, Moher D, Shekelle P. Why prospective registration of systematic reviews makes sense. Syst Rev. 2012;1:1-4.

26. Silagy CA, Middleton P, Hopewell S. Publishing protocols of systematic reviews: comparing what was done to what was planned. JAMA. 2002;287(21):2831-2834.

27. Song F, Altman DG, Glenny AM, Deeks JJ. Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses. BMJ. 2003;326(7387):472.

28. Bafeta A, Trinquart L, Seror R, et al. Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review. BMJ. 2013;347:f3675.

29. Jansen JP, Naci H. Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. BMC Med. 2013;11:159.

30. Donegan S, Williamson P, D'Alessandro U, Tudur Smith C. Assessing key assumptions of network meta-analysis: a review of methods. Res Synth Methods. 2013;4(4):291-323.

31. Salanti G, Del Giovane C, Chaimani A, et al. Evaluating the quality of evidence from a network meta-analysis. PLoS One. 2014;9(7):e99682.

32. Imrey PB. Limitations of meta-analyses of studies with high heterogeneity. JAMA Netw Open. 2020;3(1):e1919325.

33. Shim S, Yoon BH, Shin IS, et al. Network meta-analysis: application and practice using Stata. Epidemiol Health. 2017;39:e2017047.

34. Nikolakopoulou A, Higgins, JP, Papakonstantinou T, et al. CINEMA: an approach for assessing confidence in the results of a network meta-analysis. PLoS Med. 2020;17(4):e1003082.

35. Mbuagbaw L, Rochwerg B, Jaeschke R, et al. Approaches to interpreting and choosing the best treatments in network meta-analyses. Syst Rev. 2017;6:79.

36. Li T, Puhan MA, Vedula SS, et al. Network meta-analysis—highly attractive but more methodological research is needed. BMC Med. 2011;9:79.

37. Rücker G, Schwarzer G. Ranking treatments in frequentist network meta-analysis works without resampling methods. BMC Med Res Methodol. 2015;15:58.

38. Chaimani A, Porcher R, Sbidian É, et al. A Markov chain approach for ranking treatments in network meta-analysis. Stat Med. 2021;40(2):451-464.

39. Dias S, Welton NJ, Caldwell DM, et al. Checking consistency in mixed treatment comparison meta-analysis. Stat Med. 2010;29(7-8):932-944.
40. van Valkenhoef G, Dias S, Ades AE, et al. Automated generation of node-splitting models for assessment of inconsistency in network meta-analysis. Res Synth Methods. 2016;7(1):80-93.

41. Jackson D, Barrett, JK, Rice A, et al. design-by-treatment interaction model for network meta-analysis with random inconsistency effects. Stat Med. 2014;33(21):3639-3645.

42. Rücker G, Carpenter JR, Schwarzer G. Detecting and adjusting for small-study effects in meta-analysis. Biom J. 2011;53(2):351-368.

43. Copas J, Shi JQ. Meta-analysis, funnel plots and sensitivity analysis. Biostatistics. 2000;1(3):247-262.

44. Chaimani A, Salanti G. Using network meta-analysis to evaluate the existence of small-study effects in a network of interventions. Res Synth Methods. 2012;3(2):161-176.

45. Owen RK, Bradbury N, Xin Y, et al. MetaInsight: an interactive web-based tool for analyzing, interrogating, and visualizing network meta-analyses using R-shiny and netmeta. Res Synth Methods. 2019;10(4):569-581.

46. Brown S, Hutton B, Clifford T, et al. A Microsoft-Excel-based tool for running and critically appraising network meta-analyses—an overview and application of NetMetaXL. Syst Rev. 2014;3:110.

47. Papakonstantinou T, Nikolakopoulou A, Higgins JP, et al. CINEMA: software for semiautomated assessment of the confidence in the results of network meta-analysis. Campbell Syst Rev. 2020;16(1):e1080.

48. Chiocchia V, Nikolakopoulou A, Higgins JP, et al. ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis. BMC Med. 2021;19:1-13.

49. Efthimiou O, Rücker, G, Schwarzer G, et al. Network meta-analysis of rare events using the Mantel-Haenszel method. Stat Med. 2019;38(16):2992-3012.

50. Evrenoglou T, White IR, Afach S, et al. Network meta-analysis of rare events using penalized likelihood regression. Stat Med. 2022;41(26):5203-5219.
**PRISMA-NMA**

PRISMA-NMA is an extension of the PRISMA statement, created specifically to improve the reporting of systematic reviews that incorporate network meta-analyses (NMA). This extension includes a 32-item checklist, developed to address key elements considered essential for the accurate and thorough reporting of NMAs.

**Transitivity**

Transitivity is a key assumption in network meta-analysis that ensures the validity of indirect comparisons. It requires that studies comparing different interventions are similar, on average, with respect to all potential effect modifiers. It also implies that intervention A, as assessed in A vs. B studies, is equivalent to intervention A in A vs. C studies.

**Prediction interval**

A prediction interval is the interval within which the underlying effect of a new study similar to those included in a meta-analysis is expected to lie. Prediction intervals provide a way to assess the impact of the estimated heterogeneity and its uncertainty on the findings of the meta-analysis.

**CINeMA**

CINeMA (Confidence in Network Meta-Analysis) is a methodological framework for evaluating confidence or certainty of the results of network meta-analyses. A key element of CINeMA is the percentage contribution matrix, which indicates the proportion of information contributed by each study or direct comparison to each network estimate.

**P-score**

The P-score is a statistical measure used to rank treatments based on a specific outcome and shows the mean extent of certainty that a treatment is better than its competitors. It is calculated using the probabilities that one treatment is better than another, considering all direct and indirect comparisons within the network. These are determined based on the estimated relative effects and their variance.

**Consistency**

The consistency assumption refers to the agreement between different sources of evidence—direct and indirect—in a network meta-analysis. It reflects the assumption of transitivity, which considers potential clinical and methodological variations across comparisons. Consistency is assessed through statistical tests, such as the design-by-treatment interaction model and the side-splitting approach. The former assesses consistency in the entire network whereas the latter at the comparison level.

**Comparison-adjusted funnel plot**

A comparison-adjusted funnel plot is a variation of the standard funnel plot used in network meta-analysis to assess small-study effects while accounting for multiple comparisons. The horizontal axis shows the difference between study-specific effects and the corresponding comparison-specific summary effect, while the vertical axis represents the precision of the study, often measured by the inverse of its standard error. For proper interpretation, comparisons should be defined consistently (e.g., placebo vs. active treatment). A symmetric comparison-adjusted funnel plot around the zero line indicates no indication of small-study effects.

**Box 1. Explanation of key NMA terms**
## Figure legends

**Figure 1.** network graphs for different outcomes. The edge width is proportional to the number of studies for the corresponding comparison. (A) Outcome PASI90. (B) Outcome SAE: Numbers on the edges indicate the number of studies for the corresponding comparisons; node size is proportional to the number of studies that include the corresponding interventions. (C) Outcome AE: Pie charts within the nodes represent the percentage of studies with different risks of bias for the corresponding interventions (red = high, yellow = moderate, green = low). (D) Outcome DLQI: Node colors represent different treatment classes.

**Figure 2.** Boxplots of age and male percentage and scatter plots for BMI and weight for the PASI90 outcome. Seven comparisons (highlighted in blue) are selected by choosing the corresponding edges (shown in red) in the network diagram.

**Figure 3.** Forest plots for different outcomes with "PBO" as the reference treatment. (A) Outcome PASI90. (B) Outcome SAE. (C) Outcome AE. (D) Outcome DLQI. The black line represents the confidence interval and the red line represents the prediction interval. CI: Confidence Interval; PI: Prediction Interval; tau2: heterogeneity for each comparison in pairwise meta-analysis.

**Figure 4.** League table with color-coded CINEMA ratings. The upper triangle represents PASI90, and the lower triangle represents SAE. Hovering over the cell comparing MTX and BIME displays the reasons for downgrading.