# The Quality of Methodological and Reporting in Network Meta-Analysis of Acupuncture and Moxibustion: A Cross-Sectional Survey

Authors: Ting Yuan; Jun Xiong; Xue Wang; Jun Yang; Yunfeng Jiang; Xiaohong Zhou; Kai Liao; Lingling Xu
Journal: Evidence-Based Complementary and Alternative Medicine
 (2021)

---

## Abstract

Background. Acupuncture had long been a primary treatment in the healthcare system of China. In recent years, there were more and more network meta-analyses (NMAs) in the field of acupuncture and moxibustion, but the quality evaluation of NMAs was rare. Objectives. The goal of this study was to evaluate the methodological and reporting quality of NMAs and summarize the effects of different treatments of acupuncture and moxibustion. Methods. PubMed, Embase, the Cochrane Library, China National Knowledge Infrastructure Database (CNKI), WanFang Database (WF), Chinese Scientific Journal Database (VIP), and Chinese Biomedical Literature Database (CBM) were searched from inception to January 2020 without any language restriction. In addition, the unpublished studies and the references of initially included literature were also retrieved manually. We included all relevant NMAs treated with acupuncture and moxibustion; other therapies such as traditional Chinese medicine and Western medicine may also be included, but at least three types fall under the category of acupuncture in each NMA. Outcome indicators were not limited. We selected AMSTAR2 and PRISMA-NMA to evaluate the methodological and reporting quality of eligible studies, respectively. Results. In total, 29 NMAs were included finally, including 12 Chinese references and 17 English references. All eligible studies were published from May 2013 to August 2019. The number of interventions was between 4 and 22. The number of clinical trials included ranged from 10 to 121, with a total of 1098 clinical trials. The NMAs were involved in up to 23 diseases, knee osteoarthritis and primary dysmenorrhea covered with 3 NMAs separately, others focusing on chronic functional constipation, lumbar disc herniation, chronic fatigue syndrome, and the like. The Jadad scale and RoB scale were used as the bias risk assessment tools. Among them, 7 articles adopted the Jadad scale, 22 articles adopted the RoB scale (1 article adopted both the Jadad scale and RoB scale), and only 1 article did not mention the risk assessment tool. The AMSTAR2 methodological evaluation showed that the highest score was 13.5 points and the lowest was 4, with an average of 8.64 and a median of 9.5. According to the quality criteria, only one of them was in high quality, twenty-four were in medium quality, and four were in low quality. The PRISMA-NMA reporting quality evaluation showed that the highest score was 29 points and the lowest was 13.5, with an average of 23.62 and a median of 24.5; severe flaws also existed in some items, especially in “Structured summary,” “Protocol and registration,” “Search,” “Data collection process,” “Data items,” “Additional analyses,” “Risk of bias across studies,” and “Results of additional analyses.” Conclusion. The number of NMAs in the field of acupuncture and moxibustion was still in the initial stage. Overall, their methodology and reports were of moderate quality. However, severe flaws also existed in some items. Because the eligible NMAs were limited, the conclusion needed further research to confirm its authenticity and reliability.


---

## Body

## 1. Introduction

Acupuncture has a history of more than 4,000 years and is now commonly used in medical care in China. As a traditional oriental therapy, it has been widely used globally [1]. With the development of evidence-based acupuncture and moxibustion, acupuncture and moxibustion is more and more popular in clinics because of its simple treatment, quick effect, and nontoxic side effects. However, there are many kinds of acupuncture and moxibustion intervention methods, and the indications are similar. Traditional meta-analysis can only achieve pairwise direct comparison of intervention measures, but it cannot carry out indirect comparison of intervention measures without a direct comparative study, and let alone comparative analysis of various intervention measures; so, it is difficult to solve this problem.Network meta-analyses (NMAs) are still named multiple-treatments meta-analysis (MTM) or mixed treatment comparison meta-analyses (MTC). Meta-analysis is an analytical method that evaluates the relative efficiency of treatments and synthesizes evidence using a randomized controlled trial network diagram. This method is based on the combination of traditional direct/head-to-head comparison and indirect comparison for meta-analysis, which can simultaneously compare the efficacy of three or more interventions. It was developed on the basis of classical meta-analysis, which resulted in a consistent and complete set of comparisons based on all available evidence from existing research studies [2–5]. The NMA provides evidence for clinical drug selection by quantifying different interventions to treat the same disease and ranking the benefits [6]. As an emerging evidence synthesis tool, NMAs are becoming more and more popular, which can make more decisions or choices than classic meta-analysis [7–10].In recent years, NMA has made some progress in the field of acupuncture and moxibustion, and the number of publication of NMAs related to acupuncture and moxibustion is on the rise, but its quantity is still small, and its quality still lacks systematic evaluation. This study through retrieving NMAs of acupuncture and moxibustion published a comprehensive understanding of the present situation, and apply AMSTAR2 (a measurement tool to assess the methodological quality of systematic reviews) and PRISMA-NMA (PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations) to assess the methodological and reporting quality of the published NMAs in acupuncture and moxibustion field systematically, in order to offer reference to improve the quality of acupuncture and moxibustion in NMA.


## 2. Materials and Methods

### 2.1. Information Sources

PubMed, Embase, the Cochrane Library, China National Knowledge Infrastructure Database (CNKI), WanFang Database (WF), Chinese Scientific Journal Database (VIP), and Chinese Biomedical Literature Database (CBM) were searched from inception to January 2020 without any language restriction. In addition, the unpublished studies and the references of initially included literature were also retrieved manually. The comprehensive search strategy for PubMed is presented in Table1. The retrieval of other electronic databases was similar to PubMed, which adopts the combination of subject words and keywords.Table 1
Search strategy.
Source: PubMed; searched on: January 22, 2020SearchQuery#1“acupuncture” [Ti/Ab] OR “moxibustion” [Ti/Ab]#2“network meta-analysis” [Ti/Ab] OR “mixed treatment meta-analysis” [Ti/Ab] OR “multiple treatment comparison meta-analysis” [Ti/Ab] OR “bayes meta-analysis” [Ti/Ab] OR “indirect comparison” [Ti/Ab]#3“network meta-analysis” [MeSH]#4“acupuncture” [MeSH] OR “moxibustion” [MeSH]#5#1 AND #2#6#3 AND #4#7#5 OR #6


### 2.2. Eligible Criteria

We included all relevant NMAs treated with acupuncture and moxibustion; other therapies such as traditional Chinese medicine and Western medicine may also be included, but at least three types fall under the category of acupuncture in each NMA. Outcome indicators were not limited, while language limited in Chinese and English.


### 2.3. Exclusion Criteria

(1) Duplicate detection and republished literature. (2) Theoretical research. (3) Reviews, conference papers, and abstracts. (4) Incomplete data of the results. (5) Acupuncture interventions <3. (6) Non-Chinese and English literature.


### 2.4. Study Selection and Data Extraction

The search was conducted by NoteExpress 5.3.0 literature management software. NoteExpress 5.3.0 automatic duplicate check function was used and combined with manual duplicate check to eliminate the duplicate research. By reading the title and abstract, we excluded studies that obviously do not meet the inclusion criteria; downloaded and read the full text of the remaining studies to judge; and for research with incomplete data report, tried to contact the author and gain the complete data. The three evaluators (TY, XW, and JY) screened and extracted the literatures back to back independently according to the inclusion and exclusion criteria, and cross-checked the results. The included details were as follows: author information, year of publication, sample size, disease, type of study, diagnostic criteria, number of original study, description of interventions, number of interventions, comparators, outcome measures, and risk assessment tools for bias and adverse reactions. When there was any disagreement, it was resolved by the fourth researcher (JX).


### 2.5. Methodological and Reporting Quality Assessment Tools

Three independent researchers (TY, XW, and JY) evaluated the methodological and reporting quality back to back. The AMSTAR2 tool contained 16 aspects [11]. For each aspect, when the answer was “yes,” the score was 1, and when the answer was “no,” “cannot answer,” or “not applicable,” the score was 0. The total score of an NMA was calculated by counting the number of “yes” in 16 items on a scale of 0–16. A score of 12–16 was rated as “high quality,” a score of 7–11 was rated as “medium quality,” and a score of 0–6 was rated as “low quality.” The PRISMA-NMA contained 32 items [12]. For each item, a score of “1” means full compliance, “0.5” means partial compliance, and “0” means noncompliance [13, 14]. The total PRISMA-NMA score of an NMA was calculated by accumulating the scores of each item, with a range of 0–32. A score of 26–32 was rated as “high quality,” a score of 20–25.5 as rated as “medium quality,” and a score of 0–19.5 as rated as “low quality.” When there was any disagreement, it was resolved by the fourth researcher (JX).


### 2.6. Data Analysis

We analyzed the characteristics of included studies through descriptive statistical methods. All the data used were counted in the Excel 2007 spreadsheet. We described the dichotomous data in terms of number and percentage and the continuous variables in terms of median with interquartile range (IQR). And we calculated the number of papers per item, its percentage, and 95% confidence intervals. We summarized the scores according to the quality evaluation tool. AMSTAR2 and PRISMA-NMA scored 16 and 32, respectively. Finally, we calculated the total score through adding a list of each component.


## 2.1. Information Sources

PubMed, Embase, the Cochrane Library, China National Knowledge Infrastructure Database (CNKI), WanFang Database (WF), Chinese Scientific Journal Database (VIP), and Chinese Biomedical Literature Database (CBM) were searched from inception to January 2020 without any language restriction. In addition, the unpublished studies and the references of initially included literature were also retrieved manually. The comprehensive search strategy for PubMed is presented in Table1. The retrieval of other electronic databases was similar to PubMed, which adopts the combination of subject words and keywords.Table 1
Search strategy.
Source: PubMed; searched on: January 22, 2020SearchQuery#1“acupuncture” [Ti/Ab] OR “moxibustion” [Ti/Ab]#2“network meta-analysis” [Ti/Ab] OR “mixed treatment meta-analysis” [Ti/Ab] OR “multiple treatment comparison meta-analysis” [Ti/Ab] OR “bayes meta-analysis” [Ti/Ab] OR “indirect comparison” [Ti/Ab]#3“network meta-analysis” [MeSH]#4“acupuncture” [MeSH] OR “moxibustion” [MeSH]#5#1 AND #2#6#3 AND #4#7#5 OR #6


## 2.2. Eligible Criteria

We included all relevant NMAs treated with acupuncture and moxibustion; other therapies such as traditional Chinese medicine and Western medicine may also be included, but at least three types fall under the category of acupuncture in each NMA. Outcome indicators were not limited, while language limited in Chinese and English.


## 2.3. Exclusion Criteria

(1) Duplicate detection and republished literature. (2) Theoretical research. (3) Reviews, conference papers, and abstracts. (4) Incomplete data of the results. (5) Acupuncture interventions <3. (6) Non-Chinese and English literature.


## 2.4. Study Selection and Data Extraction

The search was conducted by NoteExpress 5.3.0 literature management software. NoteExpress 5.3.0 automatic duplicate check function was used and combined with manual duplicate check to eliminate the duplicate research. By reading the title and abstract, we excluded studies that obviously do not meet the inclusion criteria; downloaded and read the full text of the remaining studies to judge; and for research with incomplete data report, tried to contact the author and gain the complete data. The three evaluators (TY, XW, and JY) screened and extracted the literatures back to back independently according to the inclusion and exclusion criteria, and cross-checked the results. The included details were as follows: author information, year of publication, sample size, disease, type of study, diagnostic criteria, number of original study, description of interventions, number of interventions, comparators, outcome measures, and risk assessment tools for bias and adverse reactions. When there was any disagreement, it was resolved by the fourth researcher (JX).


## 2.5. Methodological and Reporting Quality Assessment Tools

Three independent researchers (TY, XW, and JY) evaluated the methodological and reporting quality back to back. The AMSTAR2 tool contained 16 aspects [11]. For each aspect, when the answer was “yes,” the score was 1, and when the answer was “no,” “cannot answer,” or “not applicable,” the score was 0. The total score of an NMA was calculated by counting the number of “yes” in 16 items on a scale of 0–16. A score of 12–16 was rated as “high quality,” a score of 7–11 was rated as “medium quality,” and a score of 0–6 was rated as “low quality.” The PRISMA-NMA contained 32 items [12]. For each item, a score of “1” means full compliance, “0.5” means partial compliance, and “0” means noncompliance [13, 14]. The total PRISMA-NMA score of an NMA was calculated by accumulating the scores of each item, with a range of 0–32. A score of 26–32 was rated as “high quality,” a score of 20–25.5 as rated as “medium quality,” and a score of 0–19.5 as rated as “low quality.” When there was any disagreement, it was resolved by the fourth researcher (JX).


## 2.6. Data Analysis

We analyzed the characteristics of included studies through descriptive statistical methods. All the data used were counted in the Excel 2007 spreadsheet. We described the dichotomous data in terms of number and percentage and the continuous variables in terms of median with interquartile range (IQR). And we calculated the number of papers per item, its percentage, and 95% confidence intervals. We summarized the scores according to the quality evaluation tool. AMSTAR2 and PRISMA-NMA scored 16 and 32, respectively. Finally, we calculated the total score through adding a list of each component.


## 3. Results

### 3.1. Search Results

242 related references through searching seven electronic databases and manual searches were retrieved. After reading the title and abstract, a total of 29 studies were included [15–43]. The literature screening process is shown in Figure 1.Figure 1
Flow diagram of the study.


### 3.2. Study Characteristics

The characteristics of all included NMAs were documented. After screening, 29 NMAs finally met the inclusion criteria, including 12 Chinese references (including 1 master’s thesis) and 17 English references. All eligible studies were published from May 2013 to August 2019. The study contained 4–22 types of treatments and 10–121 RCTs for 1098 RCTs in total. The NMAs were involved in up to 23 diseases, knee osteoarthritis and primary dysmenorrhea covered with 3 NMAs separately, others focusing on chronic functional constipation, lumbar disc herniation, chronic fatigue syndrome, and the like. The Jadad scale and RoB scale were used as the bias risk assessment tools. Among them, 7 articles adopted the Jadad scale, 22 articles adopted the RoB scale (1 article adopted both the Jadad scale and RoB scale), and only 1 article did not mention the risk assessment tool. The characteristics of the eligible studies are presented in Table2.Table 2
Basic characteristics of eligible studies.
Study IDDiseaseSample sizeNumber of interventionsDescription of interventionsNumber of RCTOutcomeRisk assessment toolFu et al. [15]Acute gouty arthritis493110Western medicine, acupuncture + pricking blood and cupping, Chinese medicine + cupping, acupuncture and moxibustion, Chinese medicine, Western medicine + pricking blood and cupping, pricking blood and cupping, Western medicine + acupuncture, Western medicine + massage, and Western medicine + Chinese medicine66Effective rateJadadDing [16]Abdominal distension after abdominal operation20477Acupoint application, acupoint injection, acupoint massage, moxibustion, TCM enema, TCM hot ironing, and routine nursing22Effective rateJadadLi [17]Poststroke depression189513Acupuncture + Chinese medicine, ear bean + Chinese medicine, electric acupuncture + Chinese medicine, moxibustion + Chinese medicine, auricular acupoint electroacupuncture + Chinese medicine, head acupuncture + acupoint embedding, auricular acupoint electroacupuncture + acupuncture, ear bean + acupuncture, acupuncture + acupoint injection, acupuncture + wuxing music, wuxing music + Chinese medicine, SSRIs, and acupuncture + moxibustion23Effective rateJadadBu et al. [18]Optic atrophy136911Western medicine, acupuncture, acupuncture + moxibustion, ear point + Chinese medicine, electroacupuncture, acupuncture + Chinese and Western medicine, moxibustion + Chinese and Western medicine, electroacupuncture + Chinese and Western medicine, Chinese and Western medicine, Chinese medicine, and acupuncture + Western medicine16Effective rate, vision, horizon, and visual evoked potentialRoBZhang [19]Ankylosing spondylitis22089Sulfasalazine, acupuncture, moxibustion, bee acupuncture, acupuncture + moxibustion, acupuncture + moxibustion + cupping, and moxibustion + bee needle25Effective rateRoBSong [20]Polycystic ovary syndrome460514Acupuncture-medication therapy, Western medicine, acupuncture and moxibustion, acupuncture, acupuncture + ear points, moxibustion + Chinese medicine, acupuncture + ear points, acupuncture + placebo, placebo + Western medicine, Chinese medicine + Western medicine, placebo, acupoint thread-embedding therapy and medication, Chinese medicine, and moxibustion39Ovulation rate and pregnancy rateRoBYang et al. [21]Lumbar disc herniation25894Acupuncture, acupuncture + cupping, acupuncture + massage, and acupuncture + cupping + massage30Pain improvement, effectiveness, cure rate, recurrence rate, and economic indicators.RoBLi [22]Primary dysmenorrhea46006Acupuncture, acupuncture + moxibustion, acupuncture + indirect moxibustion, warm acupuncture, electroacupuncture, and electroacupuncture + warm acupuncture56Effective rate, VAS, and dysmenorrhea symptom scoreRoBJia et al. [23]Chronic urticaria11869Acupoint catgut embedding, acupoint catgut embedding + loratadine, acupoint catgut embedding + mizolastine, acupoint catgut embedding + cetirizine, acupoint catgut embedding + epstein, loratadine, mizolastine, cetirizine, and epstein15Effective rate and adverse effectRoBLiu [24]Knee osteoarthritis34176Hyaluronic acid (HA), needle knife, needle knife + HA, needle knife + internal medicine, needle knife + massage, and needle knife + acupuncture29Effective rate, VAS, and Lysholm function scoresJadadFeng [25]Primary dysmenorrhea102599Multipoint + conventional Chinese medicine therapy, multipoint + Western medicine conventional therapy, single point + TCM, TCM, single point, multipoint, Western medicine routine therapy, and placebo103Effective rate, VAS, CMSS, and SF-MPQRoBLiu [26]Poststroke shoulder pain18989Rehabilitation, acupuncture, acupuncture + rehabilitation, acupuncture + massage, acupuncture + TCM fumigation + rehabilitation, acupuncture + massage + cupping, acupuncture + massage + rehabilitation, acupuncture + TCM + rehabilitation, and acupuncture + joint loosening + rehabilitation25Effective rate, FMA, and MBIRoBYang et al. [27]Premature ovarian insufficiency304618HRT, acupuncture, moxibustion, needle-warming moxibustion, electroacupuncture, catgut implantation at acupoint , TCM, acupuncture + TCM, moxibustion + TCM, electroacupuncture + TCM, acupoint application + TCM, auricular point sticking + TCM, acupressure + TCM, catgut implantation at acupoint + TCM, acupuncture + HRT, electroacupuncture + HRT, catgut implantation at acupoint + HRT, and acupuncture + TCM + HRT43Effective rate and adverse effectRoBZhu et al. [28]Chronic constipation1103210Acupuncture, polyethylene glycol, lactulose, linaclotide, lubiprostone, bisacodyl, prucalopride, sham acupuncture, tegaserod, and placebo40Symptoms of chronic constipation and side effectsJadadZheng [29]Chronic functional constipation43248Acupuncture, mosapride, insoluble fiber, massage, mineral water, probiotic, TENS, and moxibustion33Weekly stool frequency, Bristol score, responder rate, and adverse eventRoBQin [30]Chronic prostatitis/chronic pelvic pain syndrome12037Acupuncture, electroacupuncture, alpha-blockers, antibiotics, dual therapy, sham acupuncture, and placebo12NIH-CPSI QoL scoreRoBLi et al. [31]Knee osteoarthritis20657Common manual acupuncture, electroacupuncture, fire needle, warm needle, placebo, sham needle, and education16WOMAC, stiffness, and physical function scoresRoBMo et al. [32]Lumbar disc herniation130754Tuina, traction, acupuncture, and Chinese herbs121Invalid rate, cure rate, VAS, and JOARoBLuo et al. [33]Primary dysmenorrhea15118Traditional acupuncture, eye acupuncture, wrist-ankle acupuncture, superficial acupuncture, moxibustion, electroacupuncture, ear acupuncture, and abdominal acupuncture17Effective rateRoBYeh et al. [34]Psoriasis8696Acupuncture, acupressure, acupoint bloodletting, acupoint catgut embedding, Chinese herbal medicine, and narrow-band ultraviolet B10PASI and TCMJadad RoBChen et al. [35]Migraine36569Acupuncture, flunarizine, metoprolol, propranolol, propranolol + flunarizine, sham acupuncture, topiramate, usual care, and waiting list19Migraine episodes, the number of migraine days, migraine frequency, responder rate, and adverse event rateRoBTan et al. [36]Essential hypertension264915Electroacupuncture, moxibustion, warm needle therapy, sham acupuncture, behavioral therapy, angiotensin-converting enzyme inhibitors (ACEIs), angiotensin receptor blockers (ARBs), calcium channel blocker (CCB), beta-blocker, acupuncture-combined ACEI, acupuncture-combined CCB, acupuncture-combined behavior, electroacupuncture-combined CCB, sham acupuncture, and nontreatment31posttreatment BP changes, response rate, and MACERoBLi et al. [37]Myofascial pain syndrome169222Placebo-sham, MA, EA, DN, MET, TCT, MT, LTrP-I, MDIMST, MSN, TTM, BTX-A-TrP-I, FN, SWAM, EA&ESNC, SPM, DN&MET, BTrP-I, TrP-DN&EDU, Stretch, DN&Stretch, laser, and PT33Pain measurement (VAS, NRS, and PPT), adverse events (ROM), and functional statusRoBZhu et al. [38]Diarrhea-predominant irritable bowel syndrome93697Acupuncture, sham acupuncture, pinaverium bromide, alosetron, eluxadoline, ramosetron, and rifaximin29Effective rate, side effects (constipation and rash), and common acupuncture points.JadadXiong and Chen [39]Diabetic peripheral neuropathy26027Manual acupuncture, electroacupuncture, needle knocking acupuncture, warm needling and moxibustion, mecobalamin, no interventions, and vitamin B40global symptom improvementnonYang et al. [40]Heart failure21165Acupuncture, moxibustion, acupoint application, acupoint injection, and warming acupuncture-moxibustion26HFC and LVEFRoBZhang et al. [41]Obesity22836AAS, EA, ACE, WA, AR: acupuncture and related therapies and combination of acupuncture and related therapies.34BW, BMI, and adverse eventsRoBCorbett 2013 [42]Knee osteoarthritis97098Interferential therapy, acupuncture, TENS, pulsed electrical stimulation, balneotherapy, aerobic exercise, sham acupuncture, and muscle-strengthening exercise114WOMAC painRoBWang et al. [43]Chronic fatigue syndrome22555CbAM, SAM, Chinese herbal medicine, Western medicine, and sham acupuncture31Effective rateRoBNote: CMSS, the Cox Menstrual Symptom Scale; FMA, Fugl－Meyer motor assessment scale; MBI, modified Barthel index; TCM, traditional Chinese medicine; MACE, major adverse cardiovascular events; HFC, changes in heart function classification; LVEF, left ventricular ejection fraction; AAS, auricular acupoint stimulation; WA, warming acupuncture; ACE, acupoint catgut embedding; EA, electroacupuncture; AR, acupuncture and related therapies.


### 3.3. Methodological Quality Assessment

According to the AMSTAR2 checklist, the median score and IQR of eligible NMAs was 9.5 (6.5–10.75), and the details are presented in Tables3 and 4. The item with the best degree of compliance was “comprehensive literature search” (100%) (Table 4). Longitudinal analysis, a good degree of compliance was with item 9 (93.1%), item 11 (79.31%), and items 1, 5, and 6 (75.86%). However, a poor degree of compliance was with item 2 (17.24%) and item 3 (6.9%), and the worst degree of compliance was with item 7 (0%) and item 10 (0%).The details are presented in Tables 3 and 4 and Figure 2.Table 3
Methodological quality assessment of the included NMAs.
Item12345678910111213141516ScoreFu et al. [15]YNNPYYNPPNNNNNYN5.5Ding [16]YNNPYNNYYNNNNNNP5Li [17]YNNPYNNPYNYNPNNN5.5Bu et al. [18]YNNYYYNPYNPNPNNP7Zhang [19]YNNYNYNPYNYNPYNP6Song [20]YNNYYYNYYNPPPYYP8Yang et al. [21]YNNPNYNYYNYNYYNP8Li [22]YNNYYYNYYNYYYYPP11Jia et al. [23]YNNPYNNYYNYYYYPP9.5Liu [24]YNNYYYNYYNYYNYNP9.5Feng [25]YNNYPYNYYNYYYYPN10Liu [26]YNNPNNNYYNYNPNNP5.5Yang et al. [27]YNNYYYNYYNYYYYYP11Zhu et al. [28]NNNYYYNNYNYNNNYP6.5Zheng [29]PYYYYYNPYNYNYYPY11Qin [30]YNNYYYNYYNYYYYNY11Li et al. [31]YNNPYYNPYNYYYYYY11Mo et al. [32]PNNYYYNPYNPNNNYY7.5Luo et al. [33]YNNPNNNPYNYYNNYP6.5Yeh et al. [34]PNNPYYNPYNYYYYYP10Chen et al. [35]PYNYYNNPYNYNNPNY7.5Tan et al. [36]YNNPYYNYYNYYYNYY10.5Li et al. [37]YYNYYYNYYNYYYNNP10.5Zhu et al. [38]PNNYYYNPYNYYYPYP10Xiong and Chen [39]PNNPNNNPNNNNNNNY2.5Yang et al. [40]YYNPYYNPYNYYYNYY11Zhang et al. [41]YNNPYYNYYNYYYNNY9.5Corbett 2013 [42]YYYYYYNPYNYYYYYY13.5Wang et al. [43]YNNPNYNYYNYYYYYY10.5Score20521522220142702316161413118.64/9.5Y, yes (1 point); N, no (0 point); P, partial satisfaction (0.5 point). Item 1, did the research questions and inclusion criteria for the review include the components of PICO? Item 2, did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol? Item 3, did the review authors explain their selection of the study designs for inclusion in the review? Item 4, did the review authors use a comprehensive literature search strategy? Item 5, did the review authors perform study selection in duplicate? Item 6, did the review authors perform data extraction in duplicate? Item 7, did the review authors provide a list of excluded studies and justify the exclusions? Item 8, did the review authors describe the included studies in adequate detail? Item 9, did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review? Item 10, did the review authors report on the sources of funding for the studies included in the review? Item 11, if meta-analysis was performed, did the review authors use appropriate methods for statistical combination of results? Item 12, if meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis? Item 13, did the review authors account for RoB in individual studies when interpreting/ discussing the results of the review? Item 14, did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review? Item 15, if they performed quantitative synthesis, did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review? Item 16, did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?Table 4
Summary of methodological quality assessment.
NumberItemCompletely reportedPartially reportedNot reportedFrequency (%)95% CIFrequency (%)95% CIFrequency (%)95% CI1Components of PICO question?22 (75.86%)(0.58, 0.94)6 (20.69%)(−0.12, 0.53)1 (3.45%)(−0.32, 0.39)2Review protocol?5 (17.24%)(−0.16, 0.50)024 (82.75%)(0.68, 0.98)3Explanation of study design?2 (6.90%)(−0.28, 0.42)027 (93.10%)(0.84, 1.03)4Comprehensive literature search strategy16 (55.17%)(0.31, 0.80)13 (44.83%)(0.18, 0.72)05Study selection in duplicate?22 (75.86%)(0.58, 0.94)1 (3.45%)(−0.32, 0.39)6 (20.69%)(−0.12, 0.53)6Data extraction in duplicate?22 (75.86%)(0.58, 0.94)07 (24.14%)(−0.08, 0.56)7List of excluded studies and justify the exclusions?0029 (100%)8Study characteristics14 (48.27%)(0.22, 0.74)14 (48.27%)(0.22, 0.74)1 (3.45%)(−0.32, 0.39)9Satisfactory technique for assessing risk of bias?27 (93.10%)(0.84, 1.03)1 (3.45%)(−0.32, 0.39)1 (3.45%)(−0.32, 0.39)10Sources of funding for each study?0029 (100%)11Appropriate methods?23 (79.31%)(0.63, 0.96)3 (10.34%)(−0.24, 0.45)3 (10.34%)(−0.24, 0.45)12Assess potential impact of risk of bias on the results?16 (55.17%)(0.31, 0.80)1 (3.45%)(−0.32, 0.39)12 (41.38%)(0.14, 0.69)13Account for risk of bias when interpreting/discussing?16 (55.17%)(0.31, 0.80)5 (17.24%)(−0.16, 0.50)8 (27.59%)(−0.03, 0.59)14Satisfactory explanation for and discussion of any heterogeneity?14 (48.27%)(0.29, 0.81)2 (6.90%)(−0.28, 0.42)13 (44.83%)(0.18, 0.72)15Publication bias (small sample bias) assessed and discussed?13 (44.83%)(0.18, 0.72)4 (13.79%)(−0.20, 0.48)12 (41.38%)(0.14, 0.69)16Publication bias (small sample bias) assessed and discussed?11 (37.93%)(0.09, 0.67)15 (51.72%)(0.26, 0.77)3 (10.34%)(−0.24, 0.45)Figure 2
AMSTAR2 percentage distribution diagram (%).


### 3.4. Reporting Quality Assessment

According to the PRISMA-NMA checklist, the median score and IQR of eligible NMAs was 24.5 (20.5–26.5).The item with the best degree of compliance was item 1 “Title” (100%). Longitudinal analysis, a good degree of compliance was with item 4 (96.55%), item 12 (96.55%), item 25 (96.55%), item 18 (93.10%), item 26 (93.10%), item 3 (89.65%), item 19 (89.65%), and item 24 (86.21%). However, a poor degree of compliance was with item 23 (27.59%), item 2 (17.24%), item 5 (17.24%), item 10 (17.24%), and item 11 (10.34%), and the worst degree of compliance was with item S4 (3.45%). There are significant problems in the reporting methods and results sections, such as underreporting or selective reporting. Three studies [29, 39, 42] (10.34%) reported evaluating the risk of bias within individual studies in the methods section (item 12), but did not really evaluate it in the results section (item 19). Horizontal analysis, there were four NMAs (13.79%) that scored less than 20 points, with a minimum score of 13.5. The details are presented in Tables 5 and 6.Table 5
Reporting quality assessment of the included NMAs.
ItemSection/topicFu 2019Ding 2019Li 2018Bu 2017Zhang 2018Song 2019Yang 2016Li 2017Jia 2018Liu 2016Feng 2018Liu 2019Yang 2017Zhu 2018Zheng 20181Title1111111111111112Structured summary0.50.50.50.50.50.50.50.50.50.50.50.50.50.513Rationale10.510.51111110.511114Objectives1110.5111111111115Protocol and registration0000000000000016Eligibility criteria110.50.50.51111111110.57Information sources1111110.5111111118Search0000000101101019Study selection0.51110101111011110Data collection process0.50.50.510.50.50.510.50.510.50.50.50.511Data items0.50.510.50.50.50.50.50.50.50.50.50.50.50.5S1Geometry of the network10101101111111112Risk of bias within individual studies11111111111111113Summary measures10.510.50.50.50.51111111114Planned methods of analysis101101111111111S2Assessment of inconsistency10111111101111115Risk of bias across studies10010101111011116Additional analyses00000100111001117Study selection110.510.50.5111111111S3Presentation of network structure111111011111111S4Summary of network geometry10.50.50.50.50.500.50.50.50.50.50.500.518Study characteristics11111111111111119Risk of bias within studies11111111111111020Results of individual studies00000001101011021Synthesis of results111110.5111111111S5Exploration for inconsistency10011111111100022Risk of bias across studies10000101101001023Results of additional analyses00001000010001024Summary of evidence10100.5111111111125Limitations11111111111111126Conclusions110.5110.511111111127Funding010111110101101Summary24172020.5202418.527.5262627222525.525ItemQin 2016Li 2018Mo 2019Luo 2018Yeh 2016Chen 2019Tan 2019Li 2017Zhu 2018Xiong 2018Yang 2019Zhang 2018S. 2013Wang 201711111111111111120.50.50.50.50.510.510.50.510.510.53111111111111114111111111111115000001010010106110.510.50.5110.50.51110.5710.5110.511110.50.511181010001110110091110.511111011101010.50.50.50.50.50.50.50.500.510.50.51110.50.50.50.50.510.50.50.50.50.50.50.5S11100000110000012111111111011111311110.50.50.510.50110.50.5141101110110.51110S2111001111110101511111111101011161100000110111117111110.511111111S311111111111111S40.50.50.50.50.50.50.50.50.50.50.50.50.50.5181111111110.5110.51191111111110110.512011111111101101211111111111110.51S511101111111100220111011010100023110000001000112411111111101111251111111110111126111111111111112711100111011111Score2927.524.521.520.5252528.52713.52824.523.51924.5/23.62Table 6
Summary of reporting quality assessment.
SectionItemCompletely reportedPartially reportedNot reportedFrequency (%)95% CIFrequency (%)95% CIFrequency (%)95% CITitle1Title29 (100%)00Abstract2Structured summary5 (17.24%)(−0.16, 0.50)24 (82.76%)(0.68, 0.98)0Introduction3Rationale26 (89.65%)(0.78, 1.01)3 (10.34%)(−0.24, 0.45)04Objectives28 (96.55%)(0.90, 1.03)1 (3.45%)(−0.32, 0.39)0Methods5Protocol and registration5 (17.24%)(−0.16, 0.50)024 (82.76%)(0.68, 0.98)6Eligibility criteria19 (65.52%)(0.44, 0.87)10 (34.48%)(0.05, 0.64)07Information sources22 (75.86%)(0.58, 0.94)5 (17.24%)(−0.16, 0.50)08Search12 (41.38%)(0.14, 0.69)017 (58.62%)(0.35, 0.82)9Study selection22 (75.86%)(0.58, 0.94)2 (6.70%)(−0.28, 0.42)5 (17.24%)(−0.16, 0.50)10Data collection process5 (17.24%)(−0.16, 0.50)23 (79.31%)(0.63, 0.96)1 (3.45%)(−0.32, 0.39)11Data items3 (10.34%)(−0.24, 0.45)26 (89.65%)(0.78, 1.01)0S1Geometry of the network16 (55.17%)(0.31, 0.80)013 (44.83%)(0.18, 0.72)12Risk of bias within individual studies28 (96.55%)(0.90, 1.03)01 (3.45%)(−0.32, 0.39)13Summary measures18 (62.07%)(0.40, 0.85)11 (37.93%)(0.09, 0.67)014Planned methods of analysis23 (79.31%)(0.63, 0.96)1 (3.45%)(−0.32, 0.39)5 (17.24%)(−0.16, 0.50)S2Assessment of inconsistency23 (79.31%)(0.63, 0.96)06 (20.69%)(−0.12, 0.53)15Risk of bias across studies22 (75.86%)(0.58, 0.94)07 (24.14%)(−0.08, 0.56)16Additional analyses14 (48.27%)(0.29, 0.81)015 (51.72%)(0.26, 0.77)Results17Study selection25 (86.21%)(0.73, 0.99)4 (13.79%)(−0.20, 0.48)0S3Presentation of network structure18 (62.07%)(0.40, 0.85)01 (3.45%)(−0.32, 0.39)S4Summary of network geometry1 (3.45%)(−0.32, 0.39)26 (89.65%)(0.78, 1.01)2 (6.70%)(−0.28, 0.42)18Study characteristics27 (93.10%)(0.84, 1.03)2 (6.70%)(−0.28, 0.42)019Risk of bias within studies26 (89.65%)(0.78, 1.01)1 (3.45%)(−0.32, 0.39)2 (6.70%)(−0.28, 0.42)20Results of individual studies17 (58.62%)(0.35, 0.82)012 (41.38%)(0.14, 0.69)21Synthesis of results27 (93.10%)(0.84, 1.03)2 (6.70%)(−0.28, 0.42)0S5Exploration for inconsistency21 (72.41%)(0.53, 0.92)08 (27.59%)(−0.03, 0.59)22Risk of bias across studies13 (44.83%)(0.18, 0.72)016 (55.17%)(0.31, 0.80)23Results of additional analyses8 (27.59%)(−0.03, 0.59)021 (72.41%)(0.53, 0.92)Discussion24Summary of evidence25 (86.21%)(0.73, 0.99)1 (3.45%)(−0.32, 0.39)3 (10.34%)(−0.24, 0.45)25Limitations28 (96.55%)(0.90, 1.03)01 (3.45%)(−0.32, 0.39)26Conclusions27 (93.10%)(0.84, 1.03)2 (6.70%)(−0.28, 0.42)027Funding21 (72.41%)(0.53, 0.92)08 (27.59%)(−0.03, 0.59)


## 3.1. Search Results

242 related references through searching seven electronic databases and manual searches were retrieved. After reading the title and abstract, a total of 29 studies were included [15–43]. The literature screening process is shown in Figure 1.Figure 1
Flow diagram of the study.


## 3.2. Study Characteristics

The characteristics of all included NMAs were documented. After screening, 29 NMAs finally met the inclusion criteria, including 12 Chinese references (including 1 master’s thesis) and 17 English references. All eligible studies were published from May 2013 to August 2019. The study contained 4–22 types of treatments and 10–121 RCTs for 1098 RCTs in total. The NMAs were involved in up to 23 diseases, knee osteoarthritis and primary dysmenorrhea covered with 3 NMAs separately, others focusing on chronic functional constipation, lumbar disc herniation, chronic fatigue syndrome, and the like. The Jadad scale and RoB scale were used as the bias risk assessment tools. Among them, 7 articles adopted the Jadad scale, 22 articles adopted the RoB scale (1 article adopted both the Jadad scale and RoB scale), and only 1 article did not mention the risk assessment tool. The characteristics of the eligible studies are presented in Table2.Table 2
Basic characteristics of eligible studies.
Study IDDiseaseSample sizeNumber of interventionsDescription of interventionsNumber of RCTOutcomeRisk assessment toolFu et al. [15]Acute gouty arthritis493110Western medicine, acupuncture + pricking blood and cupping, Chinese medicine + cupping, acupuncture and moxibustion, Chinese medicine, Western medicine + pricking blood and cupping, pricking blood and cupping, Western medicine + acupuncture, Western medicine + massage, and Western medicine + Chinese medicine66Effective rateJadadDing [16]Abdominal distension after abdominal operation20477Acupoint application, acupoint injection, acupoint massage, moxibustion, TCM enema, TCM hot ironing, and routine nursing22Effective rateJadadLi [17]Poststroke depression189513Acupuncture + Chinese medicine, ear bean + Chinese medicine, electric acupuncture + Chinese medicine, moxibustion + Chinese medicine, auricular acupoint electroacupuncture + Chinese medicine, head acupuncture + acupoint embedding, auricular acupoint electroacupuncture + acupuncture, ear bean + acupuncture, acupuncture + acupoint injection, acupuncture + wuxing music, wuxing music + Chinese medicine, SSRIs, and acupuncture + moxibustion23Effective rateJadadBu et al. [18]Optic atrophy136911Western medicine, acupuncture, acupuncture + moxibustion, ear point + Chinese medicine, electroacupuncture, acupuncture + Chinese and Western medicine, moxibustion + Chinese and Western medicine, electroacupuncture + Chinese and Western medicine, Chinese and Western medicine, Chinese medicine, and acupuncture + Western medicine16Effective rate, vision, horizon, and visual evoked potentialRoBZhang [19]Ankylosing spondylitis22089Sulfasalazine, acupuncture, moxibustion, bee acupuncture, acupuncture + moxibustion, acupuncture + moxibustion + cupping, and moxibustion + bee needle25Effective rateRoBSong [20]Polycystic ovary syndrome460514Acupuncture-medication therapy, Western medicine, acupuncture and moxibustion, acupuncture, acupuncture + ear points, moxibustion + Chinese medicine, acupuncture + ear points, acupuncture + placebo, placebo + Western medicine, Chinese medicine + Western medicine, placebo, acupoint thread-embedding therapy and medication, Chinese medicine, and moxibustion39Ovulation rate and pregnancy rateRoBYang et al. [21]Lumbar disc herniation25894Acupuncture, acupuncture + cupping, acupuncture + massage, and acupuncture + cupping + massage30Pain improvement, effectiveness, cure rate, recurrence rate, and economic indicators.RoBLi [22]Primary dysmenorrhea46006Acupuncture, acupuncture + moxibustion, acupuncture + indirect moxibustion, warm acupuncture, electroacupuncture, and electroacupuncture + warm acupuncture56Effective rate, VAS, and dysmenorrhea symptom scoreRoBJia et al. [23]Chronic urticaria11869Acupoint catgut embedding, acupoint catgut embedding + loratadine, acupoint catgut embedding + mizolastine, acupoint catgut embedding + cetirizine, acupoint catgut embedding + epstein, loratadine, mizolastine, cetirizine, and epstein15Effective rate and adverse effectRoBLiu [24]Knee osteoarthritis34176Hyaluronic acid (HA), needle knife, needle knife + HA, needle knife + internal medicine, needle knife + massage, and needle knife + acupuncture29Effective rate, VAS, and Lysholm function scoresJadadFeng [25]Primary dysmenorrhea102599Multipoint + conventional Chinese medicine therapy, multipoint + Western medicine conventional therapy, single point + TCM, TCM, single point, multipoint, Western medicine routine therapy, and placebo103Effective rate, VAS, CMSS, and SF-MPQRoBLiu [26]Poststroke shoulder pain18989Rehabilitation, acupuncture, acupuncture + rehabilitation, acupuncture + massage, acupuncture + TCM fumigation + rehabilitation, acupuncture + massage + cupping, acupuncture + massage + rehabilitation, acupuncture + TCM + rehabilitation, and acupuncture + joint loosening + rehabilitation25Effective rate, FMA, and MBIRoBYang et al. [27]Premature ovarian insufficiency304618HRT, acupuncture, moxibustion, needle-warming moxibustion, electroacupuncture, catgut implantation at acupoint , TCM, acupuncture + TCM, moxibustion + TCM, electroacupuncture + TCM, acupoint application + TCM, auricular point sticking + TCM, acupressure + TCM, catgut implantation at acupoint + TCM, acupuncture + HRT, electroacupuncture + HRT, catgut implantation at acupoint + HRT, and acupuncture + TCM + HRT43Effective rate and adverse effectRoBZhu et al. [28]Chronic constipation1103210Acupuncture, polyethylene glycol, lactulose, linaclotide, lubiprostone, bisacodyl, prucalopride, sham acupuncture, tegaserod, and placebo40Symptoms of chronic constipation and side effectsJadadZheng [29]Chronic functional constipation43248Acupuncture, mosapride, insoluble fiber, massage, mineral water, probiotic, TENS, and moxibustion33Weekly stool frequency, Bristol score, responder rate, and adverse eventRoBQin [30]Chronic prostatitis/chronic pelvic pain syndrome12037Acupuncture, electroacupuncture, alpha-blockers, antibiotics, dual therapy, sham acupuncture, and placebo12NIH-CPSI QoL scoreRoBLi et al. [31]Knee osteoarthritis20657Common manual acupuncture, electroacupuncture, fire needle, warm needle, placebo, sham needle, and education16WOMAC, stiffness, and physical function scoresRoBMo et al. [32]Lumbar disc herniation130754Tuina, traction, acupuncture, and Chinese herbs121Invalid rate, cure rate, VAS, and JOARoBLuo et al. [33]Primary dysmenorrhea15118Traditional acupuncture, eye acupuncture, wrist-ankle acupuncture, superficial acupuncture, moxibustion, electroacupuncture, ear acupuncture, and abdominal acupuncture17Effective rateRoBYeh et al. [34]Psoriasis8696Acupuncture, acupressure, acupoint bloodletting, acupoint catgut embedding, Chinese herbal medicine, and narrow-band ultraviolet B10PASI and TCMJadad RoBChen et al. [35]Migraine36569Acupuncture, flunarizine, metoprolol, propranolol, propranolol + flunarizine, sham acupuncture, topiramate, usual care, and waiting list19Migraine episodes, the number of migraine days, migraine frequency, responder rate, and adverse event rateRoBTan et al. [36]Essential hypertension264915Electroacupuncture, moxibustion, warm needle therapy, sham acupuncture, behavioral therapy, angiotensin-converting enzyme inhibitors (ACEIs), angiotensin receptor blockers (ARBs), calcium channel blocker (CCB), beta-blocker, acupuncture-combined ACEI, acupuncture-combined CCB, acupuncture-combined behavior, electroacupuncture-combined CCB, sham acupuncture, and nontreatment31posttreatment BP changes, response rate, and MACERoBLi et al. [37]Myofascial pain syndrome169222Placebo-sham, MA, EA, DN, MET, TCT, MT, LTrP-I, MDIMST, MSN, TTM, BTX-A-TrP-I, FN, SWAM, EA&ESNC, SPM, DN&MET, BTrP-I, TrP-DN&EDU, Stretch, DN&Stretch, laser, and PT33Pain measurement (VAS, NRS, and PPT), adverse events (ROM), and functional statusRoBZhu et al. [38]Diarrhea-predominant irritable bowel syndrome93697Acupuncture, sham acupuncture, pinaverium bromide, alosetron, eluxadoline, ramosetron, and rifaximin29Effective rate, side effects (constipation and rash), and common acupuncture points.JadadXiong and Chen [39]Diabetic peripheral neuropathy26027Manual acupuncture, electroacupuncture, needle knocking acupuncture, warm needling and moxibustion, mecobalamin, no interventions, and vitamin B40global symptom improvementnonYang et al. [40]Heart failure21165Acupuncture, moxibustion, acupoint application, acupoint injection, and warming acupuncture-moxibustion26HFC and LVEFRoBZhang et al. [41]Obesity22836AAS, EA, ACE, WA, AR: acupuncture and related therapies and combination of acupuncture and related therapies.34BW, BMI, and adverse eventsRoBCorbett 2013 [42]Knee osteoarthritis97098Interferential therapy, acupuncture, TENS, pulsed electrical stimulation, balneotherapy, aerobic exercise, sham acupuncture, and muscle-strengthening exercise114WOMAC painRoBWang et al. [43]Chronic fatigue syndrome22555CbAM, SAM, Chinese herbal medicine, Western medicine, and sham acupuncture31Effective rateRoBNote: CMSS, the Cox Menstrual Symptom Scale; FMA, Fugl－Meyer motor assessment scale; MBI, modified Barthel index; TCM, traditional Chinese medicine; MACE, major adverse cardiovascular events; HFC, changes in heart function classification; LVEF, left ventricular ejection fraction; AAS, auricular acupoint stimulation; WA, warming acupuncture; ACE, acupoint catgut embedding; EA, electroacupuncture; AR, acupuncture and related therapies.


## 3.3. Methodological Quality Assessment

According to the AMSTAR2 checklist, the median score and IQR of eligible NMAs was 9.5 (6.5–10.75), and the details are presented in Tables3 and 4. The item with the best degree of compliance was “comprehensive literature search” (100%) (Table 4). Longitudinal analysis, a good degree of compliance was with item 9 (93.1%), item 11 (79.31%), and items 1, 5, and 6 (75.86%). However, a poor degree of compliance was with item 2 (17.24%) and item 3 (6.9%), and the worst degree of compliance was with item 7 (0%) and item 10 (0%).The details are presented in Tables 3 and 4 and Figure 2.Table 3
Methodological quality assessment of the included NMAs.
Item12345678910111213141516ScoreFu et al. [15]YNNPYYNPPNNNNNYN5.5Ding [16]YNNPYNNYYNNNNNNP5Li [17]YNNPYNNPYNYNPNNN5.5Bu et al. [18]YNNYYYNPYNPNPNNP7Zhang [19]YNNYNYNPYNYNPYNP6Song [20]YNNYYYNYYNPPPYYP8Yang et al. [21]YNNPNYNYYNYNYYNP8Li [22]YNNYYYNYYNYYYYPP11Jia et al. [23]YNNPYNNYYNYYYYPP9.5Liu [24]YNNYYYNYYNYYNYNP9.5Feng [25]YNNYPYNYYNYYYYPN10Liu [26]YNNPNNNYYNYNPNNP5.5Yang et al. [27]YNNYYYNYYNYYYYYP11Zhu et al. [28]NNNYYYNNYNYNNNYP6.5Zheng [29]PYYYYYNPYNYNYYPY11Qin [30]YNNYYYNYYNYYYYNY11Li et al. [31]YNNPYYNPYNYYYYYY11Mo et al. [32]PNNYYYNPYNPNNNYY7.5Luo et al. [33]YNNPNNNPYNYYNNYP6.5Yeh et al. [34]PNNPYYNPYNYYYYYP10Chen et al. [35]PYNYYNNPYNYNNPNY7.5Tan et al. [36]YNNPYYNYYNYYYNYY10.5Li et al. [37]YYNYYYNYYNYYYNNP10.5Zhu et al. [38]PNNYYYNPYNYYYPYP10Xiong and Chen [39]PNNPNNNPNNNNNNNY2.5Yang et al. [40]YYNPYYNPYNYYYNYY11Zhang et al. [41]YNNPYYNYYNYYYNNY9.5Corbett 2013 [42]YYYYYYNPYNYYYYYY13.5Wang et al. [43]YNNPNYNYYNYYYYYY10.5Score20521522220142702316161413118.64/9.5Y, yes (1 point); N, no (0 point); P, partial satisfaction (0.5 point). Item 1, did the research questions and inclusion criteria for the review include the components of PICO? Item 2, did the report of the review contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any significant deviations from the protocol? Item 3, did the review authors explain their selection of the study designs for inclusion in the review? Item 4, did the review authors use a comprehensive literature search strategy? Item 5, did the review authors perform study selection in duplicate? Item 6, did the review authors perform data extraction in duplicate? Item 7, did the review authors provide a list of excluded studies and justify the exclusions? Item 8, did the review authors describe the included studies in adequate detail? Item 9, did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review? Item 10, did the review authors report on the sources of funding for the studies included in the review? Item 11, if meta-analysis was performed, did the review authors use appropriate methods for statistical combination of results? Item 12, if meta-analysis was performed, did the review authors assess the potential impact of RoB in individual studies on the results of the meta-analysis or other evidence synthesis? Item 13, did the review authors account for RoB in individual studies when interpreting/ discussing the results of the review? Item 14, did the review authors provide a satisfactory explanation for, and discussion of, any heterogeneity observed in the results of the review? Item 15, if they performed quantitative synthesis, did the review authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review? Item 16, did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?Table 4
Summary of methodological quality assessment.
NumberItemCompletely reportedPartially reportedNot reportedFrequency (%)95% CIFrequency (%)95% CIFrequency (%)95% CI1Components of PICO question?22 (75.86%)(0.58, 0.94)6 (20.69%)(−0.12, 0.53)1 (3.45%)(−0.32, 0.39)2Review protocol?5 (17.24%)(−0.16, 0.50)024 (82.75%)(0.68, 0.98)3Explanation of study design?2 (6.90%)(−0.28, 0.42)027 (93.10%)(0.84, 1.03)4Comprehensive literature search strategy16 (55.17%)(0.31, 0.80)13 (44.83%)(0.18, 0.72)05Study selection in duplicate?22 (75.86%)(0.58, 0.94)1 (3.45%)(−0.32, 0.39)6 (20.69%)(−0.12, 0.53)6Data extraction in duplicate?22 (75.86%)(0.58, 0.94)07 (24.14%)(−0.08, 0.56)7List of excluded studies and justify the exclusions?0029 (100%)8Study characteristics14 (48.27%)(0.22, 0.74)14 (48.27%)(0.22, 0.74)1 (3.45%)(−0.32, 0.39)9Satisfactory technique for assessing risk of bias?27 (93.10%)(0.84, 1.03)1 (3.45%)(−0.32, 0.39)1 (3.45%)(−0.32, 0.39)10Sources of funding for each study?0029 (100%)11Appropriate methods?23 (79.31%)(0.63, 0.96)3 (10.34%)(−0.24, 0.45)3 (10.34%)(−0.24, 0.45)12Assess potential impact of risk of bias on the results?16 (55.17%)(0.31, 0.80)1 (3.45%)(−0.32, 0.39)12 (41.38%)(0.14, 0.69)13Account for risk of bias when interpreting/discussing?16 (55.17%)(0.31, 0.80)5 (17.24%)(−0.16, 0.50)8 (27.59%)(−0.03, 0.59)14Satisfactory explanation for and discussion of any heterogeneity?14 (48.27%)(0.29, 0.81)2 (6.90%)(−0.28, 0.42)13 (44.83%)(0.18, 0.72)15Publication bias (small sample bias) assessed and discussed?13 (44.83%)(0.18, 0.72)4 (13.79%)(−0.20, 0.48)12 (41.38%)(0.14, 0.69)16Publication bias (small sample bias) assessed and discussed?11 (37.93%)(0.09, 0.67)15 (51.72%)(0.26, 0.77)3 (10.34%)(−0.24, 0.45)Figure 2
AMSTAR2 percentage distribution diagram (%).


## 3.4. Reporting Quality Assessment

According to the PRISMA-NMA checklist, the median score and IQR of eligible NMAs was 24.5 (20.5–26.5).The item with the best degree of compliance was item 1 “Title” (100%). Longitudinal analysis, a good degree of compliance was with item 4 (96.55%), item 12 (96.55%), item 25 (96.55%), item 18 (93.10%), item 26 (93.10%), item 3 (89.65%), item 19 (89.65%), and item 24 (86.21%). However, a poor degree of compliance was with item 23 (27.59%), item 2 (17.24%), item 5 (17.24%), item 10 (17.24%), and item 11 (10.34%), and the worst degree of compliance was with item S4 (3.45%). There are significant problems in the reporting methods and results sections, such as underreporting or selective reporting. Three studies [29, 39, 42] (10.34%) reported evaluating the risk of bias within individual studies in the methods section (item 12), but did not really evaluate it in the results section (item 19). Horizontal analysis, there were four NMAs (13.79%) that scored less than 20 points, with a minimum score of 13.5. The details are presented in Tables 5 and 6.Table 5
Reporting quality assessment of the included NMAs.
ItemSection/topicFu 2019Ding 2019Li 2018Bu 2017Zhang 2018Song 2019Yang 2016Li 2017Jia 2018Liu 2016Feng 2018Liu 2019Yang 2017Zhu 2018Zheng 20181Title1111111111111112Structured summary0.50.50.50.50.50.50.50.50.50.50.50.50.50.513Rationale10.510.51111110.511114Objectives1110.5111111111115Protocol and registration0000000000000016Eligibility criteria110.50.50.51111111110.57Information sources1111110.5111111118Search0000000101101019Study selection0.51110101111011110Data collection process0.50.50.510.50.50.510.50.510.50.50.50.511Data items0.50.510.50.50.50.50.50.50.50.50.50.50.50.5S1Geometry of the network10101101111111112Risk of bias within individual studies11111111111111113Summary measures10.510.50.50.50.51111111114Planned methods of analysis101101111111111S2Assessment of inconsistency10111111101111115Risk of bias across studies10010101111011116Additional analyses00000100111001117Study selection110.510.50.5111111111S3Presentation of network structure111111011111111S4Summary of network geometry10.50.50.50.50.500.50.50.50.50.50.500.518Study characteristics11111111111111119Risk of bias within studies11111111111111020Results of individual studies00000001101011021Synthesis of results111110.5111111111S5Exploration for inconsistency10011111111100022Risk of bias across studies10000101101001023Results of additional analyses00001000010001024Summary of evidence10100.5111111111125Limitations11111111111111126Conclusions110.5110.511111111127Funding010111110101101Summary24172020.5202418.527.5262627222525.525ItemQin 2016Li 2018Mo 2019Luo 2018Yeh 2016Chen 2019Tan 2019Li 2017Zhu 2018Xiong 2018Yang 2019Zhang 2018S. 2013Wang 201711111111111111120.50.50.50.50.510.510.50.510.510.53111111111111114111111111111115000001010010106110.510.50.5110.50.51110.5710.5110.511110.50.511181010001110110091110.511111011101010.50.50.50.50.50.50.50.500.510.50.51110.50.50.50.50.510.50.50.50.50.50.50.5S11100000110000012111111111011111311110.50.50.510.50110.50.5141101110110.51110S2111001111110101511111111101011161100000110111117111110.511111111S311111111111111S40.50.50.50.50.50.50.50.50.50.50.50.50.50.5181111111110.5110.51191111111110110.512011111111101101211111111111110.51S511101111111100220111011010100023110000001000112411111111101111251111111110111126111111111111112711100111011111Score2927.524.521.520.5252528.52713.52824.523.51924.5/23.62Table 6
Summary of reporting quality assessment.
SectionItemCompletely reportedPartially reportedNot reportedFrequency (%)95% CIFrequency (%)95% CIFrequency (%)95% CITitle1Title29 (100%)00Abstract2Structured summary5 (17.24%)(−0.16, 0.50)24 (82.76%)(0.68, 0.98)0Introduction3Rationale26 (89.65%)(0.78, 1.01)3 (10.34%)(−0.24, 0.45)04Objectives28 (96.55%)(0.90, 1.03)1 (3.45%)(−0.32, 0.39)0Methods5Protocol and registration5 (17.24%)(−0.16, 0.50)024 (82.76%)(0.68, 0.98)6Eligibility criteria19 (65.52%)(0.44, 0.87)10 (34.48%)(0.05, 0.64)07Information sources22 (75.86%)(0.58, 0.94)5 (17.24%)(−0.16, 0.50)08Search12 (41.38%)(0.14, 0.69)017 (58.62%)(0.35, 0.82)9Study selection22 (75.86%)(0.58, 0.94)2 (6.70%)(−0.28, 0.42)5 (17.24%)(−0.16, 0.50)10Data collection process5 (17.24%)(−0.16, 0.50)23 (79.31%)(0.63, 0.96)1 (3.45%)(−0.32, 0.39)11Data items3 (10.34%)(−0.24, 0.45)26 (89.65%)(0.78, 1.01)0S1Geometry of the network16 (55.17%)(0.31, 0.80)013 (44.83%)(0.18, 0.72)12Risk of bias within individual studies28 (96.55%)(0.90, 1.03)01 (3.45%)(−0.32, 0.39)13Summary measures18 (62.07%)(0.40, 0.85)11 (37.93%)(0.09, 0.67)014Planned methods of analysis23 (79.31%)(0.63, 0.96)1 (3.45%)(−0.32, 0.39)5 (17.24%)(−0.16, 0.50)S2Assessment of inconsistency23 (79.31%)(0.63, 0.96)06 (20.69%)(−0.12, 0.53)15Risk of bias across studies22 (75.86%)(0.58, 0.94)07 (24.14%)(−0.08, 0.56)16Additional analyses14 (48.27%)(0.29, 0.81)015 (51.72%)(0.26, 0.77)Results17Study selection25 (86.21%)(0.73, 0.99)4 (13.79%)(−0.20, 0.48)0S3Presentation of network structure18 (62.07%)(0.40, 0.85)01 (3.45%)(−0.32, 0.39)S4Summary of network geometry1 (3.45%)(−0.32, 0.39)26 (89.65%)(0.78, 1.01)2 (6.70%)(−0.28, 0.42)18Study characteristics27 (93.10%)(0.84, 1.03)2 (6.70%)(−0.28, 0.42)019Risk of bias within studies26 (89.65%)(0.78, 1.01)1 (3.45%)(−0.32, 0.39)2 (6.70%)(−0.28, 0.42)20Results of individual studies17 (58.62%)(0.35, 0.82)012 (41.38%)(0.14, 0.69)21Synthesis of results27 (93.10%)(0.84, 1.03)2 (6.70%)(−0.28, 0.42)0S5Exploration for inconsistency21 (72.41%)(0.53, 0.92)08 (27.59%)(−0.03, 0.59)22Risk of bias across studies13 (44.83%)(0.18, 0.72)016 (55.17%)(0.31, 0.80)23Results of additional analyses8 (27.59%)(−0.03, 0.59)021 (72.41%)(0.53, 0.92)Discussion24Summary of evidence25 (86.21%)(0.73, 0.99)1 (3.45%)(−0.32, 0.39)3 (10.34%)(−0.24, 0.45)25Limitations28 (96.55%)(0.90, 1.03)01 (3.45%)(−0.32, 0.39)26Conclusions27 (93.10%)(0.84, 1.03)2 (6.70%)(−0.28, 0.42)027Funding21 (72.41%)(0.53, 0.92)08 (27.59%)(−0.03, 0.59)


## 4. Discussion

### 4.1. Summary of Main Findings

The goal of this cross-sectional survey was to evaluate the methodological and reporting quality of NMAs and summarize the effects of different treatments of acupuncture and moxibustion (Table6). A total of 29 acupuncture NMAs were included in this study. From the perspective of the number and publication time, the development of NMA in acupuncture and moxibustion was still in the initial stage, with a small number, but it had shown a trend of gradual growth. In terms of disease, more than half were chronic pain. In terms of intervention measures, more than half were acupuncture combined therapy. It was not difficult to find that NMAs of acupuncture and moxibustion was still limited to several diseases, and there were still large gaps in many aspects.The methodological quality of NMAs was important, so we evaluated the methodological quality of NMAs in acupuncture and moxibustion according to AMSTAR2 tool. The results showed some methodological deficiencies, particularly with regard to item 2, item 3, item 7, and item 10. The highest NMA score for each item was 13.5, the lowest was 2.5, and the median and IQR was 9.5 (6.5–10.75), indicating average methodological quality.The quality evaluation of the report showed that the quality of acupuncture NMAs was generally acceptable, indicating that the NMA researchers had a high level of evidence-based medical knowledge and scientific research literacy. Some items need to be improved, particularly with regard to the structured summary (item 2), protocol and registration (item 5), search (item 8), data collection process (item 10), data items (item 11), additional analyses (item 16), risk of bias across studies (Results section) (item 22), and results of additional analyses (item 23). From the perspective of a single NMA, the highest score was 29, the lowest was only 13.5, and the median and IQR was 9.5 (6.5–10.75), showing that the quality of reports included in the study was of average quality. PRISMA-NMA checklist was helpful to improve the reporting quality of acupuncture NMAs. Therefore, it was necessary to improve the comprehensiveness and standardization of the report.


### 4.2. Strengths and Limitations

First, this was the first study that evaluated the methodological and reporting quality of NMAs comprehensively, which complied with the methodological and reporting guidelines in the field of acupuncture and moxibustion. Even though there were two studies regarding the methodological or reporting quality of NMAs in TCM, one English article excluded acupuncture and moxibustion [44] and one Chinese article included acupuncture and moxibustion literature incompletely [45]. Second, compared with published quality studies of NMAs in acupuncture and moxibustion, this review implemented a more comprehensive and detailed literature retrieval strategy. In addition, the unpublished studies and the references initially were also retrieved manually. As a result, the results were more credible.This research also had presented some limitations: first, there was no specific methodological quality assessment tool for NMAs, even though AMSTAR2 was generally used in the quality evaluation of systematic review and meta-analysis. Second, in this study, the quality of each NMA was quantified by the assignment method, and there were some controversies on whether the weight of each item was consistent. Third, even if a comprehensive literature search strategy was used, there was no guarantee that all relevant literatures were identified. Finally, since only Chinese and English studies were included, there may be a lack of data to influence the results.


## 4.1. Summary of Main Findings

The goal of this cross-sectional survey was to evaluate the methodological and reporting quality of NMAs and summarize the effects of different treatments of acupuncture and moxibustion (Table6). A total of 29 acupuncture NMAs were included in this study. From the perspective of the number and publication time, the development of NMA in acupuncture and moxibustion was still in the initial stage, with a small number, but it had shown a trend of gradual growth. In terms of disease, more than half were chronic pain. In terms of intervention measures, more than half were acupuncture combined therapy. It was not difficult to find that NMAs of acupuncture and moxibustion was still limited to several diseases, and there were still large gaps in many aspects.The methodological quality of NMAs was important, so we evaluated the methodological quality of NMAs in acupuncture and moxibustion according to AMSTAR2 tool. The results showed some methodological deficiencies, particularly with regard to item 2, item 3, item 7, and item 10. The highest NMA score for each item was 13.5, the lowest was 2.5, and the median and IQR was 9.5 (6.5–10.75), indicating average methodological quality.The quality evaluation of the report showed that the quality of acupuncture NMAs was generally acceptable, indicating that the NMA researchers had a high level of evidence-based medical knowledge and scientific research literacy. Some items need to be improved, particularly with regard to the structured summary (item 2), protocol and registration (item 5), search (item 8), data collection process (item 10), data items (item 11), additional analyses (item 16), risk of bias across studies (Results section) (item 22), and results of additional analyses (item 23). From the perspective of a single NMA, the highest score was 29, the lowest was only 13.5, and the median and IQR was 9.5 (6.5–10.75), showing that the quality of reports included in the study was of average quality. PRISMA-NMA checklist was helpful to improve the reporting quality of acupuncture NMAs. Therefore, it was necessary to improve the comprehensiveness and standardization of the report.


## 4.2. Strengths and Limitations

First, this was the first study that evaluated the methodological and reporting quality of NMAs comprehensively, which complied with the methodological and reporting guidelines in the field of acupuncture and moxibustion. Even though there were two studies regarding the methodological or reporting quality of NMAs in TCM, one English article excluded acupuncture and moxibustion [44] and one Chinese article included acupuncture and moxibustion literature incompletely [45]. Second, compared with published quality studies of NMAs in acupuncture and moxibustion, this review implemented a more comprehensive and detailed literature retrieval strategy. In addition, the unpublished studies and the references initially were also retrieved manually. As a result, the results were more credible.This research also had presented some limitations: first, there was no specific methodological quality assessment tool for NMAs, even though AMSTAR2 was generally used in the quality evaluation of systematic review and meta-analysis. Second, in this study, the quality of each NMA was quantified by the assignment method, and there were some controversies on whether the weight of each item was consistent. Third, even if a comprehensive literature search strategy was used, there was no guarantee that all relevant literatures were identified. Finally, since only Chinese and English studies were included, there may be a lack of data to influence the results.


## 5. Conclusion

The NMAs methodological and report quality related to acupuncture and moxibustion were general, and there was still room for improvement in some aspects. For example, the researchers should design the scheme in advance before carrying out the study, design and carry out the study strictly in accordance with PICOS, and present the network structure, so as to improve the prospective and reliability of the study. Considering the importance of PRISMA-NMA checklist to NMA, we advise that the researchers should strictly follow the PRISMA-NMA checklist when writing a NMA.



---

*Source: 2672173-2021-01-11.xml*