42

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

APPENDIX A. Search Strategies

Table A-1. Search strategy for PubMed (5/29/2012, updated 9/28/2012)

Set # Terms Results

1 Dabigatran[tiab] OR desirudin[tiab] OR edoxaban[tiab] OR rivaroxaban[tiab] OR 

apixaban[tiab] OR betrixaban[tiab] OR YM150[tiab] OR razaxaban[tiab] OR “dabigatran etexilate”[Supplementary Concept] OR “desirudin”[Supplementary Concept] OR 

“edoxaban”[Supplementary Concept] OR “rivaroxaban”[Supplementary Concept] OR 

“apixaban”[Supplementary Concept] OR “betrixaban”[Supplementary Concept] OR “razaxaban hydrochloride”[Supplementary Concept] OR “factor Xa, Glu-Gly-Arg-”[Supplementary 

Concept] OR “KFA1411”[Supplementary Concept]

1319

2 (((knee[tiab] OR hip[tiab] OR elbow[ti]) AND (replacement[tiab] OR Arthroplasty[tiab]))) OR 

(“Orthopedic Procedures”[Mesh])

194066

3 #1 AND #2 298

4 (“Review”[Publication Type] OR “Review Literature as Topic”[Mesh]) OR (“Meta-Analysis 

as Topic”[Mesh] OR “Meta-Analysis”[Publication Type]) OR systematic[sb]

1782948

5 #3 AND #4 117

Table A-2. Search strategy for Embase (5/30/2012, updated 9/28/2012)

Set # Terms Results

1 ‘dabigatran’/exp OR dabigatran OR ‘desirudin’/exp OR desirudin OR ‘edoxaban’/exp 

OR edoxaban OR ‘rivaroxaban’/exp OR rivaroxaban OR ‘apixaban’/exp OR apixaban 

OR’betrixaban’/exp OR betrixaban OR ‘ym150’/exp OR ym150 OR ‘razaxaban’/exp OR 

razaxaban OR ‘factor xa inhibitors’ OR ‘factor xa inhibitor’/exp OR ‘factor xa inhibitor’ OR 

‘fxa inhibitors’ OR ‘fxa inhibitor’ OR’direct thrombin inhibitor’ OR ‘direct thrombin inhibitors’ 

OR dtis OR ‘novel anticoagulants’ OR ‘new anticoagulants’ OR ‘novel anticoagulant’ OR 

‘new anticoagulant’

10942

2 ‘orthopedic surgery’/exp OR (hip:ab,ti OR knee:ab,ti OR elbow:ab,ti AND (replacement:ab,ti 

OR arthroplasty:ab,ti))

351,364

3 #1 AND #2 1522

4 #3 limited to Systematic reviews or meta –analysis AND (embase)/lim NOT (medline)/lim 43

Table A-3. Search strategy for Cochrane Database of Systematic Reviews (5/30/2012, updated 

9/28/2012)

Set # Term Results

1 dabigatran OR desirudin OR edoxaban OR rivaroxaban OR apixaban OR betrixaban OR 

YM150 OR razaxaban OR “factor Xa inhibitors” OR “factor Xa inhibitor” OR “fxa inhibitors” 

OR “fxa inhibitor” OR “direct thrombin inhibitor” OR “direct thrombin inhibitors” OR DTIs 

OR “novel anticoagulants” OR “new anticoagulants” OR “novel anticoagulant” OR “new 

anticoagulant”

472

2 MeSH descriptor Orthopedic Procedures explode all trees OR (knee):ti,ab,kw or 

(elbow):ti,ab,kw) AND (replacement):ti,ab,kw or (arthroplasty):ti,ab,kw

8138

3 #1 AND #2 117

4 #3 limited to Systematic reviews or meta-analysis 8
43

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

APPENDIX B. Excluded Studies

All citations listed in Table B-1 were reviewed in their full-text version and excluded for the 

reason indicated. An alphabetical reference list follows the table.

Table B-1. Excluded studies with reasons

Reference Not a systematic review Does not address Key 

Questions

Cohen, 2012 Xa

Dahl, 2009 X

Dahl, 2010 X

Diamantopoulos, 2010 X

Duggan, 2009 X

Eriksson, 2011 X

Eriksson, 2009 X

Falck-Ytter, 2012 X

Friedman, 2011 X

Friedman, 2010 X

Friedman, 2011 X

Goff, 2011 X

Gomez-Outes, 2011 X

Gras, 2011 X

Holmes, 2009 X

Hull, 2010 X

Imberti, 2009 X

Jacobs, 2012 X

Kwong, 2011 X

Kwong, 2011 X

Lazo-Langner, 2009 X

Lee, 2012 X

Lereun, 2011 X

Mantha, 2011 X

Maratea, 2011 X

Melillo, 2010 X

Merli, 2009 X

Miller, 2012 X

Mont, 2011 X

Nieto, 2012 X

Poultsides, 2012 X

Prom, 2011 X

Raskob, 2012 X

Stevenson, 2009 X

Trkulja, 2010 X

Watkins, 2011 X

Wolowacz, 2011 X

Wolowacz, 2009 X

a

Rated as poor-quality systematic review and excluded.
44

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

LIST OF EXCLUDED STUDIES

Cohen A, Drost P, Marchant N, et al. The efficacy 

and safety of pharmacological prophylaxis of venous 

thromboembolism following elective knee or hip 

replacement: systematic review and network metaanalysis. Clin Appl Thromb Hemost. 2012;18(6):611-

627.

Dahl OE. Dabigatran etexilate for the prophylaxis of 

venous thromboembolism after hip or knee replacement 

rationale for dose regimen. Clin Appl Thromb Hemost. 

2009;15 Suppl 1:17S-24S.

Dahl OE, Quinlan DJ, Bergqvist D, et al. A critical 

appraisal of bleeding events reported in venous 

thromboembolism prevention trials of patients 

undergoing hip and knee arthroplasty. J Thromb 

Haemost. 2010;8(9):1966-75.

Diamantopoulos A, Lees M, Wells PS, et al. Costeffectiveness of rivaroxaban versus enoxaparin for the 

prevention of postsurgical venous thromboembolism in 

Canada. Thromb Haemost. 2010;104(4):760-70.

Duggan ST, Scott LJ, Plosker GL. Rivaroxaban: 

a review of its use for the prevention of venous 

thromboembolism after total hip or knee replacement 

surgery. Drugs. 2009;69(13):1829-51.

Eriksson BI, Dahl OE, Huo MH, et al. Oral dabigatran 

versus enoxaparin for thromboprophylaxis after primary 

total hip arthroplasty (RE-NOVATE II*). A randomised, 

double-blind, non-inferiority trial. Thromb Haemost. 

2011;105(4):721-9.

Eriksson BI, Friedman RJ. Dabigatran etexilate: pivotal 

trials for venous thromboembolism prophylaxis after 

hip or knee arthroplasty. Clin Appl Thromb Hemost. 

2009;15 Suppl 1:25S-31S.

Falck-Ytter Y, Francis CW, Johanson NA, et al. 

Prevention of VTE in orthopedic surgery patients: 

Antithrombotic Therapy and Prevention of Thrombosis, 

9th ed: American College of Chest Physicians 

Evidence-Based Clinical Practice Guidelines. Chest. 

2012;141(2 Suppl):e278S-325S.

Friedman RJ. Novel oral anticoagulants for VTE 

prevention in orthopedic surgery: overview of phase 3 

trials. Orthopedics. 2011;34(10):795-804.

Friedman RJ, Dahl OE, Rosencher N, et al. Dabigatran 

versus enoxaparin for prevention of venous 

thromboembolism after hip or knee arthroplasty: 

a pooled analysis of three trials. Thromb Res. 

2010;126(3):175-82.

Friedman RJ, Sengupta N, Lees M. Economic impact 

of venous thromboembolism after hip and knee 

arthroplasty: potential impact of rivaroxaban. Expert 

Rev Pharmacoecon Outcomes Res. 2011;11(3):299-306.

Goff T, Kontakis G, Giannoudis PV. Safety and efficacy 

of rivaroxaban for thromboprophylaxis following 

lower limb surgery: an update. Expert Opin Drug Saf. 

2011;10(5):687-96.

Gomez-Outes A, Terleira-Fernandez A, SuarezGea ML, et al. New oral anticoagulants for 

thromboprophylaxis after total hip or knee replacement: 

A meta-analysis and indirect treatment comparisons. 

Basic and Clinical Pharmacology and Toxicology. 

2011;109:34.

Gras J. Edoxaban for the prevention of thromboembolic 

events after surgery. Drugs Today (Barc). 

2011;47(10):753-61.

Holmes M, Carroll C, Papaioannou D. Dabigatran 

etexilate for the prevention of venous thromboembolism 

in patients undergoing elective hip and knee surgery: 

a single technology appraisal. Health Technol Assess. 

2009;13 Suppl 2:55-62.

Hull RD, Liang J, Brant R. Pooled analysis of trials may, 

in the presence of heterogeneity inadvertently lead to 

fragile conclusions due to the importance of clinically 

relevant variables being either hidden or lost when the 

findings are pooled. Thromb Res. 2010;126(3):164-5.

Imberti D, Dall’Asta C, Pierfranceschi MG. Oral 

factor Xa inhibitors for thromboprophylaxis in major 

orthopedic surgery: a review. Intern Emerg Med. 

2009;4(6):471-7.

Jacobs JJ, Mont MA, Bozic KJ, et al. American 

Academy of Orthopaedic Surgeons clinical practice 

guideline on: preventing venous thromboembolic 

disease in patients undergoing elective hip and knee 

arthroplasty. J Bone Joint Surg Am. 2012;94(8):746-7.

Kwong LM. Therapeutic potential of rivaroxaban in the 

prevention of venous thromboembolism following hip 

and knee replacement surgery: a review of clinical trial 

data. Vasc Health Risk Manag. 2011;7:461-6.

Kwong LM. Cost-effectiveness of rivaroxaban after 

total hip or total knee arthroplasty. Am J Manag Care. 

2011;17(1 Suppl):S22-6.
45

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

Lazo-Langner A, Hawell J, Kovacs MJ, et al. A 

systematic review and meta-analysis of proportions of 

thrombosis and bleeding in patients receiving venous 

thromboembolism (VTE) prophylaxis after orthopedic 

surgery (OS). an update. Blood. 2009;114(22).

Lee S, White CM. Upcoming oral factor Xa inhibitors 

for venous thromboembolism prophylaxis in patients 

undergoing major orthopedic surgery: rivaroxaban 

(Xarelto) and apixaban (Eliquis) review. Conn Med. 

2012;76(1):39-42.

Lereun C, Wells P, Diamantopoulos A, et al. An 

indirect comparison, via enoxaparin, of rivaroxaban 

with dabigatran in the prevention of venous 

thromboembolism after hip or knee replacement. J Med 

Econ. 2011;14(2):238-44.

Mantha S. Oral factor Xa inhibitors vs. enoxaparin for 

thromboprophylaxis after joint replacement surgery: A 

meta-analysis. Journal of Thrombosis and Haemostasis. 

2011;9:189.

Maratea D, Fadda V, Trippoli S, et al. Prevention of 

venous thromboembolism after major orthopedic 

surgery: indirect comparison of three new oral 

anticoagulants. J Thromb Haemost. 2011;9(9):1868-70.

Melillo SN, Scanlon JV, Exter BP, et al. Rivaroxaban 

for thromboprophylaxis in patients undergoing 

major orthopedic surgery. Ann Pharmacother. 

2010;44(6):1061-71.

Merli G, Spyropoulos AC, Caprini JA. Use of 

emerging oral anticoagulants in clinical practice: 

translating results from clinical trials to orthopedic 

and general surgical patient populations. Ann Surg. 

2009;250(2):219-28.

Miller CS, Grandi SM, Shimony A, et al. MetaAnalysis of Efficacy and Safety of New Oral 

Anticoagulants (Dabigatran, Rivaroxaban, Apixaban) 

Versus Warfarin in Patients With Atrial Fibrillation. Am 

J Cardiol. 2012.

Mont MA, Jacobs JJ, Boggio LN, et al. Preventing 

venous thromboembolic disease in patients undergoing 

elective hip and knee arthroplasty. J Am Acad Orthop 

Surg. 2011;19(12):768-76.

Nieto JA, Espada NG, Merino RG, et al. Dabigatran, 

Rivaroxaban and Apixaban versus Enoxaparin for 

thomboprophylaxis after total knee or hip arthroplasty: 

Pool-analysis of phase III randomized clinical trials. 

Thromb Res. 2012;130(2):183-91.

Poultsides LA, Gonzalez Della Valle A, Memtsoudis 

SG, et al. Meta-analysis of cause of death 

following total joint replacement using different 

thromboprophylaxis regimens. J Bone Joint Surg Br. 

2012;94(1):113-21.

Prom R, Spinler SA. The role of apixaban for 

venous and arterial thromboembolic disease. Ann 

Pharmacother. 2011;45(10):1262-83.

Raskob GE, Gallus AS, Pineo GF, et al. Apixaban 

versus enoxaparin for thromboprophylaxis after hip 

or knee replacement: pooled analysis of major venous 

thromboembolism and bleeding in 8464 patients from 

the ADVANCE-2 and ADVANCE-3 trials. J Bone Joint 

Surg Br. 2012;94(2):257-64.

Stevenson M, Scope A, Holmes M, et al. Rivaroxaban 

for the prevention of venous thromboembolism: a single 

technology appraisal. Health Technol Assess. 2009;13 

Suppl 3:43-8.

Trkulja V, Kolundzic R. Rivaroxaban vs dabigatran for 

thromboprophylaxis after joint-replacement surgery: 

exploratory indirect comparison based on meta-analysis 

of pivotal clinical trials. Croat Med J. 2010;51(2):113-

23.

Watkins PB, Desai M, Berkowitz SD, et al. Evaluation 

of drug-induced serious hepatotoxicity (eDISH): 

application of this data organization approach to phase 

III clinical trials of rivaroxaban after total hip or knee 

replacement surgery. Drug Saf. 2011;34(3):243-52.

Wolowacz SE. Pharmacoeconomics of dabigatran 

etexilate for prevention of thromboembolism after 

joint replacement surgery. Expert Rev Pharmacoecon 

Outcomes Res. 2011;11(1):9-25.

Wolowacz SE, Roskell NS, Plumb JM, et al. Efficacy 

and safety of dabigatran etexilate for the prevention 

of venous thromboembolism following total hip or 

knee arthroplasty. A meta-analysis. Thromb Haemost. 

2009;101(1):77-85.
46

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

APPENDIX C. Sample Data Abstraction Form

First author, year, Reference Library#

STATED OBJECTIVE OF PAPER: “ ”

METHODS:

Databases accessed for literature search: X, Y, Z... and abstracts of (meetings, Websites, etc.): 

Search date: 

Language limits for search: 

Inclusion criteria: Cut and paste from article AND ensure the following are addressed:

- Study design type:

- Patients: Any characteristic that would include or exclude (e.g., under 18 years)

- Intervention: Drugs of interest

- Comparator: What is considered a valid comparator for the drug of interest?

- Outcomes: Any of the following (also provide any definitions given by the authors): 

1. All-cause mortality

2. VTE-related mortality

3. VTE (only if DVT and PE not given separately)

4. Symptomatic DVT

5. Nonfatal PE

6. Serious AEs

7. Fatal bleeding

8. Major bleeding

9. Bleeding from the surgical site

10. Rehospitalization (includes bleeding that requires reoperation).

Exclusion criteria: Cut and paste from article

Summary of analysis approach: 

- System used (RevMan, Peto, CMA, etc.)

- Report statistic (OR, RR, RD, MD, combination?)

- Special procedures (double-checking, etc.)

- Heterogeneity addressed?

- Publication bias addressed? 

- Subgroup analyses?

- Sensitivity analyses?

-

Funding Source: Look carefully for pharma $

QUALITY: See separate quality rating form
47

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

RESULTS: Number of key questions: XX (if multiple KQs, complete this section for each KQ)

A. Number of studies included (if numbers vary by KQ, give total and number for each KQ): 

XX met eligibility; XX analyzed (were any articles specific excluded, why?)

B. Patient characteristics (range across studies):

Type of surgery: Knee replacement (n=); hip replacement (n=); either knee or hip (n=)

Sex (female-n %): ---- to ---- (XX.X to XX.X%)

Sample size (n): YY to YY, YYY

Mean age (years): ZZ.Z to ZZ.Z

Mean BMI or weight: AA.A to AA.A 

Veteran settings, if given: 

Risk factors for bleeding (prior GI bleed, anemia, renal insufficiency, DM): 

Intervention drugs: (generic name, number of studies, notes about dosage)

1. KQ1 – newer oral anticoagulants (FXa or DTI)

2. KQ2 – combined pharmacological (any type) + mechanical modalities

3. KQ3 – new oral anticoagulant (FXa or DTI)

Comparator: 

1. KQ1 – LMWH, UFH, warfarin, aspirin

2. KQ2 – pharmacological treatment alone

3. KQ3 – other newer oral anticoagulant (FXa or DTI) – direct or indirect comparison

Concurrent other drug administration? (yes/no)

- Antiplatelet drugs: 

- Other

C. Outcomes: (FXa vs LMWH) 

Outcomes definition:

Were they objectively evaluated? 

Was there missing data? 

Other specifics mentioned about quality of results:

Duration of anticoagulation:

- Knee: e.g., 5–14 days (n=)

≥15 days (n=)

- Hip: e.g., < 28 days (n=)

≥ 28 days (n=)

Followup timing:

- e.g., <14 days (n=)

- 14–30 days (n=)

- > 30 days (n=)

-
48

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

Risk of bias for primary studies: Any standard ratings given? Specific issues? (e.g., blinding? 

adjudication? poor completion rates?)

Quantitative summaries: Give # participants, # studies, summary estimate and 95% CI, I2

, RD 

(per 1,000), strength of evidence (SOE) if given. [For example: 22,838 participants, 11 studies; 

OR 0.95 (95% CI, 0.55 to 1.63); I2

=43%; RD=0 fewer events (CI, 2 fewer to 1 more) per 1,000; 

SOE=High]

Provide these data for the following outcomes, if given:

Mortality:

VTE-related mortality:

Total VTE:

Symptomatic DVT: 

Nonfatal PE: 

Serious AEs:

Major bleeding: 

Bleeding from the surgical site: 

Rehospitalization: (give reason if possible)

Other outcome of significance:

Did they measure publication bias? (via funnel plots, etc.)

Subgroup analyses: If presented, give type and outcome

Dose: 

Type of surgery:

Within drug class:

Multiple treatments:

Sensitivity analysis: If presented, give type and outcome

AUTHOR’S CONCLUSION: (take-home message): “ ”
49

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

APPENDIX D. Criteria Used in Quality Assessment of

Systematic Reviews

For reviews, first determine whether it is a systematic review. To be a systematic review, it 

must include a methods section that describes (1) a search strategy and (2) an a priori approach 

to synthesizing the data. For reviews determined to meet the systematic review criteria, assess 

methodological quality.* 

General instructions: The purpose of this rating tool is to evaluate the scientific quality of

systematic reviews. It is not intended to measure the literary quality, importance, relevance,

originality, or other attributes of systematic reviews. 

Step 1: Grade each criterion listed below as “Yes,” “No,” “Can’t tell” or “Not Applicable.” 

Factors to consider when making an assessment are listed under each criterion. Where 

appropriate (particularly when assigning a “No,” or “Can’t tell” score), please provide a 

brief rationale for your decision (in parentheses).

1.	 Is a focused clinical question clearly stated?

At a minimum, the question should be developed a priori and should clearly identify 

population and outcomes. The study question does not have to be in PICO format 

(Population, Intervention, Comparisons, Outcomes.)

[] Yes [] No [] Can’t tell [] N/A

2.	 Are the search methods used to identify relevant studies clearly described?

Search methods described in enough detail to permit replication (The report must include 

search date, databases used, and search terms (Key words and/or MESH terms must be stated 

and where feasible the search strategy should be provided.)

[] Yes [] No [] Can’t tell [] N/A

3.	 Was a comprehensive literature search performed?

At least 2 electronic sources should be searched and electronic searches should be 

supplemented by consulting: reference lists from prior reviews, textbooks, or included 

studies; specialized registries (e.g., Cochrane registries); or queries to experts in the field.

[] Yes [] No [] Can’t tell [] N/A

4.	 Was selection bias avoided?

Study reports the number of studies identified through searches, the numbers excluded, and 

gives appropriate reasons for excluding – based on explicit inclusion/exclusion criteria. 

[] Yes [] No [] Can’t tell [] N/A

5.	 Was there duplicate study selection and data extraction?

Did two or more raters make inclusion/exclusion decisions, abstract data, and assess study 

quality – either independently or with one rater over-reading the first raters result?

Was an appropriate method used to resolve disagreements (e.g., a consensus procedure)?

[] Yes [] No [] Can’t tell [] N/A
50

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

6.	 Were the characteristics of the included studies provided?

In an aggregated form such as a table, data from the original studies should be provided on 

the participants, interventions and outcomes. The ranges of characteristics in all the studies 

analyzed (e.g., age, race, sex, relevant socioeconomic data, disease status, duration, severity 

or other diseases) should be reported.

[] Yes [] No [] Can’t tell [] N/A

7.	 Was the scientific quality of the included studies assessed and documented?

A priori methods of assessment should be provided and criteria used to assess study quality 

specified in enough detail to permit replication. 

[] Yes [] No [] Can’t tell [] N/A

8.	 Were the methods used to combine the findings of studies appropriate?

For pooled results, an accepted quantitative method of pooling should be used (i.e., more 

than simple addition; e.g., random-effects or fixed-effect model). For pooled results, a 

qualitative and quantitative assessment of homogeneity (Cochran’s Q and/or I2

) should be 

performed. If only qualitative analyses are completed, the study should describe the reasons 

that quantitative analyses were not completed.

[] Yes [] No [] Can’t tell [] N/A

9.	 Was the scientific quality of the included studies used appropriately in formulating

conclusions?

The results of the methodological rigor and scientific quality should be considered in the 

analysis (e.g. subgroup analyses) and the conclusions of the review, and explicitly stated in 

formulating recommendations.

[] Yes [] No [] Can’t tell [] N/A

10.	Was publication bias assessed?

Publication bias tested using funnel plots, test statistics (e.g., Egger’s regression test), and/or 

search of trials registry for unpublished studies.

[] Yes [] No [] Can’t tell [] N/A

11.	Was the conflict of interest stated?

Potential sources of support should be clearly acknowledged in both the systematic review 

and the included studies.

[] Yes [] No [] Can’t tell [] N/A

12.	Are the stated conclusions supported by the data presented?

Were the conclusions made by the author(s) supported by the data and/or analyses reported in 

the systematic review?

[] Yes [] No [] Can’t tell [] N/A

Step 2: Rate the overall quality of the SR as “Good,” “Fair,” or “Poor” using the guidance

below. 

Good = After considering items 1-12, item 12 is rated “Yes” with no important limitations. 

This means that few of the items 1-12 are rated “No,” and none of the limitations are thought to 

decrease the validity of the conclusions. If items 3, 4, 7, or 8 are rated “No,” then the review is 

likely to have major flaws
51

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

Fair = After considering items 1-12, item 12 is rated “Yes,” but with at least some important 

limitations. This means that enough of the items 1-12 are rated “No” to introduce some 

uncertainty about the validity of the conclusions.

Poor = After considering items 1-12, item 12 is rated “No.” This means that several of items 

1-12 are rated “No,” introducing serious uncertainty about the validity of the conclusions.

*Adapted from:

1. Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool 

to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 

2007;7:10.

2. Moher D, Cook DJ, Eastwood S, et al. Improving the quality of reports of meta-analyses 

of randomised controlled trials: the QUOROM statement. Quality of Reporting of Metaanalyses. Lancet. 1999;354(9193):1896-900.

3. Marinopoulos SS, Dorman T, Ratanawongsa N, et al. Effectiveness of continuing medical 

education. Evid Rep Technol Assess (Full Rep). 2007(149):1-69.

Table D-1 shows the quality ratings for the systematic reviews included in this evidence report.

Table D-1. Quality assessment for included systematic reviews

Criteria for grading the quality of a 

systematic review (SR)

Neumann,

2012

Sobieraj, 

2012

GomezOutes, 

2012

Loke, 

2011

Ringerike, 

2011

Alves, 

2011

Q1. Is a focused clinical question clearly stated? Yes Yes Yes Yes Yes Yes

Q2. Are the search methods used to identify 

relevant studies clearly described?

Yes Yes Yes Yes Yes Yes

Q3. Was a comprehensive literature search 

performed?

Yes Yes Yes Yes Yes Yes

Q4. Was selection bias avoided? Yes Yes Yes Yes Yes Yes

Q5. Was there duplicate study selection and 

data extraction?

Yes Yes Yes Yes Yes Yes

Q6. Were the characteristics of the included 

studies provided?

Yes Yes Yes Yes No Yes

Q7. Was the scientific quality of the included 

studies assessed and documented?

Yes Yes Yes Yes Yes Can’t 

tell

Q8. Were the methods used to combine the 

findings of studies appropriate?

Yes Yes Yes Yes Yes Yes

Q9. Was the scientific quality of the included 

studies used appropriate in formulating 

conclusions?

Yes Yes Yes Yes Yes Yes

Q10. Was publication bias assessed? Yes Yes Yes Yes Yes Yes

Q11. Was the conflict of interest stated? Yes Yes Yes Yes Can’t tell Yes

Q12. Are the stated conclusions supported by 

the data presented?

Yes Yes Yes Yes Yes Yes

Overall quality Good Good Good Good Good Good
52

ew Oral Anticoagulants for Thromboprophylaxis Evidence-based Synthesis Program

eer Review Comments

Comment Response

 scope, and methods for this review clearly described?

 clear and KQs relevant to current clinical practice in VA. Scope as 

appropriate and clinically relevant. Methods are rigorous, transparent, and 

rding to latest accepted principles of evidence based medicine.

Thank you for your confidence in our process.

ents from reviewer 2. Thank you.ents from reviewer 3. Thank you.

tion of bias in our synthesis of the evidence?

ted. Transparency of methods allows for an open assessment of bias and 

ssess validity and accept results as valid for use in informing clinical 

Thank you again for your confidence in our process.

ents from reviewer 2. Thank you.

ted. Thank you.

ished or unpublished studies that we may have overlooked? 

 references to suggest. Thank you.

ents from reviewer 2. Thank you.

m aware of Thank you.

nal suggestions or comments below. If applicable, please indicate the page and line numbers from the draft report.

 transparency regarding conflict of interest

 report made by manufacturers Dr Ortel has potential conflicts of interest 

 and b. The sections Dr Ortel worked on were listed. Reader would be 

ss bias.

We added a description of Dr. Ortel’s role in the project.

e summary, 3rd paragraph, last sentence discussing ‘Disadvantages of newer 

s...’ From a clinical standpoint we are most concerned with veteran safety 

ecific antidote is a primary concern. Would edit that sentence to place this 

The recommended change has been made.

ary 35-day rate of symptomatic VTE w/o prophylaxis of 4.3% (page 1); 

mates for LMWH of 9 per 1000 symptomatic DVT, nonfatal PE 3 per 

per 1000 and major bleeding of 7 per 1000 (page 2) are extremely useful 

usy clinician to know for counseling patients, comparing with treatment 

4) and for making treatment decisions. Would include these numbers in the 

n on page 9. 

We added data on the rate of VTE to the conclusion section. We did not repeat the absolute risk reductions as this information is already contained in two locations: in the bullet points and in the summary of evidence table. We will be sure that this information is contained in the VA e-brief.
53

ew Oral Anticoagulants for Thromboprophylaxis Evidence-based Synthesis Program

Comment Response

ion section: a). first paragraph, as noted above in #3 would put in 

crease risk numbers and b). Last paragraph...‘Based on current 

nticoagulants—particularly Xa inhibitors—are a reasonable option 

ylaxis...’ Agree from this evidence synthesis that Xa inhibitors are a 

 Any suggested sequence of treatment? LMWH first then Xa? Or is Xa 

able as LMWH? Is there a way to assess the value of 4 per 1000 decrease 

VT vs an increase of 2 major bleeds per 1000 treated with a Xa?

Although there are formal methods to consider multiple outcomes to develop a rank order of interventions, none of these methods are robust. The decision whether to use thromboprophylaxis, and the particular mode, is one that involves tradeoffs between potential benefits and harms. Clinicians must consider the patient’s particular risks, values, and preferences when making this decision. Our data inform this decision. 

aragraph: ‘Dabigatran etexilate is an oral reversible DTI...’ Reversible seems 

n antidote for reversal and there is no antidote (other than stopping the 

ting it wear off). Recommend striking word ‘reversible’.

The recommended change has been made.

 Strategy first paragraph: Might be more explicit as why a synthesis of high 

ould be more effective approach to summarizing the evidence than a perhaps 

approach of searching the literature for RCTs and combining those in an 

s. Also, why limit the search only as far back as 1 Jan 2009. What is the 

earch timeframe?

We added a justification for this approach as follows: “This approach is particularly useful when different intervention options or outcomes are evaluated in multiple recent reviews and when the audience is policymakers.”

pant Characteristics: Discussion regarding no Veterans studied in the trials 

ts were predominantly female 50-75%. Given the available evidence is there 

ason(s) to believe Veterans would respond differently to these treatments or 

cable to Veterans patients? If so why? If not why not?

The applicability of the results to Veteran patients is discussion in the Applicability section of the Discussion.

a Inhibitors compared to LMWH: a reader might assume that all of these 

e in the US. While we are considering all the individual drugs, would 

ncy if drugs not available in the US were identified.

This detail has been added. Only rivaroxaban is currently available in the United States.

aph 3, first sentence “In subgroup analysis, higher doses of Xa inhibitors, but 

r lower doses, ...’ would list the doses considered high, intermediate or low 

 on page 32.

The authors did not report the doses categorized as high, intermediate, or low. However, they do give the doses studied in the individual trials and we have added the dosing ranges for apixaban and rivaroxaban, drugs approved in Canada and the United States, respectively.

 Comparisons of Interest section, 2nd paragraph ‘Low molecular weight 

n K antagonists, 3rd line, would put in dose regimen of enoxaparin (logiparin 

S, so could omit dose). Same page, section immediately below ‘Oral FXa 

ctionated heparin would list doses of fondaparinux and unfractionated 

We added the dose of enoxaparin (30 mg subcutaneously every 12 hours). Fondaparinux and unfractionated heparin were evaluated in an observational study that did not report doses.
54

ew Oral Anticoagulants for Thromboprophylaxis Evidence-based Synthesis Program

Comment Response

raph under ‘Key Points’ discussion of SRs and quality notes the industry 

k meta-analysis (Cohen 2012, ref 36) was rated fair quality. Then notes “The 

ot provide an adequate description or quality assessment of the included 

est the assumption of a constant treatment effect across different study 

ssumption inherent to network meta-analysis.” Page 32 same meta-analysis 

 and notes ‘the composite outcomes are suspect because they combine events 

ny DVT, PE, death] and major bleeding[major, clinically relevant and minor 

e very different clinical importance’. These seem to be fatal deficiencies 

ir. Appendix D pg 54 notes the quality rating scale and Pg 55 notes the 

essments. Item 12 for the study in question (Cohen 2012, ref 36) is answered 

hich there is no provision in the scoring system which notes if item 12 is 

ould be rated good or fair and if ‘No’ then Poor. A poor rating would have 

 trial from the analysis. The summary Table D-1 lists no answers for 6 of the 

s including critical items 7 and 8 which the text notes that the review is likely 

ws. Should the rating really be ‘poor’ and this analysis excluded?

We re-reviewed our quality rating for the Cohen study. We agree that it is poor quality and have excluded it from the final report.

elines’ section, First paragraph, end o paragraph notes ACCP recommends 

e of bleeding risk. Does ACCP suggest any sequence if there is an elevated 

We have added the ACCP recommendation for patients at increased bleeding risk: “For patients with increased bleeding risk, ACCP recommends intermittent pneumatic compression device or no prophylaxis.”

icability’. End of paragraph notes private sector vs VA patients potential 

comment #7 above). What should the reader conclude? This is a solid 

s but not applicable to VA patients? Or is applicable to VA patients?

We revised this section to state that the results likely apply to Veteran populations. We add a specific caution about higher comorbidities in Veterans, increasing the risk of bleeding.

clusion’: Same comments as #4 above as this section also appears on page 9 

ummary.

As stated above, we think these data should be used to make individualized decisions with patients about the choice and mode of thromboprophylaxis.

ndix D, item 8, 2nd to last line ‘...If only qualitative analyses are completed, 

scribe...’ Change ‘show’ to ‘should’.

Thank you for noting this error. It has been corrected.

tastic descriptions of confidence interval and statistical significance! Thank you.

urprised to see the evidence that newer anticoagulants did not offer much 

an ease of administration and less monitoring but also troubled to see the 

effects

Acknowledged
55

ew Oral Anticoagulants for Thromboprophylaxis Evidence-based Synthesis Program

Comment Response

his review, but might note that earlier this month the FDA approved 

eatment of deep vein thrombosis or pulmonary embolism, and to reduce the 

VT and PE following initial treatment. These are 2 other important clinical 

The role of newer anticoagulants for treatment of DVT and PE was reviewed in an earlier VA ESP report.

linical performance measures, programs, quality improvement measures, patient care services, or conferences that will be 

 If so, please provide detail? 

e inpatient performance measure for DVT prophylaxis that could be 

nce measure Technical Manual would need to be checked

We will ask that this report be sent to the VA clinical guideline and performance measure groups.

f Thank you.

Thank you.

 recommendations on how this report can be revised to more directly address or assist implementation needs.

ed above. Thank you.

Thank you.

rom my perspective, that most VA providers are unaware of these reports, 

 take the time to read them. However, I find that they are a valuable 

ence tool. 

Thank you.

with contact details of any additional individuals/stakeholders who should be made aware of this report.

mD who is PBM contact for these medications. Lisa.longo@va.gov based 

Thank you; the report has been sent to Dr. Longo, and she is one of our stakeholders in the product.

Medicine; Chief Medical Officers; Chiefs of Staff Thank you.

Thank you.
56

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

APPENDIX F. Glossary

Abstract screening

The stage in a systematic review during which titles and abstracts of articles identified in the 

literature search are screened for inclusion or exclusion based on established criteria. Articles 

that pass the abstract screening stage are promoted to the full-text review stage.

Anticoagulant agents

A class of medication that prevents coagulation (blood clotting).

ClinicalTrials.gov

A registry and results database of federally and privately supported clinical trials conducted in 

the United States and around the world. ClinicalTrials.gov provides information about a trial’s 

purpose, location, and participant characteristics among other details. 

Cochrane Database of Systematic Reviews

A bibliographic database of peer-reviewed systematic reviews and protocols prepared by the 

Cochrane Review Groups in The Cochrane Collaboration. 

Companion article

A publication from a trial that is not the article containing the main results of that trial. It may be 

a methods paper, a report of subgroup analyses, a report of combined analyses, or other auxiliary 

topic that adds information to the interpretation of the main publication.

Confidence interval (CI)

The range in which a particular result (such as a laboratory test) is likely to occur for everyone 

who has a disease. “Likely” usually means 95 percent of the time. Clinical research studies are 

conducted on only a certain number of people with a disease rather than all the people who have 

the disease. The study’s results are true for the people who were in the study but not necessarily 

for everyone who has the disease. The CI is a statistical estimate of how much the study findings 

would vary if other different people participated in the study. A CI is defined by two numbers, 

one lower than the result found in the study and the other higher than the study’s result. The size 

of the CI is the difference between these two numbers.

Data abstraction

The stage of a systematic review that involves a pair of trained researchers extracting reported 

findings specific to the research questions from the full-text articles that met the established 

inclusion criteria. These data form the basis of the evidence synthesis. 

Deep vein thrombosis (DVT)

A blood clot that develops in the deep veins of the legs.

Direct thrombin inhibitors (DTIs)

A new class of anticoagulants that bind directly to thrombin and block its interaction with its 

substrates.

DistillerSR

An online application designed specifically for the screening and data extraction phases of a 

systematic review.
57

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

Embase

The Excerpta Medica database (EMBASE) produced by Elsevier, a major biomedical and 

pharmaceutical database indexing over 3500 international journals in the following fields: 

drug research, pharmacology, pharmaceutics, toxicology, clinical and experimental human 

medicine, health policy and management, public health, occupational health, environmental 

health, drug dependence and abuse, psychiatry, forensic medicine, and biomedical engineering 

or instrumentation. There is selective coverage for nursing, dentistry, veterinary medicine, 

psychology, and alternative medicine.

Exclusion criteria

The criteria, or standards, set out before a study or review. Exclusion criteria are used to 

determine whether a person should participate in a research study or whether an individual 

study should be excluded in a systematic review. Exclusion criteria may include age, previous 

treatments, and other medical conditions. 

Factor Xa (FXa) inhibitor

A new class of anticoagulants that bind directly to factor Xa and block its interaction with other 

substrates. 

Full-text review

The stage of a systematic review in which a pair of trained researches evaluates the full-text of 

study articles for potential inclusion in the review.

GRADE

Grading of Recommendations Assessment, Development, and Evaluation (GRADE), a system of 

assessing the quality of medical evidence and evaluating the strength of recommendations based 

on the evidence.

Inclusion criteria

The criteria, or standards, set out before the systematic review. Inclusion criteria are used 

to determine whether an individual study can be included in a systematic review. Inclusion 

criteria may include population, study design, sex, age, type of disease being treated, previous 

treatments, and other medical conditions. 

Low molecular weight heparin

A class of medication used to treat thrombosis or for prophylaxis in situations that lead to a 

high risk of thrombosis. These medications have a more predictable anticoagulant response than 

naturally occurring unfractionated heparin.

Optimal information size

The number of patients that need to be included in a pooled analysis (meta-analysis) to provide 

sufficient power to detect the smallest clinically important difference in treatment effect.

PRISMA

Preferred Reporting Items for Systematic Reviews and Meta-Analyses, an evidence-based 

minimum set of items for reporting in systematic reviews and meta-analyses. 
58

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

Publication bias

The tendency of researchers to publish experimental findings that have a positive result, 

while not publishing the findings when the results are negative or inconclusive. The effect of 

publication bias is that published studies may be misleading. When information that differs 

from that of the published study is not known, people are able to draw conclusions using only 

information from the published studies.

PubMed®

A database of citations for biomedical literature from MEDLINE®, life science journals, and 

online books in the fields of medicine, nursing, dentistry, veterinary medicine, the health care 

system, and preclinical sciences.

Pulmonary embolism (PE)

Blocking of the pulmonary artery (lungs) or one of its branches by a clot.

Randomized controlled trial

A prospective, analytical, experimental study using primary data generated in the clinical 

environment. Individuals similar at the beginning of the trial are randomly allocated to two or 

more treatment groups and the outcomes the groups are compared after sufficient followup time. 

Properly executed, the RCT is the strongest evidence of the clinical efficacy of preventive and 

therapeutic procedures in the clinical setting. 

Risk

A way of expressing the chance that something will happen. It is a measure of the association 

between exposure to something and what happens (the outcome). Risk is the same as probability, 

but it usually is used to describe the probability of an adverse event. It is the rate of events (such 

as breast cancer) in the total population of people who could have the event (such as women of a 

certain age). 

Statistical significance

A mathematical technique to measure whether the results of a study are likely to be true. 

Statistical significance is calculated as the probability that an effect observed in a research study 

is occurring because of chance. Statistical significance is usually expressed as a P-value. The 

smaller the P-value, the less likely it is that the results are due to chance (and more likely that 

the results are true). Researchers generally believe the results are probably true if the statistical 

significance is a P-value less than 0.05 (p<.05).

Strength of evidence (SOE)

A measure of how confident reviewers are about decisions that may be made based on a body 

of evidence. SOE is evaluated using one of four grades: (1) High confidence that the evidence 

reflects the true effect; further research is very unlikely to change reviewer confidence in the 

estimate of effect; (2) moderate confidence that the evidence reflects the true effect; further 

research may change the confidence in the estimate of effect and may change the estimate; (3) 

low confidence that the evidence reflects the true effect; further research is likely to change the 

confidence in the estimate of effect and is likely to change the estimate; and (4) insufficient; the 

evidence either is unavailable or does not permit a conclusion.
59

Comparative Effectiveness of New Oral Anticoagulants

for Thromboprophylaxis Evidence-based Synthesis Program

Systematic review

A summary of the clinical literature. A systematic review is a critical assessment and evaluation 

of all research studies that address a particular clinical issue. The researchers use an organized 

method of locating, assembling, and evaluating a body of literature on a particular topic using a 

set of specific criteria. A systematic review typically includes a description of the findings of the 

collection of research studies. The systematic review may also include a quantitative pooling of 

data, called a meta-analysis.

Venous thromboembolism (VTE)

Obstruction of a vein or veins (embolism) by a blood clot (thrombus) in the blood stream; 

includes deep vein thrombosis (DVT) and pulmonary embolism (PE).

Vitamin K antagonist (warfarin)

An anticoagulant that acts by inhibiting the synthesis of vitamin K-dependent coagulation 

factors; i.e., I, VII, IX and X.