

Department of Veterans Affairs

Health Services Research & Development Service Evidence-based Synthesis Program

Comparative 

Effectiveness of Proton 

Irradiation Treatment

January 2015

Prepared for:

Department of Veterans Affairs

Veterans Health Administration

Quality Enhancement Research Initiative

Health Services Research & Development Service

Washington, DC 20420

Prepared by:

Evidence-based Synthesis Program (ESP)

Coordinating Center

Portland VA Medical Center

Portland, OR

Mark Helfand, MD, MPH, MS, Director 

Investigators:

Kim Peterson, MS

Ellen McCleery, MPH

Kallie Waldrip, MS
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

PREFACE

Quality Enhancement Research Initiative’s (QUERI) Evidence-based Synthesis Program (ESP) 

was established to provide timely and accurate syntheses of targeted healthcare topics of 

particular importance to Veterans Affairs (VA) clinicians, managers and policymakers as they 

work to improve the health and healthcare of Veterans. The ESP disseminates these reports 

throughout the VA, and some evidence syntheses inform the clinical guidelines of large 

professional organizations. 

QUERI provides funding for four ESP Centers and each Center has an active university 

affiliation. The ESP Centers generate evidence syntheses on important clinical practice topics, 

and these reports help: 

• develop clinical policies informed by evidence;

• guide the implementation of effective services to improve patient

outcomes and to support VA clinical practice guidelines and performance

measures; and

• set the direction for future research to address gaps in clinical knowledge.

In 2009, the ESP Coordinating Center was created to expand the capacity of HSR&D Central 

Office and the four ESP sites by developing and maintaining program processes. In addition, the 

Center established a Steering Committee comprised of QUERI field-based investigators, VA 

Patient Care Services, Office of Quality and Performance, and Veterans Integrated Service 

Networks (VISN) Clinical Management Officers. The Steering Committee provides program 

oversight, guides strategic planning, coordinates dissemination activities, and develops 

collaborations with VA leadership to identify new ESP topics of importance to Veterans and the 

VA healthcare system.

Comments on this evidence report are welcome and can be sent to Nicole Floyd, ESP 

Coordinating Center Program Manager, at Nicole.Floyd@va.gov. 

Recommended citation: Peterson K, McCleery E, Waldrip K, Helfand M. Comparative 

effectiveness of proton irradiation treatment. VA ESP Project #09-199; 2015. 

This report is based on research conducted by the Evidence-based Synthesis Program (ESP) 

Coordinating Center located at the VA Portland Health Care System, Portland, OR funded by the 

Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, 

Quality Enhancement Research Initiative. The findings and conclusions in this document are those of the 

author(s) who are responsible for its contents; the findings and conclusions do not necessarily represent 

the views of the Department of Veterans Affairs or the United States government. Therefore, no 

statement in this article should be construed as an official position of the Department of Veterans Affairs. 

No investigators have any affiliations or financial involvement (eg, employment, consultancies, honoraria, 

stock ownership or options, expert testimony, grants or patents received or pending, or royalties) that 

conflict with material presented in the report.

i
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

TABLE OF CONTENTS

Executive Summary ........................................................................................................................ 1

Abbreviations List ........................................................................................................................... 8

Introduction ................................................................................................................................... 10

Background ................................................................................................................................... 10

Methods......................................................................................................................................... 13

Topic Development ................................................................................................................... 13

Search Strategy ......................................................................................................................... 13

Study Selection ......................................................................................................................... 14

Data Abstraction ....................................................................................................................... 14

Quality Assessment ................................................................................................................... 14

 ................................................................................................... 14

Data Synthesis ........................................................................................................................... 15

Peer Review .............................................................................................................................. 15

Results ........................................................................................................................................... 16

Overview of Primary Studies and Systematic Reviews ............................................................ 17

Overview of Previous Systematic Reviews .......................................................................... 20

ICER Review ........................................................................................................................ 24

Key Questions 1, 2, and 4:

What is the effectiveness of proton beam irradiation compared to conventional x-ray-based 

external beam modalities? 

What is the effectiveness of proton beam irradiation compared to state-of-the-art therapies? 

What are the short- and long-term harms of proton beam irradiation compared to conventional 

x-ray-based external beam modalities and state-of-the-art therapies? ...................................... 28

Breast Cancer.......................................................................................................................... 28

 CNS Cancers........................................................................................................................... 29

 Esophageal Cancer............................................................................................................

 Giant Cell Tumor of the Bone ................................................................................................ 32

 Head and Neck Cancers.......................................................................................................... 33

 Uveal Hemangioma ................................................................................................................ 34

 Liver Cancer............................................................................................................................ 35

 Lung Cancer ............................................................................................................................ 35

 Meningioma ............................................................................................................................ 38

 Prostate Cancer ....................................................................................................................... 40

PBT versus IMRT............................................................................................................. 41

...... 31

Rating The Body of Evidence

ii
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

PBT versus 3D-CRT ......................................................................................................... 44

45

46

49

50

52

54

56

60

67

74

77

80

81

93

98

PBT versus brachytherapy ................................................................................................ 

PBT+photon versus photon alone ..................................................................................... 

 Secondary Malignancies in Various Cancers ......................................................................... 

Key Question 3: In patients with local recurrences after irradiation, what is the effectiveness of 

proton beam irradiation compared to conventional x-ray-based external beam modalities and 

state-of-the-art therapies? .......................................................................................................... 

Key Question 4A: What are the harms of proton beam irradiation compared to photon-based 

therapies in treating mobile targets that may move during treatment? ..................................... 

Summary of Evidence by Key Question ................................................................................... 

Discussion ..................................................................................................................................... 

References ..................................................................................................................................... 

Appendix A. Proton Beam Therapy Coverage Policies................................................................ 

Appendix B. Clinical Practice Guidelines In Other Organizations .............................................. 

Appendix C. Search Strategies ..................................................................................................... 

Appendix D. ICER Comparative Studies Excluded At Full-Text Level ...................................... 

Appendix E. Quality Assessment of Primary Studies .................................................................. 

Appendix F. Ongoing Clinical Trials............................................................................................ 

Appendix G. Quality Assessment of Systematic Reviews ........................................................... 

Appendix H. Data Abstraction Of Comparative Studies Not Included In The ICER Review ... 101

Appendix I. Peer Review Comment Disposition Table .............................................................. 104

FIGURES AND TABLES

Figure 1. Literature Flow Chart .................................................................................................... 17

Table 1. Relevant Comparative Studies ........................................................................................ 19

Table 2. Previous Systematic Reviews on the Effectiveness of Proton Beam Therapy for Cancer

....................................................................................................................................................... 21

Table 3. Comparison of ICER and ESP Findings by Cancer Type .............................................. 25

Table 4. Strength of Evidence (SOE) for Comparative Benefits and Harms of PBT or Photons or 

Mixed Photons/Electrons in Partial Breast Treatment for Early-stage Breast Cancer ................. 29

Table 5. SOE for Proton Craniospinal Irradiation (CSI) versus Conventional Photon CSI for 

Medulloblastoma........................................................................................................................... 30

Table 6. SOE for PBT versus IMRT for Spinal Cord Gliomas .................................................... 30

Table 7. SOE for PBT versus IMRT and 3D-CRT in Esophageal Cancer ................................... 31

Table 8. SOE for Combined PBT+Photons versus Photons Alone for Giant Cell Tumor of Bone

....................................................................................................................................................... 32

iii
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Table 9. SOE for Comparative Benefits of PBT or IMRT for Paranasal Sinus and Nasal Cavity 

Malignancies ................................................................................................................................. 34

Table 10. SOE for PBT versus Photons for Uveal Hemangiomas ............................................... 35

Table 11. Findings and SOE for PBT versus SBRT and CRT in Early-stage NSCLC ................ 36

Table 12. Findings and SOE for PBT versus IMRT and 3D-CRT in Locally-advanced NSCLC 38

Table 13. SOE for Comparative Benefits of PBT or Conventional Photon Therapy for Optic 

Nerve Sheath Meningioma ........................................................................................................... 39

Table 14. SOE for Comparative Benefits of Combined PBT and Photon Therapy or Photon 

Therapy Alone for Meningioma ................................................................................................... 40

Table 15. Conclusions of Comparative Studies of PBT versus IMRT, 3D-CRT, Brachytherapy, 

and Conventional Photon Therapy in Prostate Cancer ................................................................. 40

Table 16. SOE for Comparative Benefits and Harms of PBT versus IMRT in Prostate Cancer.. 43

Table 17. SOE for PBT versus 3D-CRT in Prostate Cancer ........................................................ 45

Table 18. SOE for Comparative Benefits of Conformal Photon Radiation and PBT or 

Brachytherapy for Prostate Cancer ............................................................................................... 46

Table 19. SOE for Benefits and Harms of Higher Dose PBT+Photon versus Lower Dose 

Conventional Photon in Prostate Cancer ...................................................................................... 48

Table 20. Findings and SOE for PBT versus Photon and Secondary Malignancy in Various 

Cancers .......................................................................................................................................... 50

Table 21. SOE for Comparative Benefits of PBT, SRT, or Conventional Photon Therapy for 

Recurrent Malignant Brain Tumors .............................................................................................. 51

Table 22. SOE for Comparative Benefits of PBT or X-rays for Recurrent Hepatocellular 

Carcinoma ..................................................................................................................................... 51

Table 23. Summary of Evidence by Key Question....................................................................... 54

iv
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

EXECUTIVE SUMMARY

BACKGROUND

Maximizing target tumor dose while minimizing healthy tissue damage continues to be a 

challenge in radiation therapy. Because of its appealing dosimetric characteristics, proton beam 

therapy (PBT) has held the clinical promise of allowing for higher doses of radiation to be 

delivered more safely, especially for ocular, skull base, and spinal tumors that require 

exceptional precision. But the role of protons is less clear for more common tumors, like 

prostate, where their dosimetric advantages may be diminished and for which intensitymodulated radiation therapy (IMRT) can now safely deliver optimally high radiation doses. 

To help consider the increased number of offers from University Affiliates to provide contracted 

off-site proton irradiation therapy, the VA Radiation Oncology Program requested that the 

Evidence-based Synthesis Program Coordinating Center (ESP CC) synthesize the most recent 

literature on the comparative effectiveness of PBT in various cancers. This report of that 

synthesis focuses on the following questions: 

How does PBT compare with conventional X-ray-based external beam treatments and state-ofthe-art therapies with regard to benefits and harms for both new patients and those who have 

locally recurrent tumors after irradiation?

How do the comparative effects of proton and photon beam therapies differ according to 

variation in tumor motion?

METHODS

Our research librarian searched MEDLINE® (via PubMed®), the Cochrane Clinical Register of 

Controlled Trials, and ClinicalTrials.gov using the terms “proton beam” and “cancer” to identify 

articles relevant to the key questions. We also hand-searched reference lists, consulted experts, 

requested information from proton therapy system manufacturers and centers, and searched 

ClinicalTrials.gov.

We selected studies that compared benefits (survival, quality of life, functional capacity, local 

tumor control, delivery of planned chemotherapy and radiation regimens) and harms (toxicity 

and secondary malignancies) for PBT versus other modalities in adults with any cancer type 

(except ocular). We used standardized methods to assess internal validity and the overall strength 

of evidence for each outcome. 

RESULTS

We reviewed a total of 2,774 citations. From these, we included 25 relevant primary comparative 

studies and 6 systematic reviews. Requests to manufacturers and proton beam facilities did not 

identify any additional published or unpublished comparative studies, but only 6 manufacturers 

affirmed they weren’t aware of any additional studies.

Most existing systematic reviews are outdated, as their literature searches were conducted prior 

to the publication of most of the comparative studies. The 2014 review produced by the Institute 

1
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

for Clinical and Economic Review (ICER)1 for the Washington State Health Care Authority 

Health Technology Assessment Program (HTA) included 22 of the 25 comparative studies. The 

ICER review was useful for its accurate data abstraction, but we couldn’t rely on its conclusions. 

COMPARATIVE EFFECTIVENESS OF PBT (KEY QUESTIONS 1, 2, & 4)

Breast

There is low-strength evidence of comparable 7-year cumulative local recurrence for single field 

PBT versus photon-based 3D conformal accelerated partial-breast irradiation for patients with 

stage I breast cancer, but various 7-year skin toxicities were more common in the proton therapy 

group (range, increased from 15-28% to 54-90%). However, there was no difference in patients’ 

ratings of good or excellent for 7-year overall cosmetic outcomes or in local failure rates. This 

evidence came from one fair-quality prospective trial of 98 patients with stage I breast cancer 

treated between October 2003 and April 2006 at Massachusetts General Hospital.2

Esophageal

There is low-strength evidence that, when used in trimodal therapy (neoadjuvant chemoradiation 

followed by surgical resection), IMRT and proton beam have comparable risk of postoperative 

pulmonary complications (OR: 2.23; 95% CI: 0.86-5.75) and GI complications (OR: 1.02; 95% 

CI: 0.47-2.25), 3-dimensional conformal radiation therapy (3D-CRT) and proton beam have 

comparable risk of GI complications (OR: 2.31; 95% CI: 0.69-7.74; 28.4% vs 18.1), but that 3DCRT has a higher risk than proton beam of pulmonary complications (OR: 9.13; 95% CI: 1.83-

45.42; 30.3% vs 13.9%). When given alone, there is low-strength evidence that proton therapy is 

associated with a higher risk of acute pneumonitis compared with IMRT/3D-CRT (33% vs 15%; 

P=.04). 

Medulloblastoma

There is low-strength evidence that PBT 54.6 GyE and photon therapy 52.9 Gy have comparable 

2-year overall and progression-free survival, proportion of patients with treatment breaks, and 

locoregional failure, but some 1-month toxicities were less common in the proton beam therapy 

group, including medical management of esophagitis (5% vs 57%; P<.001), > 5% weight loss 

(16% vs 64%; P=.004), and Grade ≥ 2 nausea/vomiting (26% vs 71%; P=.004). This evidence 

came from a fair-quality retrospective cohort study of 40 adults with medulloblastoma treated at 

the MD Anderson Cancer Center from 2003 to 2011.3

Non-small cell lung cancer

In patients with locally-advanced NSCLC, there is low-strength evidence that, even at a higher 

dose (74 Gy), acute risk of severe esophagitis (grade ≥ 3) at 6 months for PBT is similar to 3DCRT 63 Gy, but lower than with IMRT 63 Gy (6% vs 28%; P<.0001).4 This evidence came from 

one cohort study of 652 patients with NSCLC, mostly clinical stage IIIA-B and mean age of 66 

years, who were treated at MD Anderson Cancer Center between 2000 and 2008. Evidence on 

survival and 15-17 month toxicity in a subgroup of those patients given concurrent 

chemotherapy was insufficient to draw conclusions. There is also insufficient evidence to draw 

conclusions about proton-based stereotactic ablative therapy for early-stage lung cancer

compared with photon-based stereotactic ablative therapy.

2
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Prostate

Table 1 summarizes the numbers and types of comparative studies and our conclusions for each 

different comparison of PBT to IMRT, 3D-CRT, brachytherapy, and conventional photon 

therapy, respectively. 

Table 1. Conclusions for Comparative Studies in Prostate Cancer

Comparator

Benefits (Strength of Evidence 

Grade)

Harms (Strength of Evidence 

Grade)

PBT vs IMRT Similar Quality of Life (QOL) (low 

SOE): 2 historically-controlled 

cohorts (N=1695):5,6

Transiently lower GU toxicity at 0-

6 months for PBT (low SOE),7 but 

similar GI and GU toxicity at 12-24 

months (low to moderate),7,8 and 

increased GI toxicity with PBT at 4-

5 years (low SOE): 4 retrospective 

cohorts 8-10 N=34,185

PBT vs 3D-CRT Similar QOL (insufficient SOE), but 

survival vs 3D-CRT remains 

unknown: 1 historically-controlled 

cohort5

; N=218 

Increased acute GI toxicity with 

PBT (low SOE): 1 retrospective 

cohort9

; N=NR

PBT vs 

brachytherapy 

Similar 8-yr survival and distant 

metastasis (low SOE): 1 historicallycontrolled cohort11; N=282 

No evidence

PBT+photon vs 

photon alone

Similar overall 5-8 year survival and 

QOL (low SOE): (1 RCT, 2 cohort 

studies; N=567) 12-14

Increased 8-year rectal bleeding and 

urethral stricture (low SOE): 1 

RCT14; N=202

Spinal cord glioma

There is low-strength evidence that use of PBT may be disadvantageous for highly infiltrative 

tumors such as intermedullary spinal cord gliomas, as demonstrated by a reduced chance of 5-

year overall survival (photon vs proton, aHR: 55.82; 95% CI: 1.34-2316.8).15 This conclusion is 

based on one retrospective cohort study of 32 patients treated for intramedullary gliomas at 

Massachusetts General Hospital with either PBT (N=10) or IMRT (N=22) at an average dose of 

51 Gy in 1.8 median daily fractions over 29 treatments. 

Mixed cancer types – secondary malignancies

There is insufficient evidence to draw conclusions about how PBT compares to other radiation 

modalities in the risk of secondary malignancy. While secondary cancer risk for PBT patients 

was half that of photon patients in a retrospective cohort of patients with a variety of nonmetastatic cancers, this study had numerous methodological limitations.

16

Other cancer types

Although we found comparative studies in giant cell tumors of the bone, head and neck cancer, 

uveal hemangiomas, and meningiomas, they provided insufficient evidence for drawing 

conclusions. 

3
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

KEY QUESTION 3: PATIENTS WITH LOCAL RECURRENCES 

There is insufficient evidence to draw conclusions about the comparative effects of PBT versus 

other radiation modalities among patients with recurrent tumors. We identified 2 comparative 

studies on recurrent tumors, one among patients with recurrent malignant brain tumors17 and one 

among patients with recurrent liver cancer,18 but both studies were rated poor quality due to their 

failure to account for potentially important confounding. 

KEY QUESTION 4A: EFFECTS OF TUMOR MOTION VARIABILITY

There is insufficient evidence to determine how the comparative effects of proton and photon 

beam therapies differ according to variation in tumor motion. Although dosimetric studies 

comparing methods of accounting for respiratory motion in treatment planning report that 4-

dimensional computed tomography (4DCT) imaging decreases doses to normal structures 

compared with other multiphase,19 free-breathing,20 or 3-dimensional computed tomography 

(3DCT) imaging,21 how this translates to clinical outcomes is not clear. We did not identify any 

studies that evaluated clinical outcomes of interest based on variability in tumor motion, imaging 

and planning methods used to account for respiratory motion, or quality assurance standards. 

DISCUSSION 

For the cancer sites and types reviewed here, there are no reliable data from long-term 

randomized trials on survival, quality of life, or functional capacity of patients who underwent 

PBT compared with any other modality. We could not fully assess the overall net health benefit 

of proton beam therapy versus its comparators because comparative observational studies did not 

consistently report many outcomes of greatest interest. Comparative risk of secondary 

malignancies was only evaluated by one poor-quality retrospective cohort study of nonmetastatic cancer patients treated with PBT or photon modalities for a variety of cancers.

16

Ability to deliver planned treatments was only reported by one small retrospective cohort study.3

No studies reported functional capacity outcomes or overall severe late toxicity. Table 2 

summarizes the key findings on comparative benefits and harms that are supported by lowstrength evidence. Although we found comparative studies in giant cell tumors, head and neck 

cancer, uveal hemangiomas, and meningiomas, they provided insufficient evidence for drawing 

conclusions. There is insufficient evidence to draw conclusions about the comparative effects of 

PBT versus other radiation modalities among patients with recurrent tumors or how the 

comparative effects of proton and photon beam therapies differ according to variation in tumor 

motion. 

4
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Table 2. Key Findings Supported by Low-strength Evidence

Cancer 

Type Results (Low Strength of Evidence) Studies*

Breast 

N/A one fair 

prospective 

study 

(N=98)

Long-term self-reported cosmetic outcomes and local failure rate 

Increased skin toxicity for accelerated partial breast irradiation 

when delivered in single-field: 54-90% (PBT) vs 15-28% (3DCRT)

Esophageal

Lower 30-day post-op pulmonary complications when given as 

trimodal therapy (OR: 9.13 (3D-CRT vs PBT)) 

2 fair 

studies

(N=519)

When given as trimodal therapy, IMRT and proton beam have 

similar risk of postoperative pulmonary complications (OR: 2.23; 

95% CI: 0.86-5.75) and GI complications (OR: 1.02; 95% CI: 

0.47-2.25) and 3D-CRT and proton beam have comparable risk 

of GI complications (OR: 2.31; 95% CI: 0.69-7.74),

Acute Pneumonitis: 33% (PBT) vs 15% (IMRT/3D-CRT), P=.04 

Medulloblastoma 

1-month medical management of esophagitis: 5% (PBT) vs 57% 

(photon), P<.001; > 5% weight loss: 16% (PBT) vs 64% 

(photon), P=.004; Grade ≥ 2 nausea/vomiting: 26% (PBT) vs 

71% (photon), P=.004 one fair study 

(N=40) 2-year overall and progression-free survival, proportion of 

patients with treatment breaks, and locoregional failure

N/A

NSCLC

6-month severe esophagitis (grade ≥ 3) in locally advanced 

NSCLC: 6% (PBT) vs 28% (IMRT), P<.0001

one fair study 

(N=652) 

6-month severe esophagitis (grade ≥ 3) in locally advanced 

NSCLC: 6% (PBT) vs 8% (3D-CRT), P=.42

N/A

Prostate

GU toxicity at 0-6 months: PBT=5.9% vs IMRT=9.5%; OR 

(PBT vs IMRT): 0.60; 95% CI: 0.38-0.96

vs IMRT: 5 fairpoor (N=8987)

vs 3D-CRT: 2 

fair-poor 

(N=19281) 

vs 

brachytherapy: 1 

fair (N=282) 

PBT+photon vs 

photon alone: 3 

fair-poor, 2 

prospective 

(N=567) 

Urinary incontinence: 3.3 per 100 person-years (PBT) vs 3.7 per 

100 person-years (3D-CRT)

Erectile dysfunction: 7.4 per 100 person-years (PBT) vs 5.3 per 

100 person-years (3D-CRT)

Similar 2-year QOL vs IMRT/3D-CRT

Similar 8-year overall survival and freedom from distant 

metastasis vs brachytherapy

Similar 8-year survival, QOL, urethral stricture, gross hematuria 

(PBT+photon vs photon alone) 

Similar 6-month GI toxicity and 12-24 month GI/GU toxicity vs 

IMRT

5
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Cancer 

Type Results (Low Strength of Evidence) Studies*

Prostate 

(continued) 

Any GI toxicity after 5 years: 20.1 per 1,000 patient-years (PBT) 

vs 8.3 per 1,000 patient-years (IMRT); HR (PBT vs IMRT): 3.32 

(95% CI: 2.12-5.20); GI toxicity at 46-50 months: procedures: 

18% (IMRT) vs 21% (PBT); RR (IMRT vs PBT): 0.82 (0.70-

0.97); diagnoses: 12% (IMRT) vs 18% (PBT); RR (IMRT vs 

PBT): 0.66 (95% CI: 0.55-0.79) 

Increased 1-year GI toxicity: aHR (PBT vs 3D-CRT): 2.13 (95% 

CI: 1.45-3.13) 

8-year rectal bleeding: 32% (PBT+photon) vs 12% (photon 

alone); P=.002

Spinal cord 

glioma

N/A

one cohort study 

(N=32) 

N/A

5-year survival: aHR (photon vs proton): 55.82; 95% CI: 1.34-

2316.8

= PBT advantage; = comparable outcome; = PBT increased harm; *retrospective unless 

otherwise noted. 

The main limitations of the comparative studies were that they (1) were retrospective and some 

used historical control groups for the photon-based comparator groups, (2) potentially gave 

proton beam groups an unfair advantage by comparing them to photon-based groups with poorer 

prognostic profiles without accounting for the important differences, (3) lacked data on radiation 

dose and field size, (4) could not reliably differentiate toxicity grade, (5) did not measure many 

outcomes of greatest interest, including recurrence, ability to deliver planned chemotherapy and 

radiation regimens, functional capacity, overall severe late toxicity, and secondary malignancies, 

and (6) may have limited applicability to current practices and across all 14 available facilities, 

since they mainly came from only 3 proton facilities and involved patients treated between 1991 

and 2003. The biggest gaps in evidence include the lack of assessment of many important 

outcomes including recurrence, ability to deliver planned chemotherapy and radiation regimens, 

functional capacity, overall severe late toxicity, and secondary malignancies. Also, although 

there are many studies that have compared the dosimetric impact between different methods of 

accounting for tumor motion in treatment planning, including 4DCT imaging, multiphase, freebreathing, or 3DCT imaging, we found no studies of how they compare in clinical outcomes. 

CONCLUSION

Despite the common claim that the advantage of proton beam therapy is self-evident, 

comparative studies have not demonstrated any common clinical situations in which proton beam 

therapy has an important clinical advantage over photon radiotherapy modalities on meaningful 

long-term health outcomes, but have uncovered low-strength evidence of the potential for 

increased late toxicity compared with IMRT and 3D-CRT for breast, esophageal, prostate, and 

spinal cord glioma cancers. Existing comparative studies have numerous methodological 

deficiencies that limited our confidence in their findings, and their findings may have limited 

applicability across all US proton beam facilities. Although numerous randomized controlled 

trials are underway that carry the promise of improved toxicity measurement, it is unclear 

6
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

whether they will fully address gaps in evidence on other important outcomes including 

recurrence, ability to deliver planned chemotherapy and radiation regimens, functional capacity, 

overall severe late toxicity, and secondary malignancies. Because this is still a rapidly evolving 

field, with ongoing efforts to improve techniques and reduce costs, this review may need 

frequent updating to keep up-to-date with emerging research.

7
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

ABBREVIATIONS LIST

3D-CRT

3DCT

4DCT

3-dimensional conformal radiation therapy

3-dimensional computed tomography 

4-dimensional computed tomography

ACR American College of Radiology

ADT

aHR

AHRQ

APBI

aOR

AMSTAR

ASTRO

BCBS

CER

CI

Androgen deprivation therapy

Adjusted hazard ratio 

Agency for Health Research and Quality

Accelerated partial breast irradiation

Adjusted odds ratio 

A Measurement Tool to Assess Systematic Reviews

The American Society for Radiation Oncology

Blue Cross Blue Shield

Comparative effectiveness research

Confidence interval

CGE Cobalt gray equivalent

CNS Central nervous system

CRT Conventional radiotherapy

CSI Craniospinal irradiation

CT Computed tomography

CTAF

ECRI

California Technology Assessment Forum

Emergency Care Research Institute

EPIC Expanded Prostate Cancer Index Composite

ESP CC Evidence-based Synthesis Program Coordinating Center

FDA Food and Drug Administration

FDG Fluorodeoxyglucose

GI

GRADE

Gastrointestinal

Grading of Recommendations Assessment, Development 

and Evaluation

GU Genitourinary 

Gy Gray

GyE

HR

HTA

Gray equivalent

Hazard ratio

Health Technology Assessment Program

ICER Institute for Clinical and Economic Review

IMRT Intensity-modulated radiation therapy 

NICE

NSD

NR

National Institute for Health and Care Excellence

No significant difference 

Not reported

NRC Nuclear Regulatory Commission

NSCLC

OR

Non-small cell lung cancer

Odds ratio

PBT Proton Beam Therapy

PCSI Prostate Cancer Symptom Indices

8
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

PET Positron emission tomography

PROST-QA Prostate Cancer Outcomes and Satisfaction with Treatment 

Quality Assessment

PSA Prostate-specific antigen

QOL

QUERI

Quality of life

Quality Enhancement Research Initiative

RBE

RCT

RR

Relative biological effectiveness

Randomized control trial 

Relative risk

SBRT Stereotactic body radiotherapy

SEER Surveillance Epidemiology and End Results

SOE Strength of evidence

TEC

UA

Technology Evaluation Center

University Affiliates

VA TAP Veterans Affairs Technology Assessment Program

VHA Veterans Health Administration

9
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

EVIDENCE REPORT

INTRODUCTION 

The Veterans Health Administration’s (VHA) current policy on proton therapy is that it “will not 

be deemed appropriate as routine therapy for organ-confined adenocarcinoma of the prostate.” 

According to a May 2012 Department of Veterans Affairs (VA) Memorandum on the 

appropriate use of proton beam therapy for radiation therapy treatment, the reason for this is that 

although “it has become clear that proton therapy improves the safety of radiation delivery to 

tumors requiring exceptionally precise treatment,” such as ocular melanomas, sacral and base of 

skull chordomas, and para-spinal primitive neuroectodermal tumors, “less certain are the 

advantages of proton therapy for tumors which can be treated successfully with photon therapy 

or other treatment modalities, such as by resection.” However, the VHA continues to evaluate 

the possibility of contracting with University Affiliates (UA) to provide proton irradiation as part 

of off-site radiation therapy care. To help inform their decision to support or not support a proton 

therapy center for Veterans, the VHA is interested in whether any evidence has emerged that 

identifies any additional tumor sites and presentations as candidates for improved outcomes 

following proton irradiation. Therefore, the VHA National Radiation Oncology Program Office 

(10P4H) requested that the VA Evidence-based Synthesis Program Coordinating Center (ESP

CC) conduct an evidence review to examine the benefits and harms of proton irradiation 

treatment compared to other available conventional and state-of-the-art radiation treatment 

modalities for a variety of cancer types.

BACKGROUND

Radiation therapy harms both malignant and healthy tissues and higher absorbed radiation dose 

increases cellular damage. Administering sufficient radiation to the target tumor while sparing 

adjacent healthy tissue continues to be a challenge in radiation therapy. For external beam 

radiotherapy, maximizing the dose to the target tumor and minimizing healthy tissue damage is 

achieved by (1) delivering beams of radiation via paths that spare critical and vulnerable tissues, 

(2) using multiple beams that intersect at the tumor site, avoiding undue exposure in other tissue, 

and (3) delivering radiation in smaller doses over successive sessions allowing healthy tissue to 

recover between treatments.22 The most common radiotherapy options use these methods to 

varying degrees.

Several forms of radiotherapy modalities are available, which vary in the type of beams used and 

their delivery methods. Brachytherapy is a form of internal radiation that involves implanting 

encapsulated radioactive sources within or adjacent to tumors. These sources deposit beta 

radiation or alpha particles in immediately neighboring tissue and so deliver very little radiation 

to healthy tissues. However, brachytherapy has very specific indications and is not used for many 

cancers.23Among the many types of external radiotherapy modalities available, photon beams are 

still the most widely used, including 3-dimensional conformal radiation therapy (3D-CRT), 

intensity-modulated radiation therapy (IMRT), and tomotherapy. Since the peak dose of 

radiation from a photon beam is deposited near the entrance to tissue, different methods of 

delivering photon beams have been developed to maximize the dose to the target tumor and 

minimize the dose to healthy tissues. IMRT is one of the most advanced methods of delivering 

10
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

high-dose radiation with photons. IMRT reduces radiation to normal tissue by splitting the 

overall dose across multiple fields of small volumes coming from different directions that are 

custom tailored to each unique tumor shape. Other advances in IMRT include image-guided 

radiation therapy, which may improve the precision of dose delivery, and volumetric modulated 

arc therapy, which reduces the time needed to deliver each treatment. 

Less widely used are heavier particle beams, such as protons and carbon ions. Currently only 

proton beam therapy (PBT) has been cleared by the US Food and Drug Administration (FDA) 

for use in the US. Compared to photons, protons have inherently different energy distribution 

patterns that can potentially spare more healthy tissue in front of and behind the tumor target. 

Tissue in the proton beam’s path to the target tumor receives a small radiation dose, tissue 

surrounding the target receives even less radiation, and the maximum dose of radiation can be 

directed to the tumor target.24,25 Because of its theoretical dosimetric advantages, proton therapy 

has held the clinical promise of allowing for higher doses of radiation to be delivered more 

safely. However, protons’ exit dose advantage has been shown to deteriorate with increased 

beam path distances.26 Therefore, there may be less opportunity for proton beam therapy to 

improve clinical outcomes for deep-seated tumors, like prostate, for which IMRT can now safely 

deliver optimally high radiation doses. Further, compared to photon radiation, dose delivery in 

PBT is more sensitive to tissue density and heterogeneity.25,26 Because of this sensitivity, organ 

motion can have a greater impact on PBT dose delivery compared with photon radiation.26

Published coverage policies consider PBT medically necessary only for cancers requiring 

exceptional precision in dose delivery (see Appendix A). The American Society for Radiation 

Oncology (ASTRO) supports the use of PBT for ocular tumors, tumors that approach or are 

located at the base of the skull, certain tumors of the spine, primary hepatocellular cancer treated 

in a hypofractionated regimen, solid tumors in children treated with curative intent, and in 

patients with genetic syndromes for whom minimizing the total volume of radiation is crucial.27

For prostate cancer, ASTRO recommends “Proton beam therapy for primary treatment... should 

only be performed within the context of a prospective clinical trial or registry.”27 With the 

exception of hepatocellular cancer, which is not explicitly covered by other organizations, other 

coverage policies generally agree with ASTRO’s conclusions. PBT for ocular, skull-base, and 

pediatric tumors is usually considered medically necessary, while PBT for more common 

cancers, such as lung cancer and prostate cancer, is usually considered not medically necessary 

and therefore is not covered. Even so, treatment of common cancers with PBT is widespread. In 

a study of Medicare patients receiving proton beam therapy between 2006 and 2009, the most 

common diagnosis on Medicare claims was prostate cancer (69.9%), followed by lung (7.1%), 

eye (6.6%), and other or unknown conditions including leukemia, lymphoma, skin, and 

unspecified sites (6.5%).28

In 1988, the FDA granted 510(k) marketing clearance to the PBT center and system at the Loma 

Linda University Proton Beam Therapy System (K872369). Currently, 6 manufacturers produce 

proton therapy systems that have received marketing clearance under the 510(k) process: 

Hitachi, Ltd., Mitsubishi Electric Inc., IBA Inc., Varian Medical Systems Inc., Sumitomo Inc., 

and Mevion Medical Systems, Inc. Proton therapy accelerators that are designed and built on-site 

are not subject to FDA premarket approval processes.24

11
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

According to the National Association of Proton Therapy, there are 14 PBT centers in operation 

in the United States with 10 additional centers under construction (http://protontherapy.org/map.htm). Facility components vary, but are typically customized, multi-room 

buildings that must include an accelerator and required shielding, treatment rooms (gantry- or 

fixed-beam), patient alignment and digital imaging equipment, and treatment-planning rooms 

with computerized control units.24 Multi-room proton beam facilities can cost between 100 and 

225 million dollars to construct and equip and between 15 and 25 million dollars annually to 

maintain.24 Similarly, the treatment itself remains costly compared with other radiation therapy 

modalities. Among Medicare patients treated for prostate cancer in 2008 and 2009, the median 

amount reimbursed by Medicare for PBT was $32,428 while the median amount reimbursed for 

matched IMRT patients was $18,575.7

As with other external beam radiation therapies, treatment typically occurs once daily for 5 days, 

up to 8 weeks, and patients attend pretreatment planning sessions to confirm tumor location and 

plan for patient immobilization, including designing body casts, bite-molds, and masks.24

Additional expenses associated with PBT is the creation of individual brass apertures for each 

beam, requiring special facilities and personnel. Two types of rooms are used in treatment: 

gantry-beam rooms rotate the beam around the patients, and fixed-beam rooms direct the beam 

horizontally. The patient is first positioned in the treatment room and then the proton beam is 

delivered to the target tumor. Each treatment session lasts a half-hour to a hour, with most of this 

time devoted to patient positioning.24 Because of the limited number of proton beam centers and 

therefore reduced availability of PBT compared with other modalities, travel and treatment delay 

may be a burden for patients.

Recently, manufacturers have started developing smaller proton beam radiation therapy systems 

in the hopes of reducing some of the initial costs of implementing PBT.24,26 Some argue that 

more affordable PBT systems will deescalate the debate on the use of PBT for more common 

cancers and lead to more individualized treatment using a combination of treatment modalities.26

The development and utilization of more precise proton beam delivery methods, such as pencilbeam scanning technology, may also lead to better clinical outcomes compared to photon 

modalities.

12
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

METHODS 

TOPIC DEVELOPMENT 

The following key questions guiding this systematic review were developed after a topic 

refinement process that included a review of published peer-reveiwed literature and consultation 

with the technical expert panel, operational partners, and experts in the field: 

KQ1: What is the effectiveness of proton beam irradiation compared to conventional X-raybased external beam modalities?

KQ2: What is the effectiveness of proton beam irradiation compared to state-of-the-art therapies?

KQ3: In patients with local recurrences after irradiation, what is the effectiveness of proton beam 

irradiation compared to conventional X-ray-based external beam modalities and state-of-the-art 

therapies?

KQ4: What are the short- and long-term harms of proton beam irradiation compared to 

conventional X-ray-based external beam modalities and state-of-the-art therapies?

KQ4A: What are the harms of proton beam irradiation compared to photon-based therapies in 

treating mobile targets that may move during treatment? 

We selected studies that compared benefits (survival, quality of life, functional capacity, local 

tumor control, delivery of planned chemotherapy and radiation regimens) and harms (toxicity 

and secondary malignancies) for PBT versus other radiation modalities in adults with any cancer 

type, with the exception of ocular cancer. 

SEARCH STRATEGY

To identify articles relevant to the key questions, our research librarian searched MEDLINE® 

(via PubMed®), the Cochrane Central Register of Controlled Trials, and ClinicalTrials.gov using 

the terms “proton beam” and “cancer” (see Appendix C for complete search strategies). 

Additional citations were identified from hand searching reference lists and consultation with 

content experts. We limited the search to published and indexed articles involving human 

subjects and available in the English language. 

To identify additional published, unpublished and ongoing studies, we searched 

ClinicalTrials.gov, and requested information from and searched the websites of manufacturers 

and proton therapy centers. Formal Requests for Scientific Information were sent to

manufacturers of proton therapy systems (Hitachi, IBA, Mevion Medical Systems, Mitsubishi 

Electric, Sumitomo Heavy Industries, and Varian Medical Systems) and proton beam therapy 

centers located at Loma Linda University Medical Center, University of California San 

Francisco Radiation Oncology, Massachusetts General Hospital, Indiana University Health 

Center, University of Florida, ProCure Proton Therapy Centers (Oklahoma City, New Jersey, 

and Seattle), University of Pennsylvania, CDH Proton Center, Hampton University, Washington 

University School of Medicine, Scripps Proton Therapy Center, and ProTom International. All 

ciations were imported into an electornic database (EndNote X).

13
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

STUDY SELECTION

We selected studies that compared benefits (survival, quality of life, functional capacity, local 

tumor control, delivery of planned chemotherapy and radiation regimens) and harms (toxicity 

and secondary malignancies) for PBT versus other radiation modalities in adults with any cancer 

type (except ocular). In studies with a comparison to brachytherapy, soreness, bruising and 

surgical complications were also considered. Titles, abstracts, and full-text articles were 

reviewed by one investigator then checked by another. All disagreements were resolved by 

consensus. We first reviewed titles and abstracts for potential inclusion, then reviewed relevant 

full-text articles. Articles meeting eligibility criteria were included for data abstraction.

DATA ABSTRACTION

We evaluated the following information from each study: study site, years of recruitment, sample 

size, patient demographic information, proton and photon treatment protocols, follow-up time, 

and outcomes assessed including benefits and harms. We checked the accuracy of data 

abstraction for these elements from the 2014 review produced by ICER for the Washington State 

Health Care Authority Health Technology Assessment Program (HTA).

1 We found their 

abstraction to be reliable for these elements, so we only abstracted data from studies that were 

not already included in the ICER review. All data abstraction was first completed by one 

reviewer and then checked by another. 

QUALITY ASSESSMENT 

We used the Cochrane Collaboration’s Risk of Bias Tool29 to rate the internal validity of 

included RCTs based on adequate sequence generation, allocation concealment, blinding, 

assessment of incomplete data, outcome reporting bias, and other sources of bias. We used the 

methods from the Drug Effectiveness Review Project to assess the quality of observational 

studies.30 Additionally, we used AMSTAR to assess the quality of included systematic reviews.

31

All internal validity ratings were first completed by one reviewer and then checked by another. 

All disagreements were resolved by consensus.

RATING THE BODY OF EVIDENCE

We rated the overall strength of evidence about each outcome based on the guidance established 

for the Evidence-based Practice Center Program of the Agency for Healthcare Research and 

Quality (AHRQ).32 This method considers the internal validity, directness of comparison, 

consistency of effect, precision of estimate, and reporting bias across the studies relevant to that 

outcome. We assigned the following evidence grades:

High: We are very confident that the estimate of effect lies close to the true effect for this 

outcomes. 

Moderate: We are moderately confident that the estimate of effect lies close to the true 

effects for this outcome. 

Low: We have limited confidence that the estimate of effect lies close to the true effects 

for this outcome. 

14
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Insufficient: We have no evidence, we are unable to estimate an effect, or we have no 

confidence in the estimate of effect for this outcome.

DATA SYNTHESIS 

We qualitatively synthesized the body of literature for each key question after abstracting 

relevant data and producing strength of evidence tables. We did not conduct meta-analyses 

because few studies reported on outcomes of interest and the studies were methodologically

heterogeneous. 

PEER REVIEW

Eight invited peer reviewers provided comments on the draft version of this systematic review. 

See Appendix I for the peer review disposition table. 

15
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

RESULTS

RESULTS OF SEARCH AND STUDY SELECTION

Figure 1 provides the results of the study selection process. We identified 3,052 titles and 

abstracts from the database search and 352 additional references through manual searching, 

suggestions from content experts, or from requests for scientific information. We reviewed 334 

full-text articles and included 51 articles in our synthesis. 

Scientific information requests to proton therapy system manufacturers and representatives of 

proton beam therapy centers resulted in submissions from IBA, Sumitomo, Varian, University of 

Pennsylvania, the Proton Therapy Consortium, and the Proton Collaborative Group. None 

identified any primary comparative studies that we did not also identify through our own 

searching, but they did provide extensive information on ongoing studies and identified a recent 

systematic review completed by ICER on proton beam therapy for various cancers.1 Our 

searching of ClinicalTrials.gov did not identify any additional published or unpublished 

completed comparative studies, but our review of abstracts from the 2014 PTCOG-NA 

conference identified one unpublished comparative study of 25 patients with head and neck 

cancer treated at Memorial Sloan Kettering Cancer Center that we briefly discuss in the head and 

neck cancer section.

Nine ongoing RCTs were identified via requests for scientific information and ClinicalTrials.gov 

(see Appendix F). Comparators to PBT include conventional photon beam therapy (N=3), IMRT 

(N=5), SBRT (N=1), and 3D-CRT (N=1). Populations include non-small cell lung cancer 

(NSCLC), prostate cancer, meningioma, glioblastoma, esophageal cancer, and oropharyngeal 

cancer patients. These ongoing studies are being conducted at the University of Texas MD 

Anderson Cancer Center, Massachusetts General Hospital, and the University of Pennsylvania, 

as well as other centers in the US and abroad.

16
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Figure 1. Literature Flow Chart

352 records identified through hand 

searching reference lists, content expert 

recommendation, requests for scientific 

information, and website searches of 

proton beam therapy centers

2,440 titles and abstracts excluded

334 full-text articles assessed for eligibility

283 full-text articles excluded (see 

Appendix D for ICER comparative 

studies excluded at full-text level)

11 ineligible population 

15 ineligible intervention

14 ineligible comparator

23 ineligible outcome

11 ineligible study design (eg case 

report, qualitative methods) 

90 ineligible publication type (eg

abstract, editorial, letter, narrative 

review, etc)

11 outdated or ineligible systematic 

review

2 non-English language 

106 didn’t meet noncomparative 

N≥100 criterion

51 articles included in synthesis 

addressing one or more key questions 

(25 comparative, 6 systematic reviews, 

20 noncomparative) 

43 articles addressing KQ1, KQ2, or 

KQ4 (23 comparative, 6 systematic 

reviews, 14 noncomparative)

9 articles addressing KQ3(2 

comparative, 7 noncomparative)

OVERVIEW OF PRIMARY STUDIES AND SYSTEMATIC REVIEWS 

We identified a total of 25 primary comparative studies. Table 1 shows which comparators were 

evaluated for different cancer sites. IMRT was the comparator in 12 studies4-10,15,33-36, including 

6 observational studies in prostate cancer patients.5-10 Other comparators were 3D-CRT (6 

studies),2,4,9,33,34,36 conventional photon therapy (5 studies),3,17,18,37,38 and combination of proton 

plus photon radiation compared to either photon alone (5 studies)12-14,39,40 or to brachytherapy (1 

studies).11 Table 1 provides a detailed listing of all 25 comparative studies.

0 articles addressing KQ4A 

3,052 records identified through 

database searching

1,964 from PubMed® on 12/10/14 

111 from the Cochrane Central 

Register of Controlled Trials on 

12/10/14 

50 from ClinicalTrials.gov on 

12/10/14 

927 from journals specific to cancer 

and medical physics on 12/10/14 

2,774 records screened for eligibility after removal of duplicates

17
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Most comparative studies were either retrospective cohorts (17 studies)3,7-10,12,15-18,33,35-40 or 

cohort studies including a historical control (3 studies).4,11,34 Additionally, we identified one 

randomized controlled trial,14 2 controlled before/after studies,5,6

 one prospective cohort study,13

and one prospective phase 1 nonrandomized trial.2 We rated the majority of studies as poor 

quality (14 studies)3,5,6,12,15-18,34,35,37-40 and the remainder as fair quality (11 studies).2,4,7-11,13,14,33,36

The main quality concerns were failing to adequately adjust for potential confounders and 

potentially biasing selection by comparing outcomes from PBT patients with historical results 

(see Appendix E for complete quality assessments of included comparative studies). 

18
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Table 1. Relevant Comparative Studies

Cancer type Comparison No. and design of comparative studies

Breast APBI: Single-field PBT vs 3D CRT 1 fair-quality prospective phase 1 nonrandomized trial (N=98): GallandGirodet et al 20142

CNS: Medulloblastoma PBT vs conventional photon 1 poor-quality retrospective cohort (N=40): Brown et al 20133

CNS: Spinal cord glioma PBT vs IMRT 1 poor-quality retrospective cohort (N=32): Kahn et al 201115

CNS: Recurrent malignant 

brain tumor

PBT vs conventional photon 1 poor-quality retrospective cohort (N=26): Mizumoto et al 201317

Esophageal PBT vs IMRT or 3D-CRT 2 fair-quality retrospective cohorts (N=519): McCurdy et al 2013,33 Wang 

et al 201336

Giant cell tumors of bone PBT+photon vs conventional photon 1 poor-quality retrospective cohort (N=20): Chakravarti et al 199939

Head/neck: Malignant clival 

tumors

PBT vs IMRT 1 poor-quality retrospective cohort (N=6): Solares et al 200635

Liver: Recurrent 

hepatocellular carcinoma

PBT vs conventional photon 1 poor-quality retrospective cohort (N=8): Otsuka et al 200318

Lung PBT vs IMRT or 3D-CRT 1 fair-quality and 1 poor-quality cohort study with historical 

controls(N=1,104): Gomez et al 2012,4 Sejpal et al 201134

Meningiomas PBT vs conventional photon 1 poor-quality retrospective cohort (N=25): Arvold et al 200937

PBT+photon vs various photon 

modalities

1 poor-quality retrospective cohort (N=31): Hug et al 200040

Prostate PBT vs IMRT Benefits: 2 poor-quality controlled before/after studies (N=1,695): Gray et 

al 2013,5 Hoppe et al 20146

Harms: 4 fair-quality retrospective cohorts (N=34,185): Kim et al 2011,9

Sheets et al 2012,10 Yu et al 2013,

7 Fang et al 20148

PBT vs 3D-CRT Benefits: 1 poor-quality controlled before/after study (N=218): Gray et al 

20135

Harms: 1 fair-quality retrospective cohort (N=12,107): Kim et al 20119

Prostate PBT+photon vs brachytherapy 1 fair-quality cohort study with a historical control (N=282): Coen 201211

PBT+photon vs photon alone 1 fair-quality RCT, 1 poor-quality prospective cohort, and 1 poor-quality 

retrospective cohort (N=567): Shipley et al 1995,14 Galbraith et al 2001,13

Duttenhaver et al 198312

Uveal hemangioma PBT vs conventional photon 1 poor-quality retrospective cohort (N=44): Hocht et al 200638

Various cancers PBT vs various photon modalities 1 poor-quality retrospective cohort (N=1,116): Chung et al 201316

19
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Overview of previous systematic reviews

The ESP generally prioritizes existing good-quality systematic reviews as the best source of 

evidence to guide our reports. We identified numerous reviews conducted on proton beam 

therapy in the past 5 years.

1,24,41-54 Table 2 briefly summarizes their scope and relevant 

conclusions. However, many existing reviews had limited usefulness because they (1) were 

outdated as their literature searches were conducted prior to the publication of most of the 

comparative studies,

45,46,48,52 (2) were noncomparative,41,42,47 or (3) did not review full-text 

articles of primary studies.24,43,44 The 2014 review by the Institute for Clinical and Economic 

Review (ICER)1 is the most comprehensive, including 22 of the 25 comparative studies we 

identified. Although we had hoped to rely on the ICER review findings, we undertook our own 

analysis of the primary data because our conclusions differed from ICER’s as discussed below. 

For the other systematic reviews identified, we assessed their internal validity (Appendix G) and 

included findings from those that included a majority of comparative studies1,53,54 or conducted 

indirect meta-analyses.49-51

20
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Table 2. Previous Systematic Reviews on the Effectiveness of Proton Beam Therapy for Cancer

Review Funding source Cancer types

Search end 

date Scope Relevant Conclusions

Included in synthesis

Agency for 

Healthcare Research 

and Quality 

(AHRQ) 201453

U.S. Department 

of Health and 

Human Services

Prostate March 7, 

2014 

Includes 2 of 10 comparative 

studies: Sheets et al 2012 and 

Kim et al 2011

Evidence is inadequate.

Institute for Clinical 

and Economic 

Review (ICER) 

20141

 

Produced for 

Washington State 

Health Care 

Authority Health 

Technology 

Assessment 

Program (HTA)

Various 

(government 

grants, non-profit 

organizations, 

health plans/ 

provider 

contributions, 

manufacturer 

grants, contracts, 

contributions), 

funding is not 

accepted from 

manufacturers or 

private insurers 

to perform 

review of 

specific 

technologies.

Various February 

2014 

Includes 22 of 25 comparative 

studies; excludes: GallandGirodet et al 201 (breast cancer) 

and Fang et al 2014 (prostate 

cancer) that were published after 

search date and Duttenhaver et al 

1983 in prostate cancer. 

Net health benefit is incremental 

for brain/spinal (reduced harms, 

low strength), comparable for 

uveal hemangiomas (low 

strength), liver (low strength), 

lung (moderate strength), 

prostate (moderate strength), and 

insufficient for all other cancers. 

California 

Technology 

Assessment Forum 

(CTAF) 201254

NR Prostate August 

2012 

Covers comparisons to 

conventional photon and 

brachytherapy, but but only 

includes 1 of 6 studies of IMRT 

(Sheets et al 2012) because most 

were published after its search

date 

Proton beam therapy was not 

found to be as beneficial as any 

established alternatives. 

21
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Review Funding source Cancer types

Search end 

date Scope Relevant Conclusions

Grimm et al 201249 NR Prostate 2010 Indirect comparison of PSA-free 

progression of uncontrolled 

studies of various treatments

For low risk prostate cancer, 

higher average progression-free 

survival was reported for 

brachytherapy compared with all 

other treatment modalities

Grutters et al 201050 Siemens Medical 

Solutions

Lung August 

2008 

Indirect comparison of summary 

event rates from uncontrolled 

studies of proton vs uncontrolled 

studies of photon. Did not 

include two comparative studies 

published after its search date 

(Sejpal et al 2011, Gomez et al

2012)

5-yr overall survival rate for 

proton beam was higher than for 

conventional radiation therapy, 

but similar to SBRT

Patel et al 201451 Mayo 

Foundation for 

Medical 

Education and 

Research

Head & Neck: 

Paranasal sinus 

and nasal cavity

April 2014 Indirect comparison of between 

pooled event rates from 

uncontrolled studies of PBT 

(including 1-8 cohorts) and 

uncontrolled studies of IMRT 

(including 2-8 cohorts) 

Disease-free survival at 5 years 

and locoregional control at 

longest follow-up were 

significantly higher with PBT 

compared with IMRT 

Not included in synthesis – outdated 

American Society 

for Therapeutic 

Radiology and 

Oncology (ASTRO) 

201245

National Cancer 

Institute

Various November 

2009 

Only includes 2 comparative 

studies on prostate (Shipley et al

1995, Duttenhaver et al 1983) as 

most comparative studies were 

published after its search date.

CNS, GI, head/neck: Insufficient 

evidence

Lung, prostate: No clear benefit 

over existing therapies

Amichetti et al 

201046

NR Head & Neck: 

Skull base 

chondrosarcoma

June 2008 4 uncontrolled studies PBT has a high probability of 

“medium- and long-term cure” 

with few complications.

De Ruysscher et al 

201248

European 

Investment Bank 

Various December 

2010 

Only includes one comparative 

study on uveal hemangioma 

(Hocht et al 2006) as most 

comparative studies were 

published after its search date. 

Except for rare indications such 

as childhood cancer, the gain 

from proton therapy is 

controversial. 

22
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Review Funding source Cancer types

Search end 

date Scope Relevant Conclusions

Ramaekers et al 

201152

None Head & Neck February 

2010 

Indirect comparison of summary 

event rates from uncontrolled 

studies of proton vs uncontrolled 

studies of photon

Except for paranasal and 

sinonasal cancer, survival and 

tumor control for proton therapy 

were generally similar to the best 

available photon radiotherapy.

Not included in synthesis – noncomparative 

Blue Cross Blue 

Shield (BCBS) 2011 

– in press42

AHRQ Lung June 2010 8 uncontrolled studies Evidence is insufficient to make 

any conclusions about PBT for 

NSCLC

Cianchetti et al 

201247

NR Head & Neck: 

Sinonasal 

malignancies

October 

2011 

5 uncontrolled studies Promising results must be 

confirmed in further studies. 

Not included in synthesis for various reasons 

ECRI 201444 Various 

(memberships, 

clients, contracts 

– often 

governmental)

Prostate February 

2014 

Hotline response – review of 

abstracts, not full texts

N/A

ECRI 201343 Various 

(memberships, 

clients, contracts 

– often 

governmental)

Brain, head, neck, 

skull base 

April 2013 Hotline response – review of 

abstracts, not full texts

N/A

ECRI 201324 Various 

(memberships, 

clients, contracts 

– often 

governmental)

Various July 2012 Overview, RCTs only No analysis possible because no 

appropriately designed trials 

were identified. 

Blue Cross Blue 

Shield (BCBS) 2011 

– in press41

AHRQ Prostate October 

2010 

Unclear, citations not available Evidence from comparative 

studies is insufficient to make 

conclusions on PBT for prostate 

cancer

23
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

ICER review

The ICER review used a comprehensive search and broad prespecified study selection criteria to 

minimize study selection bias.

1

 They used appropriate methods for rating study quality and the 

strength of the evidence but provided insufficient detail to assess the validity of their ratings. 

The ICER review proved useful for its accurate data abstraction, but we couldn’t rely on it for 

quality assessment or conclusions. Our conclusions often differed from ICER’s (Table 3) and 

ICER provided insufficient details to verify how they reached their conclusions. There is no clear 

pattern to the discrepancies between our conclusions and ICER’s. For CNS, lung, and prostate 

cancers, we drew separate conclusions for each comparison of PBT to IMRT, 3D-CRT, 

conventional photon, etcetera, respectively, whereas ICER’s conclusions merged data from 

various comparisons despite dissimilar findings. Our strength of evidence ratings were higher 

than ICER’s for esophageal cancer and lower for both liver cancer and uveal hemangiomas. 

24
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Cancer type Comparison Comparative studies

Findings of comparative 

studies

ICER 

conclusions 

ESP comments on 

ICER’s Conclusions

Breast APBI: Singlefield PBT vs 3D 

CRT

1 fair-quality 

prospective phase 1 

nonrandomized trial 

(N=98): GallandGirodet et al 2014 

Benefits: Similar local 

recurrence

Harms: Higher 7-yr mod-severe 

dyspigmentation and 

patchy/marked atrophy, but 

similar fat necrosis, fibrosis, 

breast pain, rib fracture 

N/A – not 

included 

because 

published after 

ICER’s search 

end date 

N/A

CNS: 

Medulloblastoma

PBT vs 

conventional 

photon 

1 poor-quality 

retrospective cohort 

(N=40): Brown et al 

2013 

Benefits: NSD in 2-yr overall 

and progress-free survival and 

locoregional failure 

Harms: Lower medical 

management of esophagitis

Overall 

incremental net 

benefit with 

PBT: equal 

benefits (low 

SOE), lower 

harms (low 

SOE)

Overgeneralization, 

conclusion applies to 

medulloblastoma, but 

not spinal cord glioma CNS: Spinal cord 

glioma

PBT vs IMRT 1 poor-quality 

retrospective cohort 

(N=32): Kahn et al 2011 

Benefits: Increased mortality 

within 5 years (adjusted HR:

55.82; (95% CI: 1.34-2316.8)) 

Harms: No long-term toxicity

Esophageal PBT vs IMRT 

or 3D-CRT

2 fair-quality 

retrospective cohorts 

(N=519): McCurdy et al

2013, Wang et al 2013 

Benefits: Not analyzed

Harms:

Pneumonitis: Worse for PB 

GI toxicity: NSD

Post-op pulmonary 

complications: PBT less than 

3D-CRT and similar to IMRT 

Evidence is 

inadequate to 

draw 

conclusions 

Disagree that evidence is 

inadequate 

Liver PBT vs 

conventional 

photon 

1 poor-quality 

retrospective cohort 

(N=8): Otsuka et al 2003 

Benefits: similar survival

Harms: Not reported 

Equal benefits 

(low SOE), 

equal harms 

(low SOE)

Disagree: evidence is 

insufficient because 

serious imprecision 

(N=8), poor quality, 

unknown consistency

Table 3. Comparison of ICER and ESP Findings by Cancer Type 

25
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Cancer type Comparison Comparative studies

Findings of comparative 

studies

ICER 

conclusions 

ESP comments on 

ICER’s Conclusions

Lung PBT vs IMRT 

or 3D-CRT

1 fair-quality and 1 

poor-quality cohorts 

with historical control 

groups (N=854); Gomez 

et al 2012, Sejpal et al 

2011 

Benefits: similar months of 

survival in Sejpal et al 2011

Harms: 

severe esophagitis similar to 

3D-CRT and lower than IMRT 

at 6 months

Equal benefits 

(low SOE), 

equal harms 

(low SOE) 

Overgeneralization; 

harms do not appear 

similar compared with 

IMRT 

Prostate PBT vs IMRT Benefits: 2 poor-quality 

controlled before/after 

studies (N=1695): Gray 

et al 2013, Hoppe et al

2014 

Harms: 3 fair-quality 

retrospective cohorts 

(N=7,292): Kim et al 

2011, Sheets et al 2012, 

Yu et al 2013

Benefits: NSD in QOL 

Harms: GI toxicity: similar 

early ‘treatment-related’ (Yu et 

al 2013), worse late in 2 studies 

(Sheets et al 2012, Kim et al 

2011) 

Equal benefits 

(moderate 

SOE), 

comparable 

major harms 

(moderate 

SOE)

Overgeneralizations, 

findings vary by 

PBT vs 3D-CRT Benefits: 1 poor-quality comparison 

controlled before/after 

study (N=218): Gray et 

al 2013 

Harms: 1 fair-quality 

retrospective cohort 

(N=19,063): Kim et al 

2011

Harms: risk of GI toxicity 

higher among patients that 

underwent PBT 

PBT vs 

brachytherapy 

1 fair-quality cohort 

with historical control

(N=282): Coen et al 

2012 

Benefits: similar 8-yr survival

Harms: NR

26
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Cancer type Comparison Comparative studies

Findings of comparative 

studies

ICER 

conclusions 

ESP comments on 

ICER’s Conclusions

Prostate PBT+photon vs 

photon alone

1 poor quality RCT, 1 

poor quality prospective 

cohort, and 1 fair quality 

prospective 

cohort(N=567); Shipley 

et al 1995, Duttenhaver 

et al 1983, Galbraith et 

al 2001

Benefits: similar 5-yr overall 

survival, 8-year overall survival, 

quality of life, and health status

Harms: similar rectal, urinary 

symptoms in two studies, higher 

rectal bleeding in one study 

Head and neck 

cancers

PBT vs IMRT 1 poor-quality 

retrospective cohort

(N=6): Solares et al 

2006

Benefits: similar 8-month 

overall survival

Harms: NR

Insufficient Agree

Uveal 

hemangioma

PBT vs 

conventional 

photon 

1 poor-quality 

retrospective cohort 

(N=44): Hocht et al 

2006 

Benefits: NSD in vision 

stabilization

Harms: 

Grade ≥ 3 harms: Comparable 

lacrimation

Equal benefits 

(low SOE), 

equal harms 

(low SOE)

Disagree that evidence is 

sufficient for drawing 

conclusions 

Giant cell tumors 

of the bone 

PBT+photon vs 

conventional 

photon 

1 poor-quality 

retrospective cohort 

(N=20): Chakravati et al 

1999 

Benefits: disease progression 

lower in radiation-only 

subgroup and higher in partial 

resection+radiation subgroup 

Harms: NR

Insufficient Agree

Meningiomas PBT vs 

conventional 

photon 

1 poor-quality 

retrospective cohort 

(N=25): Arvold et al 

2009 

Benefits: similar improved 

vision, but lower stable and 

more worsened

Harms: NSD in late 

asymptomatic retinopathy

Insufficient Agree

27
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

KEY QUESTIONS 1, 2, AND 4:

What is the effectiveness of proton beam irradiation compared to 

conventional x-ray-based external beam modalities?

What is the effectiveness of proton beam irradiation compared to 

state-of-the-art therapies?

What are the short- and long-term harms of proton beam irradiation 

compared to conventional x-ray-based external beam modalities and 

state-of-the-art therapies?

OVERALL 

We could not fully assess the overall net health benefit of proton beam therapy versus its 

comparators because comparative observational studies did not consistently report many 

outcomes of greatest interest. No studies reported functional capacity outcomes or overall severe 

late toxicity. No prospective comparative studies reported secondary malignancies. Ability to 

deliver planned treatments was only reported by one small retrospective cohort study.3

 

BREAST CANCER

One small, nonrandomized trial of PBT versus photon-based 3-dimensional conformal 

accelerated partial-breast irradiation provided low-strength evidence of comparable 7-year 

cumulative local recurrence rates, and higher rates of some 7-year skin toxicities, including 

moderate/severe dyspigmentation and patchy/marked atrophy.2 This trial was published after the 

ICER review. Fat necrosis, moderate/severe fibrosis, 7-year moderate/severe breast pain, 5-year 

rib fracture. and 7-year self-reported cosmetic outcomes did not differ (Table 4). Treatment was 

administered twice daily over 4 consecutive days at a dose of 32 Gy in 8 fractions. In the proton 

beam group, only one field was treated per fraction using the passive double scattering system. 

Patients’ median age was 61 years. The type of breast cancer was invasive ductal carcinoma 

(92%), tubular (5%), or mucinous (3%), and median tumor size was 0.9 cm. The tumor grade 

was 1 in 47% of patients, 2 in 42%, and 3 in 10%. Breast cancer cells tested negative for 

estrogen receptors in 11% of patients, progesterone receptors in 20%, and were triple-negative in 

10%. Authors suggested that their inferior cosmetic results of PBT may have been due to their 

use of only a single field per fraction, which may have resulted in a greater entry/skin dose. They 

recommend that use of multiple beam scattering or a scanning-beam technique may better 

minimize skin toxicity. In fact, there were no cases of grade 3 or higher skin toxicity at 5 years in 

a series of 100 patients at Loma Linda when proton beam therapy at was delivered at 40 Gy in 10 

fractions, once daily over 2 weeks, using multiple fields.55

With the advent of intensity-modulated proton therapy (IMPT), it may now be more feasible to 

expand the role of PBT beyond partial breast irradiation. However, we found no studies of IMPT 

for whole breast or nodal irradiation. It will be important to consider whether IMPT can improve 

on the 7.4%-per-gray rate for major coronary events within the first 5 years observed in a 

population-based case-control study of 2168 women who underwent external radiotherapy for 

breast cancer between 1958 and 2001 in Sweden and Denmark.56

28
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Table 4. Strength of Evidence (SOE) for Comparative Benefits and Harms of PBT or 

Photons or Mixed Photons/Electrons in Partial Breast Treatment for Early-stage Breast 

Cancer

Strength of evidence: Low for all 

outcomes Findings

No. studies (N): 1 prospective 

nonrandomized trial (98)

Study limitations: Medium 

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

7-year cumulative local recurrence: 11% (PBT) vs 4% (3D-CRT); 

P=.22 

Toxicity

Moderate/severe dyspigmentation: 90% (PBT) vs 27% (3D-CRT); 

P<.0001; telangiectasia: 70% (PBT) vs 28% (3D-CRT); P<.0001; 

patchy/marked atrophy: 54% (PBT) vs 15% (3D-CRT); P<.0001; 

fat necrosis: 10% (PBT) vs 12% (3D-CRT); P=.7329; 

moderate/severe fibrosis: 51% (PBT) vs 32% (3D-CRT); P=.1490; 

moderate/severe breast pain: 21% (PBT) vs 17% (3D-CRT); 

P=.4597; rib fracture at 5 years: 1 (PBT) vs 3 (3D-CRT); P=.072

CNS CANCERS

Summary

We identified 2 comparative studies in patients with CNS cancers. One is a retrospective cohort 

study that compared PBT to conventional photon therapy in 40 adults with medulloblastoma 

from the MD Anderson Cancer Center in Texas.3

 The other is a retrospective cohort study that 

compared PBT to intensity-modulated radiation therapy (IMRT) in 32 patients treated for 

intramedullary gliomas at Massachusetts General Hospital.15 Based on these 2 studies, the ICER 

review concluded that there is low-strength evidence that proton beam therapy has an 

incremental net benefit in the form of decreased harms. We agree with ICER’s conclusion as 

specifically related to medulloblastoma and only for acute toxicity, but not for intramedullary 

gliomas.

Medulloblastoma: Proton versus conventional photon

One poor-quality retrospective cohort study of 40 adults with medulloblastoma provided lowstrength evidence that PBT 54.6 GyE and photon therapy 52.9 Gy have comparable benefits, but 

proton beam therapy was associated with reduced acute toxicity (Table 5).3 Long-term toxicity 

was not reported. 

29
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Table 5. SOE for Proton Craniospinal Irradiation (CSI) versus Conventional Photon CSI 

for Medulloblastoma 

Strength of evidence: Low for all 

outcomes Findings

No. Studies (N): 1 retrospective 

cohort (40) 

Study limitations: High 

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

2-year overall survival: 94% (PBT) vs 90% (photon); P=.67

2-year progression-free survival: 94% (PBT) vs 85% (photon); 

P=.27 

Proportion of patients with treatment breaks: 0% (PBT) vs 10% 

(photon); P=.27

Locoregional failure: 5% (PBT) vs 14% (photon); P=.41

Acute toxicity (one month) 

Patients with > 5% weight loss: 16% (PBT) vs 64% (photon); 

P=.004 

Medical management of esophagitis: 5% (PBT) vs 57% (photon); 

P<.001 

Grade ≥ 2 nausea/vomiting: 26% (PBT) vs 71% (photon); P=.004

Spinal cord glioma: Proton versus IMRT

There is low-strength evidence that use of PBT may be disadvantageous for highly infiltrative 

tumors such as intermedullary spinal cord gliomas.15 This conclusion is based on one 

retrospective cohort study of 32 patients treated for intramedullary gliomas at Massachusetts 

General Hospital with either PBT (N=10) or IMRT (N=22) at an average dose of 51 Gy in 1.8 

median daily fractions over 29 treatments. The main limitation of this study is that patients in the 

proton beam group had more favorable prognostic factors, such as lower age (14 years compared 

with 44 years) and a higher rate of partial resection (70% compared with 55%). However, after 

adjustment for age and pathology, proton beam-treated patients were more likely to die within 5 

years (adjusted hazard ratio, 55.82; 95% CI: 1.34-2316.8). Local recurrence occurred in 20% of 

proton beam-treated patients and 23% in those treated with IMRT, but interpretation of this 

result is limited by the lack of multivariate analysis.

Table 6. SOE for PBT versus IMRT for Spinal Cord Gliomas

Strength of evidence: Low for 

survival, insufficient for others Findings

No. Studies (N): 1 retrospective 

cohort (32) 

Study limitations: High 

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

5-year overall survival: 20% (PBT) vs 32% (IMRT); aHR (PBT vs 

IMRT): 55.82 (95% CI: 1.34-2,316.8) 

Local recurrence: 20% (PBT) vs 23% (IMRT); P=.89 

Acute toxicity (one month)

Not reported 

Long-term toxicity

“No patients experienced significant long-term toxicity.” 

30
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of evidence: Low for all 

outcomes Findings

Postoperative complications

No. Studies (N): 1 retrospective 

cohort (N=444) 

Study limitations: Medium

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Postoperative pulmonary complications: OR (IMRT vs PBT): 2.23

(95% CI: 0.86-5.75); OR (3D-CRT vs PBT): 1.83 (95% CI: 1.83-

45.42); 30.3% (3D-CRT) vs 13.9% (PBT) 

GI complications: OR (IMRT vs PBT): 1.02 (95% CI: 0.47-2.25); 

OR (3D-CRT vs PBT): 2.31 (95% CI: 0.69-7.74); 28.4% (3DCRT) vs 18.1% (PBT)

ESOPHAGEAL CANCER

Summary

Comparative evidence in esophageal cancer is limited to 2 fair-quality retrospective cohort 

studies that compare acute toxicity, but not benefits, between PBT, IMRT, or 3-dimensional 

conformal radiation therapy (3D-CRT) in patients treated at the MD Anderson Cancer Center in 

Texas from 1998 to 2011 (N=75)33 and from 2003 to 2011 (N=444).36 Based on these 2 studies, 

the ICER review concluded that “Evidence is limited and inadequate to compare the potential 

harms of PBT relative to other radiation modalities in patients with esophageal cancer, 

particularly in comparison to IMRT.” We disagree with ICER’s conclusion and conclude that 

these studies provide low-strength evidence of some differences in acute toxicity. But evidence 

on benefits and long-term toxicity is still needed to adequately assess net health benefit of proton 

beam. 

Detailed analysis

Radiation modality (50.4 Gy of PBT, IMRT, or 3D-CRT) was associated with variation in 30-

day post-operative complications when it was a component of trimodal therapy comprised of 

neoadjuvant chemoradiation followed by surgical resection. There is low-strength evidence that 

IMRT and proton beam have comparable risk of postoperative pulmonary complications (OR 

2.23; 95% CI: 0.86-5.75) and GI complications (OR 1.02; 95% CI: 0.47-2.25), 3D-CRT and 

proton beam have comparable risk of GI complications (OR 2.31; 95% CI: 0.69-7.74; 28.4% vs 

18.1), but that 3D-CRT has a higher risk than proton beam of pulmonary complications (OR: 

9.13; 95% CI: 1.83-45.42; 30.3% vs 13.9%).36 The second study evaluated the role of lung lobes 

in radiation pneumonitis in 75 patients who received restaging [18F]-fluorodeoxyglucose (FDG) 

positron emission tomography (PET) imaging 1 to 3 months after chemoradiation involving 50.4 

Gy of either PBT, IMRT or 3D-CRT.33 This study provides low-strength evidence that proton 

therapy is associated with a higher risk of acute pneumonitis compared with IMRT/3D-CRT 

(33% vs 15%; P=.04). 

Table 7. SOE for PBT versus IMRT and 3D-CRT in Esophageal Cancer

31
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of evidence: Low for all 

outcomes Findings

Acute Pneumonitis

No. Studies (N): 1 retrospective 

cohort (N=75) 

Study limitations: Medium

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Acute Pneumonitis: 33% (PBT) vs 15% (IMRT/3D-CRT); P=.04

GIANT CELL TUMOR OF THE BONE

Combined PBT+photons versus photons alone

We identified one poor-quality retrospective comparative cohort that evaluated the clinical 

effectiveness of combination proton and photon treatment versus photon-only treatment in giant 

cell tumor of bone. ICER was the only systematic review to include this study and they 

determined that there was insufficient evidence to make a conclusion regarding PBT for giant

cell tumor of bone. We agree with their conclusion. In this study, patients were treated with 

PBT+photons (N=6; mean: 59 GyE) or photons alone (N=14; mean: 52 Gy) with a median 

duration of follow up of 9.3 years.39 Thirteen patients also received partial tumor resection. In 

this study, the 10-year lack of progression rate was 17% for the PBT+photons patients and 14% 

among the photons only patients (P=.88). No harms were reported in this study. This study did 

not control for differences between groups including age (median: 23.5 years for PBT+photons 

patients; 52.5 years for photons only patients). Further, it is unclear how patients were selected 

for the PBT+photons group or the photons alone group (see Appendix E for complete quality 

assessment).

Table 8. SOE for Combined PBT+Photons versus Photons Alone for Giant Cell Tumor of 

Bone 

Strength of evidence: Insufficient 

for all outcomes Findings

No. Studies (N): 1 retrospective 

cohort (20) 

Study limitations: High

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

10-year lack of progression rate: 17% (PBT+photons) vs 14% 

(photons); P=.88 

32
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

HEAD AND NECK CANCERS

Summary

Comparative evidence for proton beam versus IMRT in head and neck cancers is limited to one 

small (N=6) poor-quality retrospective cohort35 and 2 indirect meta-analyses of non-comparative 

studies.51,52 We agree with ICER’s conclusion that the cohort study provided insufficient 

evidence to draw conclusions about PBT in head and neck cancers. The indirect meta-analyses of 

non-comparative studies also provide insufficient evidence to draw conclusions about proton 

beam in head and neck cancers due to unacceptably high methodological limitations. 

Detailed analysis

PBT versus IMRT

Direct evidence

One small (N=6) poor-quality retrospective cohort35 provided insufficient evidence to draw 

conclusions about PBT and IMRT for cancers of the head and neck. In the cohort study, when 

radiation therapy was given following endoscopic resection of malignant clival tumors, overall 

survival was 100% in the proton group after 8 months and 67% in the IMRT group after 16-24 

months.35 However, we have no confidence in these findings because the study was extremely 

small with many limitations (such as no baseline demographic information on patients by 

treatment group), and while the comparison was direct, consistency was unknown and the 

survival estimates were imprecise. 

We also identified a PTCOG-NA 2014 conference proceeding of a new comparative study 

including 25 patients with major salivary gland cancer or cutaneous squamous cell carcinoma 

metastases. These patients were treated at the Memorial Sloan Kettering Cancer Center and the 

results suggest the potential for reduced acute toxicity with PBT versus IMRT, but we could not 

assess the strength of the evidence beacause the abstract did not provide sufficient detail to 

adequately address internal validity.

57

Indirect evidence

The meta-analyses of non-comparative studies also provide insufficient evidence to draw 

conclusions about proton beam in head and neck cancers due to unacceptably high 

methodological limitations. Both meta-analyses used similar statistical methods, but the more 

recent one is more comprehensive since it includes more studies.51 Patel et al made indirect 

comparisons between pooled event rates from uncontrolled studies of PBT (including 1-8 

cohorts) and uncontrolled studies of IMRT (including 2-8 cohorts) for patients with paranasal 

sinus and nasal cavity malignancies.51 They found no difference in overall survival at longest 

follow-up, 5-year overall survival, disease-free survival at longest follow-up, and 5-year 

locoregional control, but found improved disease-free survival at 5 years (RR: 1.44; 95% CI: 

1.01-2.05) and locoregional control at longest follow-up (RR: 1.26; 95% CI: 1.05-1.51) in the 

proton therapy group. Harms were pooled across all charged particle therapy cohorts, so PBTspecific harms could not be compared with those after photon modalities. However, we have 

serious concerns about the methods that Patel et al used for indirect comparison. Because there 

are no common control groups in the uncontrolled studies, it is impossible to verify the 

33
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

underlying prognostic comparability between the PBT and IMRT populations. But, at minimum, 

there does appear to be the potential for temporal and location confounding. For example, the 

analysis of 5-year disease-free survival included one study of 36 patients from Massachusetts

General Hospital treated between 1991 and 2001 for proton therapy and 3 studies of 187 total 

patients, mostly from Belgium treated between 1998 and 2009 for IMRT. 

Table 9. SOE for Comparative Benefits of PBT or IMRT for Paranasal Sinus and Nasal 

Cavity Malignancies 

UVEAL HEMANGIOMA 

One poor-quality retrospective cohort provides comparative evidence on the benefits and harms 

of radiation treatment with protons or photons in uveal hemangiomas. Based on this study, the 

ICER review concluded that there was low-strength evidence suggesting that PBT has 

comparable benefits and harms compared with other treatment modalities for uveal 

hemangiomas. We disagree, and conclude that this study provides insufficient evidence to draw 

conclusions about PBT compared with photons for the treatment of uveal hemangioma. In this 

study, patients were treated with protons (N=25; 20-22.5 CGE) or photons (N=19; 16-30 Gy) 

between 1993 and 2002 at the Charite Campus Benjamin Franklin center in Berlin and followed 

up for a median of 27.7 months. There was no statistically significant difference between 

radiation modality on stabilization of visual acuity (P=.43), optic disc/optic nerve atrophy 

(P=.27), or retinopathy (P=.098).38 There were no significant differences in treatment side effects 

between treatment groups. Grade 3 side effects occurred in 2 patients in each treatment group 

(P=.7718) and Grade 4 side effects occurred in one patient in the PBT group (P=.3789). While 

baseline uveal hemangioma thickness was different between the 2 groups, it is not clear that this 

would affect prognosis. 

Strength of evidence: Insufficient 

for all outcomes Findings 

No. Studies (N): 4-16 cohort studies 

(202-539) 

Study limitations: High

Directness: Indirect

Consistency: Inconsistent

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

Overall survival at longest duration of follow-up: RR (PBT vs 

IMRT): 1.02 (95% CI: 0.77-1.35) 

5-year overall survival: RR(PBT vs IMRT): 1.39 (95% CI: 0.99-

1.94) 

Disease-free survival at longest duration of follw-up:RR(PBT vs 

IMRT): 0.98 (95% CI: 0.40-2.42) 

5-year disease-free survival: RR(PBT vs IMRT): 1.44 (95% CI: 

1.01-2.05) 

Locoregional control at longest duration of follow-up: RR: 1.26 

(95% CI: 1.05-1.51) 

5-year locoregional control: RR(PBT vs IMRT): 0.73 (95% CI: 

0.15-3.58) 

34
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of evidence: Insufficient 

for all outcomes Findings

No. Studies (N): 1 retrospective 

cohort (44) 

Study limitations: Medium

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

Stabilization of visual acuity: no difference; P=.43 

Late toxicity

Grade 3: no difference; P=.7718 

Grade 4: no difference; P=.3789 

Table 10. SOE for PBT versus Photons for Uveal Hemangiomas 

LIVER CANCER

We identified one retrospective cohort examining PBT among patients with recurrent liver 

cancer.18 This study is discussed under Key Question 3. 

LUNG CANCER

Summary

Comparative evidence on the use of PBT versus IMRT,4,34 3D-CRT,4,34 SBRT,50 or conventional 

radiotherapy50 in non-small cell lung cancer (NSCLC) patients comes from 2 historicallycontrolled cohort studies4,34 and one meta-analysis that compared pooled event rates from case 

series of CRT, SBRT and PBT.50 Based on the historically controlled cohort studies,4,34 the ICER 

review concluded that “Moderate evidence suggests that rates of treatment-related toxicities with 

PBT are comparable to those seen with other radiation modalities in patients with lung cancer.” 

We disagree, and conclude that for locally advanced NSCLC these studies provide low-strength 

evidence that, even at a higher dose, PBT’s risk of 6-month severe esophagitis is lower than 

IMRT and similar to 3D-CRT. However, there is insufficient evidence on survival and 15- to 17-

month toxicity when given with concurrent chemotherapy. Comparability of late toxicity is 

unknown. For early stage NSCLC, due to unacceptable methodological limitations, the indirect 

meta-analysis provides insufficient evidence to draw conclusions about the comparison of PBT 

with SBRT or conventional radiotherapy.50

Detailed analysis

Proton-based ablative RT versus photon-based ablative RT for early stage NSCLC

The indirect meta-analysis by Grutters et al provides insufficient evidence to draw conclusions 

about the comparison of proton beam versus stereotactic body radiotherapy (SBRT) or 

conventional radiotherapy (CRT) for NSCLC due to methodological limitations which are 

described below.50 Grutters et al made indirect comparisons between adjusted pooled event rates 

for 2- and 5-year disease-specific and overall survival outcomes from 11 uncontrolled cohorts of 

CRT, 11 of SBRT, and 5 of PBT, and found that 5-year overall survival was significantly higher 

among PBT patients compared with CRT patients (P=.014) and similar to SBRT patients (Table 

11).50 A strength of this analysis is that it included adjustment for some sources of potential 

confounding. The percentage of medically inoperable patients significantly influenced model 

35
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

coefficients, so it was used to correct the pooled outcome estimates. Age, percentage of small 

tumors (<3 cm), percentage of medically inoperable patients, and median follow-up time were 

ruled out as effect modifiers. However, there is a remaining risk of potential confounding by 

variation in study setting and temporal trends. For example, in the outcome of 5-year diseasespecific survival, most CRT cohorts were from the Netherlands or the US and were followed as 

early as 1976 to usually the mid-1990s. In contrast, most SBRT cohorts were from Japan and 

followed as early as 1994 to usually the mid-2000s. Finally, only 2 PBT cohorts, from the US 

and Japan, were included in this outcome and recruitment dates were unknown for one cohort. 

Table 11. Findings and SOE for PBT versus SBRT and CRT in Early-stage NSCLC

Strength of evidence: Insufficient 

for all outcomes Findings

No. Studies (N):Indirect metaanalysis of 27 uncontrolled cohorts 

(N=2,401) 

Study limitations: High

Directness: Indirect

Consistency: Inconsistent

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

2-year overall survival:53% (CRT) (95% CI: 46-60%); 70%

(SBRT) (95% CI: 63-77%); 61% (PBT) (95% CI: 47-75%) 

2-year disease-specific survival:67% (CRT) (95% CI: 59-76%); 

83% (SBRT) (95% CI: 75-92%); 74% (PBT) (95% CI: 61-87%) 

5-year overall survival:19% (CRT) (95% CI: 15-24%); 42% 

(SBRT) (95% CI: 34-50%); 40% (PBT) (95% CI: 24-55%) 

5-year disease-specific survival:43% (CRT) (95% CI: 31-56%); 

63% (SBRT) (95% CI: 50-75%); 52% (PBT) (95% CI: 32-72%) 

PBT versus IMRT or 3D-CRT for locally advanced NSCLC

Benefits

There is insufficient evidence to draw a conclusion about survival after higher-dose proton beam 

(74 Gy) versus 3D-CRT (63 Gy) or IMRT (63 Gy) with concurrent chemotherapy in patients 

with primarily stage IIIA-B unresectable NSCLC.34 One poor-quality historically controlled 

cohort study included 202 patients who were treated at MD Anderson Cancer Center in Texas 

with either proton beam between 2006 and 2008, IMRT between 2003 and 2005, or with 3DCRT between 2001 and 2003. Follow-up time was 17.9 months (range: 2.3-76.1) for 3D-CRT 

patients, 17.4 months (range: 1.8-65.5) for IMRT patients, and 15.2 months (range: 3.3-27.4) for 

PBT patients (P<.001). Median survival times were 17.7, 17.6, and 24.4 months for 3D-CRT, 

IMRT, and PBT, respectively, but not statistically significantly different between groups 

(P=.1061). This study is limited by lack of adjustment for important baseline characteristics such 

as: older mean age, higher proportion of nonwhite, lower proportion of clinical disease stage 3B 

and above, lower tumor volume, lower proportion with adjuvant chemotherapy, and more recent 

treatment in the proton beam group. This lack of adjustment confounded the effect of PBT on 

survival. Also, differences between radiation modalities in methods used to account for tumor 

motion may have confounded their effects on survival. The authors indicated that 4-dimensional 

CT scanning had been used to account for tumor motion since 2004, but that would only apply to 

the proton group and the last year of the IMRT group, but not the 3D-CRT group. Finally, the 

study is potentially limited by differences in lung cancer care in the time periods studied. 

36
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Harms 

There is low-strength evidence that, even at a higher dose (74 Gy), acute risk of severe 

esophagitis (grade ≥ 3) at 6 months for proton beam therapy (6%) is similar to 3D-CRT 63 Gy 

(8%; P=.42) and lower than IMRT 63 Gy (28%, P<.0001).4 One cohort study involving 652 

patients with NSCLC, mostly clinical stage IIIA-B and mean age of 66 years, followed patients 

who were treated at MD Anderson Cancer Center between 2000 and 2008 and evaluated harms 

among those treated with PBT versus 3D-CRT or IMRT.4 Across the whole cohort, predictors of 

severe esophagitis were concurrent chemotherapy and treatment with over 30 fractions, but not 

age, gender, smoking status, stage, or histology. Risk of Grade 3 esophagitis at 15-17 months 

was lower with proton beam therapy 74 Gy (5%) than with 3D-CRT 63 Gy (18%; P=.02) and 

IMRT 63 Gy (39%; P<.0001) in a subgroup of 202 patients with primarily stage IIIA-B NSCLC 

treated with concurrent chemotherapy during that same time period.34 In the subgroup with 

concurrent chemotherapy, risk of Grade 3 pneumonitis for PBT (2%) was similar to IMRT (6%, 

P=.23) but lower than 3D-CRT (30%, P<.0001). However, higher dose PBT was associated with 

a higher risk of Grade 3 dermatitis (24%) than 3D-CRT (5%, P=.005) and similar to IMRT 

(17%, P=.30). There were no statistically significant differences in Grade 3 fatigue, with rates of 

19% for proton beam, compared with 15% for IMRT (P=.54) and 29% for 3D-CRT (P=.09). 

Limitations of these toxicity findings are the same as described above for benefits. Although in 

the study of the subgroup undergoing concurrent chemotherapy the authors found tumor size to 

be independent of Grade 2 or greater toxicities and disease stage to be independent of 

pneumonitis overall, we don’t know how applicable these findings are to Grade 3 toxicity and 

still cannot rule out potential confounding by the other unbalanced factors. Also, we cannot 

account for the discrepancy in magnitude of the rates of Grade 3 esophagitis (5-6%) and 

pneumonitis (2%) for proton therapy between the historically-controlled cohort studies4,34 and 

those from a case series from the same facility (MD Anderson Cancer Center, 2006-2009)58,59

and other proton facilities between 2001 and 2010 for esophagitis (11%) and pneumonitis (2-

6%).60-64 

37
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Table 12. Findings and SOE for PBT versus IMRT and 3D-CRT in Locally Advanced 

NSCLC

Strength of evidence: Insufficient 

for survival time and 15-17 month 

toxicity, low for 6-month 

esophagitis Findings

Median survival time and 15-17 

month toxicity

No. Studies (N): 1 cohort with 

historical control (N=202)

Study limitations: High

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

Median survival time: 17.7 months (3D-CRT), 17.6 months 

(IMRT), 24.4 months (PBT); P=.1061 

Toxicity

Esophagitis: 5% (PBT), 39% (IMRT) (P<.0001), 18% (3D-CRT) 

(P=.02) 

Pneumonitis: 2% (PBT), 6% (IMRT) (P=.23), 30% (3D-CRT) 

(P<.0001) 

Dermatitis: 24% (PBT), 17% (IMRT), (P=.30), 5% (3D-CRT) 

(P=.005). 

Fatigue: 19% (PBT), 15% (IMRT) (P=.54), 29% (3D-CRT) 

(P=.09)

6-month severe esophagitis

No. Studies (N): 1 cohort with 

historical control (N=652)

Study limitations: Medium

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Toxicity

Severe esophagitis at 6 months: 6% (PBT), 8% (3D-CRT) (P=.42), 

28% (IMRT) (P<.0001) 

MENINGIOMA

Summary

Comparative evidence for PBT versus conventional photon therapy is limited to one small 

(N=22) poor-quality retrospective cohort37 and evidence for combined PBT and photon therapy 

versus photon therapy alone is limited to another small (N=31) poor-quality retrospective 

cohort.40 The ICER review only included the study of PBT versus conventional photon therapy37

and concluded that the evidence was insufficient to draw conclusions about PBT for 

meningiomas. We agree with their conclusion since this small study did not provide any data or 

statistical tests on benefits or harms by treatment group. Based on the additional retrospective 

cohort we identified, we concluded that there is insufficient evidence to make a conclusion about 

the benefits and harms of combined PBT and photon therapy for meningioma.40

38
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of evidence: Insufficient 

for all outcomes Findings

No. Studies (N): 1 (25)

Study limitations: High

Directness: Direct

Consistency: Unknown 

Precision: Imprecise 

Reporting bias: Undetected

Other issues: None

Benefits and toxicity

“There was no significant difference at most recent follow-up 

between proton and photon irradiation with regard to tumor 

control, visual outcome, or treatment side effects.”

Detailed analysis

PBT versus conventional photon

One poor-quality retrospective cohort provides insufficient evidence on the use of PBT for 

meningioma.37 All patients with optic nerve sheath meningioma treated at Massachusetts General 

Hospital between 1999 and 2006 were included in the study (N=25), with 13 patients receiving 

photon irradiation, 9 receiving proton irradiation, and 3 patients receiving combination PBT and 

photon treatment (average dose: 51.4, 51.1, and 57 GyE, respectively). The authors state, “There 

was no significant difference at most recent follow-up between proton and photon irradiation 

with regard to tumor control, visual outcome, or treatment side effects.” Unfortunately, no data 

or statistical tests were reported by treatment group. Another limitation of this study is that they 

did not account for the potential confounding effects of the larger average tumor size in the 

proton group (4.15 mL vs 2.25 and 3.63 mL), which could have a worse prognosis, or its shorter 

duration of follow-up (12.5 months vs 42 and 78 months). 

Table 13. SOE for Comparative Benefits of PBT or Conventional Photon Therapy for 

Optic Nerve Sheath Meningioma 

PBT+photon versus photon alone

Evidence on the benefits and harms of combined PBT and photon therapy (59.5-72.0 CGE) 

compared with photon therapy alone (40.5-76 Gy) is limited to one small, poor-quality 

retrospective cohort study and is insufficient to draw conclusions about combined PBT and 

photon therapy for meningioma.40 In this study, 5-year local control was significantly higher in 

the combined therapy group (80% vs 17%; P=.008), but was not different at 8 years (40% vs 

17%, estimated from figure 2). Among patients with malignant meningioma (N=13), 5- and 8-

year overall survival was again significantly higher in the combined therapy group (100% vs 

44%; P=.025). Late toxicity outcomes were reported in 3 patients (2 cases of symptomatic 

necrosis and one case of extensive visual deficits), but the study did not state which treatment 

these patients received. Because baseline characteristics were not reported by treatment group, it 

is unclear whether the 2 groups were balanced at baseline on prognostic factors. 

39
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of evidence: Insufficient 

for all outcomes Findings

No. Studies (N): 1 (31)

Study limitations: High

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

5-year local control:80% (PBT+photon) vs 17% (photon); P=.008 

8-year local control:40% (PBT+photon) vs 17% (photon) 

5- and 8-year overall survival among malignant meningioma 

patients:100% (PBT+photon) vs 44% (photon); P=.025 

Table 14. SOE for Comparative Benefits of Combined PBT and Photon Therapy or Photon 

Therapy Alone for Meningioma 

PROSTATE CANCER

Summary

Previous reviews have reached divergent conclusions about the comparative effectiveness of 

proton beam in prostate cancer.1,53,54 The ICER review concluded that there is low-strength 

evidence of comparable benefits and moderate-strength evidence of comparable major harms for 

proton beam without distinguishing between the various comparators (eg, IMRT, 3D-CRT, 

conventional photon, etc). For the comparison of proton beam to IMRT or 3D-CRT, the 

December 2014 AHRQ Comparative Effectiveness Review53 on therapies for clinically localized 

prostate cancer concluded that evidence is inadequate to make a conclusion on the effectiveness 

of PBT versus IMRT9,10 or 3D-CRT.9

 For the comparisons of proton beam to brachytherapy and 

conventional photon therapy, the 2012 California Technology Assessment Forum (CTAF) 

review concluded that PBT has not been shown to be as beneficial as any established alternatives 

and, “Thus the role of proton beam therapy for localized prostate cancer within the current list of 

treatment options remains unclear.” Table 15 summarizes the numbers and types of comparative 

studies and our conclusions for each different comparison of PBT to IMRT, 3D-CRT, 

brachytherapy, and conventional photon therapy, respectively. 

Table 15. Conclusions of Comparative Studies of PBT versus IMRT, 3D-CRT, 

Brachytherapy, and Conventional Photon Therapy in Prostate Cancer

Comparator Benefits (Strength of Evidence 

Grade)

Harms (Strength of Evidence 

Grade)

PBT vs IMRT Similar Quality of Life (QOL) (low

SOE): 2 historically-controlled 

cohorts (N=1695):5,6

Transiently lower GU toxicity at 0-

6 months for PBT (low SOE),7 but 

similar GI and GU toxicity at 12-24 

months (low to moderate),

7,8 and 

increased GI toxicity with PBT at 4-

5 years (low SOE): 4 retrospective 

cohorts 8-10 N=34,185

PBT vs 3D-CRT Similar QOL (insufficient SOE), but 

survival vs 3D-CRT remains 

unknown: 1 historically-controlled 

cohort5; N=218

Increased acute GI toxicity with 

PBT (low SOE): 1 retrospective 

cohort9

; N=NR

40
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Comparator Benefits (Strength of Evidence 

Grade)

Harms (Strength of Evidence 

Grade)

PBT vs 

brachytherapy

Similar 8-yr survival and distant 

metastasis (low SOE): 1 historicallycontrolled cohort11; N=282

No evidence

PBT+photon vs 

photon alone

Similar overall 5-8 year survival and 

QOL (low SOE): (1 RCT, 2 cohort 

studies; N=567) 12-14

Increased 8-year rectal bleeding and 

urethral stricture (low SOE): 1 

RCT14; N=202

Detailed analysis

PBT versus IMRT

Benefits

No study has directly compared the survival of patients with prostate cancer who were treated 

with either PBT or IMRT.5-7,9,10 But based on consistent findings from 2 historically-controlled 

cohort studies, there is low-strength evidence of no significant differences between PBT and 

IMRT in bowel, urinary, and sexual quality of life at 2 years (Table 16).5,6 The main limitation of 

both studies is their use of a historical IMRT control group from a different site; therefore, it is 

impossible to rule out confounding by temporal trends or site-specific variables. In both studies, 

the IMRT groups were drawn from the same population of patients from the Prostate Cancer 

Outcomes and Satisfaction with Treatment Quality Assessment (PROST-QA) Consortium who 

were treated between March 2003 and March 2006 at dose ranges of 75.6-79.2 Gy. The PBT 

groups were 95 patients treated with 74-82 Gy(RBE) at Massachusetts General Hospital between 

August 2004 and December 2008 and 1,243 patients who had been treated with 78-82 Gy(RBE) 

at the University of Florida between 2006 and 2010. Patients were in their mid-60s and most had 

clinical T1 disease. Both studies had between-group differences at baseline, including that PBT 

patients were more likely to be older, white, have smaller prostate volumes, and to be less likely 

to receive androgen deprivation therapy (ADT). Results from the Hoppe 2014 study (Table 16) 

are likely more valid, though, as they used statistical techniques to control for the clinical 

diversity, whereas the Gray 2013 study did not. An additional major limitation of the Gray 2013 

study is that they used different instruments to assess quality of life in each group: the Prostate 

Cancer Symptom Indices (PCSI) scale for the PBT group and the Expanded Prostate Cancer 

Index Composite (EPIC) for the IMRT group. 

Harms

Some dose-escalation studies have found increased GI complication risks with increased dose of 

photons,65 but not with protons.66 But when compared head-to-head in prostate cancer, proton 

beam has only transiently reduced risk of 6-month acute GU toxicity versus IMRT, but may 

increase risk of late GI toxicity after 4-5 years. 

GI and GU toxicity at 6 months. One fair-quality retrospective cohort study provides lowstrength evidence of transiently lower GU toxicity for PBT versus IMRT at 0-6 months, but no 

significant difference in GI toxicity (Table 16).7

 This study by Yu and colleagues was a 

population-based study of Medicare claims data from the Chronic Condition Warehouse 

database that matched 314 patients with early-stage prostate cancer treated with proton beam 

41
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

database that matched 314 patients with early-stage prostate cancer treated with proton beam 

during 2008-2009 at an unknown dose with 628 treated with IMRT.7 Moderate to severe GI and 

GU toxicity at 12 months was assessed based on diagnosis or procedure codes. 

GI and GU toxicity at 12 to 24 months. Two fair-quality retrospective cohort studies provide 

moderate-strength consistent evidence of no significant difference between proton beam therapy 

and IMRT in early GI or GU toxicity (Table 16).

7,8 The first was the study by Yu and colleagues 

described above.

7

The second study was a matched comparison of prospectively-collected

clinician-reported grade 2 or greater GI and GU toxicity for 94 pairs of patients with localized 

low-intermediate risk prostate cancer who received 79.2 Gy delivered with either proton beam 

therapy or IMRT between 2010 to 2012 at University of Pennsylvania’s proton center.8 Despite 

methodological differences (ie, claims vs primary source data), the 2 studies were consistent in 

finding no significant differences in GI or GU toxicity at 12 months. The 2014 retrospective 

cohort study from the University of Pennsylvania proton center also provides low-strength

evidence of no significant differences in GI or GU toxicity at 24 months .

8

GI toxicity at 4-5 years. Two fair-quality, population-based retrospective cohort studies of 

Medicare claims data linked to the Surveillance Epidemiology and End Results (SEER) database 

provide low-strength evidence of an increased risk of late GI toxicity at 4-5 years (Table 16).9,10

The first of the SEER-database studies, by Kim et al, included patients diagnosed with earlystage localized prostate cancer between 1992 and 2005 and used ICD-9 or CPT procedure codes 

to assess grade 3 to 4 bleeding, ulceration, fistula, stricture, and colostomy that developed at least 

6 months after diagnosis and required intervention.9 The second of the SEER-database studies, 

by Sheets et al, assessed risk of unspecified GI morbidity-related procedures (including 

colonoscopy) and diagnoses that occurred at least 12 months after diagnosis in patients 

diagnosed with early-stage prostate cancer between 2000 and 2007.10 The main limitations of 

both studies include their high potential for exposure and outcome misclassification biases.67-69

Regarding exposure ascertainment, risk of bias was high because dose and field size specifics 

were unknown; therefore, the increased risk of late GI toxicity with PBT may have been entirely 

due to higher doses. Regarding outcome ascertainment, risk of bias was high because of the 

questionable reliability of using surrogate procedure (including colonoscopy) and diagnosis 

code-based measures to detect the actual clinical events of interest. Also, both studies may suffer 

from potential confounding by study site since likely a high majority of proton patients were 

treated at the single Loma Linda study site, whereas IMRT patients were likely treated at a 

variety of sites.

42
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of evidence Findings

2-year quality of life: Low

No. Studies (N): 2 historicallycontrolled cohort studies (N=1,695) 

Study limitations: High 

Directness: Direct

Consistency: Consistent

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Patients with differences in EPIC summary scores > 50% SD from 

Hoppe 2014: 

Bowel: 37% (PBT) vs 38% (IMRT); P=.99 

Urinary incontinence: 32% (PBT) vs 34% (IMRT); P=.99

Urinary irritative/obstructive: 17% (PBT) vs 19% (IMRT); P=.99

Sexual: 40% (PBT) vs 41% (IMRT); P=.99 

6-month GI and GU toxicity: Low

No. Studies (N): 1 retrospective 

cohort studies (N=1263) 

Study limitations: Medium

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

6-month GI diagnosis or procedure codes: PBT=2.9% vs 

IMRT=3.6%; OR, PBT vs IMRT: 0.84 (0.42 to 1.66) 

6-month GU diagnosis or procedure codes: PBT=5.9% vs 

IMRT=9.5%; OR, PBT vs IMRT: 0.60 (0.38 to 0.96) 

12-month GI and GU toxicity: 

Moderate

No. Studies (N): 2 retrospective 

cohort studies (N=1130) 

Study limitations: Medium

Directness: Direct

Consistency: Consistent 

Precision: Imprecise

Reporting bias: Undetected

Early GI toxicity (12 months)

Claims database analysis: 9.9% (PBT) vs 10.2% (IMRT); OR

(PBT vs IMRT): 0.97 (95% CI: 0.61-1.53) 

Clinician-reported grade ≥ 2 GI toxicity: 4.3% (PBT) vs 13.8% 

(IMRT); OR (PBT vs IMRT): 0.27 (95% CI: 0.06-1.24) 

Early GU toxicity (12 months)

Claims database analysis: 18.8% (PBT) vs 17.5% (IMRT); OR

(PBT vs IMRT): 1.08 (95% CI: 0.76-1.54) 

Clinician-reported grade ≥ 2 GU toxicity:21.3% (PBT) vs 28.7% 

(IMRT); OR (PBT vs IMRT): 0.69 (95% CI: 0.32-1.51) 

24-month GI and GU toxicity: 

Low

No. Studies (N): 1 retrospective 

cohort study (N=188) 

Study limitations: Medium

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Late GI toxicity (24 months):

Clinician-reported grade ≥ 2 GI toxicity:12.8% (PBT) vs 10.8% 

(IMRT); HR (PBT vs IMRT): 1.24 (95% CI: 0.53-2.94)

Late GU toxicity (24 months):

Clinician-reported grade ≥ 2 GU toxicity: 12.8% (PBT) vs 18.3% 

(IMRT); HR (PBT vs IMRT): 0.56 (95% CI: 0.22-1.41)

Table 16. SOE for Comparative Benefits and Harms of PBT versus IMRT in Prostate 

Cancer

43
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

4-5 year GI toxicity: Low

No. Studies (N): 2 retrospective 

cohort studies (N=6,350) 

Directness: Indirect

Consistency: Consistent

Precision: Imprecise

Reporting bias: Undetected

Any GI toxicity after 5 years: 20.1 (PBT) vs 8.3 (IMRT) per 1,000 

patient-years; HR (PBT vs IMRT): 3.32 (95% CI: 2.12-5.20) 

Procedures (including colonoscopy) at 46-50 months: 18 (IMRT) 

vs 21 (PBT) per 100 person-years; RR (IMRT vs PBT): 0.82 (95% 

CI: 0.70-0.97) 

Diagnoses at 46-50 months: 12(IMRT) vs 18(PBT) per 100 

person-years; RR (IMRT vs PBT): 0.66 (95% CI: 0.55-0.79) 

PBT versus 3D-CRT

Benefits

One poor-quality controlled before/after study provides insufficient evidence on the QOL of 

prostate cancer patients after PBT or 3D-CRT.5

 This study compares a historical cohort of 123 

men who underwent 3D-CRT (66.4-79.2 Gy) between 1994 and 2000 at Harvard-affiliated 

hospitals with 95 men who underwent PBT (74-82 Gy(RBE)) between 2004 and 2008 at 

Massachusetts General Hospital. QOL was assessed in both cohorts using the PCSI. After 24 

months, both the 3D-CRT (P<.001) and PBT (P=.004) treatment groups reported significantly 

improved mean bowel/rectal QOL scores compared with baseline. Neither group reported 

significantly different urinary irritation/obstruction or urinary incontinence QOL scores 

compared with baseline. Differences were not assessed between treatment groups. This study has 

a number of limitations, including using a historical cohort from a different site and not 

controlling for temporal or site differences and uncontrolled baseline differences between the 

treatment groups. The PBT group was younger, had lower baseline prostate-specific antigen 

(PSA) values, and included more patients with clinical T1 disease than the 3D-CRT group, 

potentially improving overall outcomes relative to the 3D-CRT group. 

Harms

One fair-quality retrospective cohort provides low-strength evidence that PBT results in more 

frequent acute GI toxicities compared with 3D-CRT for prostate cancer.9 Patients included in this 

study were residents of SEER regions that were diagnosed with T1-T2 clinically localized 

prostate cancer between 1992 and 2005. A total of 19,063 patients between 66 and 85 years of 

age that underwent 3D-CRT, IMRT, or PBT were included, but the study did not report how 

many patients were treated with each modality, or the average dose of each treatment. The 

overall rate of any GI toxicity was 20.1 events per 1,000 patients treated with PBT followed for 

one year and 9.2 events per 1,000 patients treated with 3D-CRT followed for one year. After 

controlling for year of cancer diagnosis, comorbidity, age group, clinical stage at diagnosis, 

SEER region, race, marital status, poverty status, and cancer grade, the risk of GI toxicity was 

higher among patients that underwent PBT compared with patients that underwent 3D-CRT 

(HR:2.13; 95% CI: 1.45-3.13). 

44
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of evidence: Insufficient 

for bowel/rectal, urinary 

irritation/obstruction, and 

urinary incontinence, low for risk 

of GI toxicity Findings

Bowel/rectal, urinary 

irritation/obstruction, and 

urinary incontinence

No. Studies (N): 1 cohort with 

historical control (N=218)

Study limitations: High

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

24-month mean bowel/rectal QOL scores compared with baseline: 

significantly improved (P<.001 (3D-CRT), P=.004 (PBT)) 

24-month mean urinary irritation/obstruction QOL scores 

compared with baseline: no difference (P>.05 (3D-CRT and PBT)) 

24-month mean urinary incontinence QOL scores compared with 

baseline: no difference (P>.05 (3D-CRT and PBT))

Risk of acute GI toxicity

No. Studies (N): 1 cohort 

(N=19,063) 

Study limitations: Medium

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Adjusted hazard ratio (PBT vs 3D-CRT): 2.13 (95% CI: 1.45-3.13)

Table 17. SOE for PBT versus 3D-CRT in Prostate Cancer

PBT versus brachytherapy

A 2012 study by Coen et al provided low-strength evidence of similar 8-year overall survival and 

freedom from distant metastasis after combined proton and photon radiation (79.2 GyE) or 

brachytherapy (125I or 103Pd to doses of 145 Gy or 115 Gy, respectively), but no comparative 

evidence on toxicity. Coen et al included 141 patients that were treated with a combination of 

conformal photon radiation and PBT (79.2 GyE) at either the Loma Linda University Medical 

Center or at Massachusetts General Hospital between 1996 and 1999.11,70 These patients were 

matched on T stage, Gleason score, PSA level, and age to 141 patients treated with permanent 

prostate brachytherapy at Massachusetts General Hospital between 1997 and 2002. Eight-year 

overall survival was 93% among combination therapy patients and 96% among brachytherapy 

patients (P=.45). Eight-year freedom from distant metastasis was 99% among combination 

therapy patients and 96% among brachytherapy patients (P=.42). Subgroup analyses by risk 

group, T stage, initial PSA level, and Gleason score did not reveal any significant differences 

between the two treatment groups. 

Alternatively, a 2012 meta-analysis by Grimm et el that was based on single-arm brachytherapy 

studies and two PBT studies71,72 utilizing data from the PROG/ACR 95-09 RCT70 concluded that 

brachytherapy had higher average progression-free survival. However, because the Grimm et al 

45
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of evidence: Low for all 

outcomes Findings

No. Studies (N): 1 (282)

Study limitations: Medium

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

8-year overall survival:93% (PBT+photon) vs 96% 

(brachytherapy); P=.45 

8-year freedom from distant metastasis:99% (PBT+photon) vs 

96% (brachytherapy) P=.21 

meta-analysis suffered from numerous methodological limitations, we used the higher strength 

evidence from Coen 2012 to form the basis of our conclusion about how PBT compares to 

brachytherapy. The Grimm 2012 meta-analysis indirectly compared the proportion of patients 

with PSA progression-free survival in uncontrolled studies of radical prostatectomy, 

brachytherapy, EBRT, androgen deprivation therapy, high intensity focused ultrasound, and high 

dose radiotherapy for low and intermediate risk prostate cancer.49 The authors of this study 

concluded that for low risk prostate cancer, higher average progression-free survival was 

reported for brachytherapy compared with all other treatment modalities. But since there were no 

common control groups among the included studies, comparability of the study populations 

cannot be verified. The authors also did not control for potential confounders such as the date or 

location of treatment in each study. For example, patients receiving PBT were treated at Loma 

Linda University and Massachusetts General Hospital between 1996 and 1999,70 while one 

brachytherapy study included patients treated at a hospital in Michigan and a hospital in 

Germany from 1991-2002 and 1986-1999, respectively.73

Table 18. SOE for Comparative Benefits of Conformal Photon Radiation and PBT or 

Brachytherapy for Prostate Cancer 

PBT+photon versus photon alone

Benefits

For overall survival, there was low-strength evidence that photon therapy with a proton “boost” 

and conventional photon therapy had comparative benefits at 8 and 5 years. The 8-year data 

comes from the fair-quality RCT of 202 patients with advanced stage T3 and T4 treated between 

1982 and 1992, which found survival rates of 55% for photon therapy with a proton “boost” and 

51% for photon therapy alone.14 At 5 years, survival rates reported in the RCT were 75% and 

80% for photon therapy with a proton “boost” and only photon therapy, respectively14 and 62% 

(estimated from figure 3E in Duttenhaver et al) in both groups in a poor-quality retrospective 

cohort with a broader population of patients with stage T1-T4 who were treated in the decade 

earlier (1973-1979).12 The retrospective cohort study was rated poor quality for this outcome 

since the potentially confounding effects of tumor differentiation were not controlled for and 

could lead to better survival outcomes. 

For other 8-year outcomes, there was no difference in Kaplan-Meier estimates of disease-free 

survival (72% vs 62%), or local control (77% vs 60%) overall, but local control was significantly 

improved in the subgroup of patients with poorly differentiated (Gleason score: 4-5) tumors 

(85% vs 40%; P=.0014).14

46
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Neither health-related QOL nor health status differed overall between treatment groups at 18 

months in the fair-quality prospective cohort study that followed patients in 5 different treatment 

groups (surgery, conventional radiation, PBT, a combination of conventional external-beam 

radiation and PBT, and watchful waiting).13 

It is important to note that in all 3 studies, the total dose was greater in the combined treatment 

group compared with the group that just received conventional photon therapy: 75.6 Gy versus 

67.2 Gy in the RCT;14 70-76.5 CGE versus 60-68.4 Gy in the retrospective cohort study;12 and 

74-75 Gy versus 65-70 Gy in the prospective cohort study.13 Further, the radiotherapy techniques 

used in the RCT14 and the retrospective cohort12 are outdated and so these outcomes may not be 

applicable to current radiotherapy methods. 

Harms

For 8-year harms, there was low-strength evidence that rectal bleeding was higher among the 

combined therapy group (32% vs 12%; P=.002) while urethral stricture and gross hematuria were 

not significantly different between the 2 groups.

14 For earlier time points, there was no difference 

in toxicity.12,13 As reported in the retrospective cohort study, urinary and rectal symptom 

incidence up to 5 years were not significantly different between the 2 treatment groups, but the 

median follow-up time was not reported.12 Finally, there was low-strength evidence of no 

difference in sexual, GI, and urinary treatment symptoms between combined PBT and photon 

therapy groups after 18 months.13 As discussed above, in all 3 studies, the total dose was greater 

in the combined PBT and photon therapy compared with the photon-only group.

47
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of 

Evidence 

Grade

Study Design: 

No. Studies 

(N)

Study 

Limitations

Directness Consistency PrecisionReporting 

Bias

Other 

Issues Findings

Outcome: 8-year overall survival

Low RCT: 1 (202) Medium Direct Unknown Imprecise Undetected None No difference: 55%

(PBT+photon) vs 51% (photon) 

Outcome: 5-year overall survival

Low RCT: 1 (202)

Retrospective 

cohort: 1 (180)

Medium

High

Direct Inconsistent Imprecise Undetected None No difference: 75%

(PBT+photon) vs 80% (photon) 

Outcome: quality of life and health status

Low Prospective 

cohort: 1 (185)

Medium Direct Unknown Imprecise Undetected None No difference

Outcome: 8-year rectal bleeding

Low RCT: 1 (202) Medium Direct Unknown Precise Undetected None 32% (PBT+photon) vs 12%

(photon); P=.002

Outcome: 8-year urethral stricture

Low RCT: 1 (202) Medium Direct Unknown Imprecise Undetected None 19% (PBT+photon) vs 8%

(photon); P=.07

Outcome: 8-year gross hematuria

Low RCT: 1 (202) Medium Direct Unknown Imprecise Undetected None 14% (PBT+photon) vs 8%

(photon); P=.25

Outcome: 18-month PTSS measure: sexual, gastrointestinal, and urinary

Low Prospective 

cohort: 1 (185)

Medium Direct Unknown Imprecise Undetected None No difference

Table 19. SOE for Benefits and Harms of Higher Dose PBT+Photon versus Lower Dose Conventional Photon in Prostate 

Cancer

48
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

SECONDARY MALIGNANCIES IN VARIOUS CANCERS

There is insufficient evidence to draw conclusions about how proton beam therapy compared to 

other radiation modalities in risk of secondary malignancy. We identified one retrospective 

cohort study that provided comparative evidence on secondary malignancies in non-metastatic 

cancer patients treated with PBT or photon modalities for a variety of cancers, but it has 

numerous methodological limitations which are described below.16 A total of 558 patients treated 

with PBT at Massachusetts General Hospital/Harvard Cyclotron between 1973 and 2001 were 

matched to 558 patients treated with external beam photon radiation included in the National 

Cancer Institute’s Surveillance, Epidemiology, and End Results (SEER) Program cancer registry 

based on cancer site, histology, age at treatment (±10 years), year of treatment (±5 years), and 

sex. Most patients in the PBT group received 20% of their total dose from photon radiation and 

the SEER cohort likely contained very few patients that received IMRT. The most common 

tumor sites in both groups were CNS (32%), head and neck (24%), genitourinary (33%), and 

musculoskeletal (7.7%). Fewer patients had GI (2.7%), lung (0.4%), and lymphoma (0.2%) 

cancers. After a median follow-up time of 6.7 years for PBT patients and 6.0 years for photon 

radiation patients, secondary malignancies developed in 5.2% and 7.5% of the PBT and photon 

patients, respectively. Among PBT patients, secondary malignancies included 26 solid tumors, 2 

lymphomas, and one leukemia. Among photon patients, secondary malignancies included 38 

solid tumors and 4 lymphomas. The incidence rate of secondary malignancies was 6.9 and 10.3 

cancers per 1,000 person-years among PBT and photon patients, respectively (P=.085) while 10-

year cumulative incidence rates were 5.4% and 8.6%, respectively (P=.085). After controlling for 

age at treatment, sex, tumor site, and year of diagnosis, the risk for developing secondary cancer 

among PBT patients was half that of photon patients (HR=0.52; 95% CI: 0.32-0.85; P<.01). 

The main limitations of this study are its potential for unmeasured confounding due to (1) 

missing data, (2) unknown radiation field size or dose, and (3) variation in ascertainment 

methods across groups. First, no information on secondary malignancies was obtained in 27% of 

patients in the proton therapy group who had no follow-up appointments or could not be reached 

by mail or phone. As it is plausible that the loss to follow-up is associated with secondary 

malignancy events, exclusion of those patients could have either over- or under- estimated the 

effects of proton therapy. Second, although patients were matched by treatment site and 

histology in an attempt to control for irradiation volume, we still cannot rule out the potential for 

unmeasured confounding based on this factor because the comparability of the radiation dose 

and field size are unknown. Thirdly, we cannot rule out the risk of differential misclassification 

due to variation of ascertainment methods across groups. Data for the proton group are likely 

more reliable because they were “...abstracted from pathology reports, radiology reports, 

operative notes and clinic visit notes in accordance with a standardized protocol. Patients were 

also contacted by mail and scripted telephone calls to obtain data. The second cancer incidence 

was verified by review of pathology reports.” Ascertainment of second cancers in the photon 

group was based on diagnosis codes from the SEER database, which are known to be prone to 

misclassification.

Also, these findings may not be widely generalizable. PBT patients that were not matched with 

SEER patients were significantly different than matched PBT patients, potentially limiting the 

generalizability of the findings to all PBT patients. Unmatched PBT patients were younger, more 

likely to be female, followed for a shorter amount of time, and were more likely to have rare 

49
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of evidence: Insufficient 

for all outcomes Findings

No. Studies (N):1 retrospective 

cohort (N=1,116) 

Study limitations: High

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Secondary malignancy incidence rate:6.9 (PBT) vs 10.3 (photon) 

per 1,000 person-years; P=.085 

10-year cumulative secondary malignancy incidence rates:5.4%

(PBT) vs 8.6% (photon); P=.085; Adjusted hazard ratio (PBT vs 

photon): 0.52 (95% CI: 0.32-0.85) 

malignancies. Further, it is not clear that the majority of secondary malignancies are a result of 

treatment because they mostly occurred within the first 5 years of treatment when solid cancers 

are not plausibly attributed to radiation therapy.74 Bekelman and colleagues estimated that in the 

first 5 years after treatment, the incidence rate of secondary malignancies was 11.4 and 25.1 

cancers per 1,000 person-years among PBT and photon patients, respectively, while after 5 

years, the incidence rate of secondary malignancies was 5.7 and 5.8 cancers per 1,000 personyears among PBT and photon patients, respectively. 

Table 20. Findings and SOE for PBT versus Photon and Secondary Malignancy in Various 

Cancers

KEY QUESTION 3: In patients with local recurrences after irradiation, 

what is the effectiveness of proton beam irradiation compared to 

conventional x-ray-based external beam modalities and state-of-theart therapies?

There is insufficient evidence to draw conclusions about the comparative effects of PBT versus 

other radiation modalities among patients with recurrent tumors. We identified 2 comparative 

studies on recurrent tumors, one among patients with recurrent malignant brain tumors17 and one 

among patients with recurrent liver cancer,18 but both studies were rated poor quality due to their 

failure to account for potentially important confounding. 

Proton versus SRT or conventional photon in recurrent malignant brain tumors

One retrospective cohort17 provides insufficient evidence to draw conclusions about PBT 

compared with SRT or conventional photon therapy for recurrent malignant brain tumor. Patients 

with recurrent malignant brain tumor after radiotherapy were reirradiated with conventional 

photons (N=8), stereotactic radiotherapy (N=10), or PBT (N=8) between 2005 and 2010. Patients 

received an average reirradiation dose of 43.7 (conventional photon), 41.7 (SRT), or 39.5 Gy 

(PBT). The median follow-up period for survivors was 19.4 months. After a median of 11.6 

months, 5 patients (62.5%) in the conventional photon group had died, 5 patients (50%) in the 

SRT group had died, and 5 patients (62.5%) in the PBT group had died (P>.99). Local recurrence 

occurred in 2 patients (25%) in the conventional photon group, 3 patients (30%) in the SRT 

group, and 2 patients (25%) in the PBT group (P>.99). Two patients, both in the SRT treatment 

group, experienced radiation necrosis. This study was rated poor quality because it did not 

50
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Strength of evidence: Insufficient 

for all outcomes Findings

account for the potentially confounding effects of younger age (51 vs 64.5 years (SRT)), lower 

reirradiation dose and larger average tumor size (70.4 vs 11.5 cc (SRT)). 

Table 21. SOE for Comparative Benefits of PBT, SRT, or Conventional Photon Therapy 

for Recurrent Malignant Brain Tumors17

No. Studies (N): 1 retrospective 

cohort (26) 

Study limitations: High

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

Mortality after a median of 11.6 months:62.5% (conventional 

photon), 50% (SRT), 62.5% (PBT); P>.99 

Local recurrence:25% (conventional photon), 30% (SRT), 25% 

(PBT); P>.99 

Proton versus photon in recurrent liver cancer

One poor-quality retrospective cohort18 provides insufficient evidence to draw conclusions about 

PBT compared with conventional photon therapy for recurrent liver cancer. This study included 

8 patients with recurrent hepatocellular carcinoma, 5 treated with protons (68.8-84.5 Gy) and 3 

treated with x-rays (60 or 70 Gy). Similar numbers of patients died in the proton and x-ray 

treatment groups after a median of 18 and 15.5 months, respectively (80% vs 100%; P=.62). This 

study was rated poor quality mainly because it did not account for the potentially better 

prognosis in the proton group due the lower median age (56 vs 60 years) and smaller average

tumor size (2.7 vs 3.6 cm). 

Table 22. SOE for Comparative Benefits of PBT or X-rays for Recurrent Hepatocellular 

Carcinoma18

Strength of evidence: Insufficient 

for all outcomes Findings

No. Studies (N): 1 retrospective 

cohort (8) 

Study limitations: High

Directness: Direct

Consistency: Unknown 

Precision: Imprecise

Reporting bias: Undetected

Other issues: None

Benefits

Median survival time:18 months (PBT) vs 15.5 months (X-ray)

Noncomparative studies of proton treatment for various recurrent tumors

Any treatment form is less effective in the setting of disease recurrence, including proton 

therapy. Compared with results from single-arm studies of proton beam in patients with primary 

tumors, there is lower survival and higher recurrences in single-arm studies of proton beam in 

patients with recurrent tumors. In a cohort of chordoma patients with primary (N=36) or locally 

recurrent (N=14) malignancies, 8-year local recurrence after treatment with a combination of 

51
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

photons and protons (median dose: 76.6 CGE) was greater among patients with recurrent 

compared with primary tumors (53% vs 15%; P=.002).75 Similarly, single-arm studies of 

recurrent nasopharyngeal carcinoma,76 hepatocellular carcinoma,77 and non-small cell lung 

cancer78 report lower rates of overall survival compared with single-arm studies of patients with 

primary tumors of the same type.58,79,80 However, we have limited confidence that the reduced 

benefits of proton therapy were primarily due to recurrent tumors, as the difference may have 

been confounded by clinical and methodological variability between the groups of single-arm 

studies, including that some of the recurrent tumor studies used a lower dose of radiation,76,78

were published earlier,76 and included younger patients76,77 compared with primary tumor 

studies. 

KEY QUESTION 4A: What are the harms of proton beam irradiation 

compared to photon-based therapies in treating mobile targets that 

may move during treatment?

Tumors of the lungs, esophagus, liver, pancreas, breast, prostate, and kidneys are subject to 

respiratory motion and the magnitude and trajectory of this motion can vary across patients. 

Respiratory motion can cause problems during imaging, planning, and delivery of radiation 

therapy, potentially leading to reduced dose calculation accuracy, difficulty quantifying the 

magnitude of margins, and differences between planned and delivered doses. Methods used to 

account for respiration motion include respiratory gating (radiation is delivered during a specific 

point in the breathing cycle), breath-hold, forced shallow-breathing involving a stereotactic body 

frame, and real-time tumor tracking. To determine the harms of proton beam irradiation 

compared to photon-based therapies in treating mobile targets that may move during treatment, 

we looked for studies that reported on (1) comparability of tumor motion between treatment 

groups and whether tumor motion affected clinical outcomes of interest, (2) existence or 

comparability of imaging and planning methods used to account for respiratory motion, and (3) 

existence or comparability of quality assurance standards used to monitor respiratory motion 

control methods. 

Although dosimetric studies comparing methods of accounting for respiratory motion in 

treatment planning report that 4DCT imaging decreases doses to normal structures compared 

with other multiphase,19 free-breathing,20 or 3DCT imaging,21 how this translates to clinical 

outcomes is not clear. We did not identify any studies that evaluated clinical outcomes of interest 

based on variability in tumor motion, imaging and planning methods used to account for 

respiratory motion, or quality assurance standards. Only 4 comparative studies explicitly 

describe imaging and planning methods used to account for respiratory motion, but none 

evaluated the effects of variability in methods.11,18,33,34 One retrospective cohort study of lung 

cancer patients34 treated with 3D-CRT (2001-2003), IMRT (2003-2005), and PBT (2006-2008) 

noted that methods of managing tumor motion were used with IMRT and PBT, but not 3D-CRT. 

Respiration-induced 3D motion of lung tumors was assessed using 4DCT imaging,81 but the 

authors did not discuss management of this movement. Survival times did not differ by treatment 

group. In one retrospective cohort study of esophageal cancer patients,33 radiation dose was 

calculated with a free-breathing treatment planning CT or with an average CT calculated from a 

4DCT image set, and around 66% of images were attenuation corrected with mid-inspiratory 

breath-hold from the PET/CT scanner. Clinical outcomes were not discussed by method of 

radiation dose calculation. In one retrospective cohort study of recurrent hepatocellular 

52
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

carcinoma,18 a respiration-gated irradiation system was used on all patients. Finally, in one 

cohort study of PBT prostate cancer patients case-matched with brachytherapy patients,11 PBT 

patients were immobilized for daily treatment using body casts.70

53
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Key Questions 1, 2, and 4

Population Comparator Findings

Breast Single field PBT vs 

photon-based 3D 

conformal 

accelerated partialbreast irradiation

We found low-strength evidence of comparable 7-year 

cumulative local recurrence, but higher rates of some 7-

year skin toxicities with PBT, including moderate/severe 

dyspigmentation, patchy/marked atrophy, but not for fat 

necrosis or moderate/severe fibrosis.

CNS: 

Medulloblastoma

Proton CSI vs 

photon CSI 

We found low-strength evidence that PBT 54.6 GyE and 

photon therapy 52.9 Gy have comparable benefits, but 

proton beam therapy was associated with reduced acute 

toxicity.

CNS: Spinal cord 

glioma

PBT vs IMRT We found low-strength evidence that use of PBT may be 

disadvantageous for highly infiltrative tumors such as 

intermedullary spinal cord gliomas.

Esophageal IMRT vs PBT

and 

3D-CRT vs PBT

There is low-strength evidence that trimodal therapy with 

neoadjuvant chemoradiation, surgical resection and either 

proton beam, IMRT or 3D-CRT have comparable risk of 

30-day postoperative GI complications, but that 30-day 

postoperative pulmonary complications for trimodal 

therapy with proton beam are lower than with 3D-CRT 

and similar to IMRT. There is also low-strength evidence 

that proton therapy is associated with a higher risk of 3-

month pneumonitis compared with IMRT/3D-CRT.

Evidence on benefits and long-term toxicity is still needed 

to adequately assess net health benefit of proton beam.

Giant cell tumor of 

the bone 

Combined PBT+ 

photons vs photons 

alone

We identified insufficient evidence to draw conclusions

regarding PBT for giant cell tumor of bone. 

Head and neck PBT vs IMRT We identified insufficient evidence to draw conclusions 

about PBT in head and neck cancers.

Uveal hemangioma PBT vs photons We identified insufficient evidence to draw conclusions 

about PBT compared with photons for the treatment of 

uveal hemangioma.

NSCLC PBT vs SBRT or 

CRT

Evidence in early-stage NSCLC was insufficient to draw 

conclusions. 

NSCLC 

(continued) 

PBT vs IMRT or 

3D-CRT

In patients with locally advanced NSCLC, there is lowstrength evidence that, even at a higher dose (74 Gy), 

acute risk of severe esophagitis (grade ≥ 3) at 6 months for 

PBT is similar to 3D-CRT 63 Gy and lower than IMRT 63 

Gy. Evidence on survival and 15-17 month toxicity in a 

subgroup of patients given concurrent chemotherapy was 

insufficient. 

SUMMARY OF EVIDENCE BY KEY QUESTION

The table below summarizes the evidence for each key question. 

Table 23. Summary of Evidence by Key Question

54
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Key Questions 1, 2, and 4

Population Comparator Findings

Meningioma PBT vs 

conventional 

photon 

PBT+ photon vs 

photon alone 

We identified insufficient evidence on the use of PBT or 

combined PBT and photon therapy for meningioma. 

Prostate PBT vs IMRT We identified low-strength evidence of no significant 

differences between PBT and IMRT in bowel, urinary, 

and sexual quality of life at 2 years. 

Transiently lower GU toxicity at 0-6 months for PBT (low 

SOE), but similar GI and GU toxicity at 12-24 months 

(low to moderate), and increased GI toxicity with PBT at 

4-5 years (low SOE).

PBT vs 3D-CRT We identified insufficient evidence on the QOL of 

prostate cancer patients after PBT or 3D-CRT.

There is low-strength evidence that PBT results in more 

frequent acute GI toxicities compared with 3D-CRT for 

prostate cancer.

PBT vs 

brachytherapy 

There is low-strength evidence of similar 8-year overall 

survival and freedom from distant metastasis after 

combined proton and photon radiation (79.2 GyE) and 

brachytherapy (125I or 103Pd to doses of 145 Gy or 115 Gy, 

respectively), but no comparative evidence on toxicity.

PBT+ photon vs 

photon alone

There is low strength evidence that combined PBT and 

photon therapy and conventional photon therapy have 

similar overall survival at 8 years and 5 years.

For 8-year harms, there was low-strength evidence that 

rectal bleeding was higher among the combined therapy 

group while urethral stricture and gross hematuria were 

not significantly different between the 2 groups.

Secondary 

malignancies – 

various cancers

PBT vs photon There is insufficient evidence to draw conclusions about 

how proton beam therapy compares to other radiation 

modalities in risk of secondary malignancy.

Key Question 3

There is insufficient evidence to make conclusions about the use of PBT among patients with recurrent

cancers.

Key Question 4A

We did not identify any studies that evaluated clinical outcomes of interest based on variability in 

tumor motion, imaging and planning methods used to account for respiratory motion, or quality 

assurance standards.

55
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

DISCUSSION 

For all cancer sites and types, except for ocular and pediatric cancers which were not reviewed 

here, there are no reliable data from long-term randomized trials on survival, quality of life, or 

functional capacity of patients who underwent PBT compared with any other modality. We could 

not fully assess the overall net health benefit of proton beam therapy versus its comparators 

because comparative observational studies did not consistently report many outcomes of greatest 

interest. No studies reported functional capacity outcomes or overall severe late toxicity. No 

prospective comparative studies reported secondary malignancies. Ability to deliver planned 

treatments was reported by only one small retrospective cohort study. 

Despite the common claim that the advantage of proton beam therapy is self-evident in all 

circumstances, comparative studies have not demonstrated any common clinical situations in 

which proton beam therapy has a measurable advantage over photon radiotherapy modalities on 

meaningful long-term health outcomes, but have uncovered the potential for increased harm. The 

only advantages we found were for early toxicities: (1) for medulloblastoma, proton beam may 

reduce risk of 1-month medical management of esophagitis (5% vs 57%; P<.001), > 5% weight 

loss (16% vs 64%; P=.004), and Grade ≥ 2 nausea/vomiting (26% vs 71%; P=.004); (2) for 

esophageal cancer, compared to 3D-CRT, proton beam may lead to a substantial risk reduction in 

post-operative pulmonary complications (aOR 9.13 for 3D-CRT vs PBT; 30.3% vs 18.1%) when 

used in trimodal therapy (neoadjuvant chemoradiation followed by surgical resection; 28.4% vs 

18.1%); (3) for NSCLC, proton beam may reduce risk of 6-month severe esophagitis compared 

to IMRT (6% vs 28%; P<.0001); and (4) for prostate cancer, compared with IMRT, proton beam 

may transiently reduce risk of 6-month GU diagnosis or procedure codes (OR 0.60; 95% CI, 

0.38-0.96; 5.9% vs 9.5%). Increased harms include (1) for partial breast irradiation, various skin 

toxicities are more common with PBT than with 3D-CRT when PBT is delivered in single-fields

(range, 54-90% vs 15-28%), (2) for esophageal cancer, acute pneumonitis was more common 

with PBT than IMRT and 3D-CRT (33% versus 15%; P=.04), (3) for prostate cancer, there is 

variable increased risk for GI toxicity compared with IMRT and 3D-CRT and for late rectal 

bleeding for a high-dose combination of protons plus photons compared with photons alone, and 

(4) for spinal cord gliomas, there is an 18% increased risk of 5-year mortality. We only found 

one clinical scenario in which there was at least low-strength evidence that proton beam is 

comparable to other treatment modalities across outcomes and that was for prostate cancer, in 

which combined proton and photon radiation had similar 8-year overall survival and freedom 

from distant metastasis compared with brachytherapy. Although we found comparative studies in 

giant cell tumors of the bone, head and neck cancer, uveal hemangiomas, and meningiomas, they 

provided insufficient evidence for drawing conclusions. There is insufficient evidence to draw 

conclusions about the comparative effects of PBT versus other radiation modalities among 

patients with recurrent tumors or how the comparative effects of proton and photon beam 

therapies differ according to variation in tumor motion. 

Strength of evidence

Overall, however, we have low confidence that these findings reflect the true overall effects of 

proton beam therapy because the body of comparative studies for proton beam therapy has 

numerous deficiencies. Typical circumstances that can increase our confidence in the certainty of 

the findings are when multiple individual studies within an evidence base have a high degree of 

similarity in the direction or magnitude of effect (consistency) and when the potential for random 

56
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

error is reduced through the use of sufficient sample sizes (precision). But in most cases among 

the comparative studies for proton beam therapy, consistency was unknown because there was 

typically only a single study for each outcome within each cancer type and the effect estimates 

were imprecise because of small sample sizes. The main limitations in study design and conduct 

include that most studies (1) were retrospective and some used historical control groups for the 

photon-based comparator groups, (2) potentially gave proton beam groups an unfair advantage 

by comparing them to photon-based groups with poorer prognostic profiles without accounting 

for the important differences, (3) lacked data on radiation dose and field size, (4) could not 

reliably differentiate toxicity grade, and (4) did not measure many outcomes of greatest interest, 

including recurrence, ability to deliver planned chemotherapy and radiation regimens, functional 

capacity, overall severe late toxicity, and secondary malignancies. 

Applicability

There are 2 characteristics of the studies worth mentioning that may restrict the applicability of 

their findings. First, the majority of the proton beam treatment groups came from one of 3 proton 

facilities that are among the oldest in operation (Loma Linda, MD Anderson, or Massachusetts 

General). It is unclear whether the patient outcomes of these centers would generalize to other 

facilities with less experience treating patients and that may have difference standards of care. 

Second, the majority of the studies include patients that were treated as far back as 1991 to 2003, 

and it is unclear whether their findings would generalize to current standards of care, which may 

reflect improvements in proton beam administration skill levels and other advances. 

Comparison to previous systematic reviews

It is difficult to directly compare our conclusions to those of many previous reviews as most 

included many fewer of the more recently published comparative studies. The 2014 review 

produced by ICER for the Washington State Health Care Authority Health Technology 

Assessment Program (HTA) had the largest overlap with our scope, but our conclusions differed 

greatly from theirs. There is no clear pattern to the discrepancies between our conclusions. We 

did not detect a clear conflict of interest for ICER as their funding statement indicated support 

from various sources, including government grants, non-profit organizations, health plans/ 

provider contributions, manufacturer grants, contracts, contributions, but that it was “not 

accepted from manufacturers or private insurers to perform reviews of specific technologies.” 

The differences in conclusions could also not clearly be attributed to differences in eligibility 

criteria, study inclusion, or methods of assessing internal validity or rating strength of evidence. 

General limitations of our systematic review

As with other types of research, the limitations of this systematic review are important to 

recognize. The generalizability of the results are limited by the scope of the key questions and 

inclusion criteria and by the generalizability of the studies included. The main methodological 

limitation of this review within the defined scope is the exclusion of studies published in 

languages other than English. But it is likely that findings from studies published in non-English 

language journals may have limited applicability to the VA populations. 

57
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Gaps in evidence and future research recommendations

Gaps in evidence include the lack of studies on the newer PBT delivery systems and methods (ie, 

pencil beam scanning), assessment of many important outcomes including recurrence, ability to 

deliver planned chemotherapy and radiation regimens, functional capacity, overall severe late 

toxicity, and secondary malignancies. Also, although there are many studies that have compared 

the dosimetric impact between different methods of accounting for tumor motion in treatment 

planning, including 4DCT imaging, multiphase, free-breathing, or 3DCT imaging, we found no 

studies of how they compare in clinical outcomes. It is clear that further comparative studies are 

needed to address the these gaps in evidence. We will discuss the ideal characteristics of future 

studies using the PICOS framework. 

For population and intervention, to further minimize the risk of confounding, we recommend use 

of greater standardization in measurement of and greater reporting of important patient 

characteristics by treatment group and proton beam dose and delivery parameters and better 

accounting for baseline differences. For outcomes, at minimum, future studies should start with 

using reliable and standardized methods for measuring overall early and late toxicity that account 

for severity grade, rather than focusing an entire study on just one specific toxicity reflecting a 

wide range of severity. Ideally, future studies should measure many outcomes of greatest 

interest, including recurrence, ability to deliver planned chemotherapy and radiation regimens, 

functional capacity, overall severe late toxicity, and secondary malignancies. For setting, to 

improve the generalizability and precision, we recommend using a multi-site design. The Proton 

Therapy Consortium offers an opportunity for its 14 participating proton beam facilities to 

collectively conduct a cooperative study that would combine data across multiple study sites, 

which would increase the overall generalizability of their findings. For study design, it is not 

clear that RCTs are necessary or possible, and well-designed prospective cohort studies may be 

acceptable. Some practitioners of and advocates for PBT have argued that conducting studies 

that randomize patients to PBT or photon-based treatment is unethical.82 Due to the superior dose 

distribution achieved by PBT, they argue, protons can provide therapy superior to photons in 

almost every circumstance and so randomizing patients to receive proton or non-proton therapy 

would preclude the requirement of equipoise needed to ethically conduct a RCT.82 Other 

clinicians and experts in radiation oncology question whether PBT’s dosimetric characteristics 

translate into measurable clinical benefits or increased survival for patients.83 The lack of 

obvious clinical benefit in some observational studies84 lead some to question whether 

conducting RCTs would in fact be unethical.83 We identified 9 ongoing RCTs that are comparing 

proton beam therapy to conventional photon beam therapy (N=3), IMRT (N=5), SBRT (N=1), 

and 3D-CRT (N=1) and include the cancer types of non-small cell lung cancer (NSCLC), 

prostate cancer, meningioma, glioblastoma, esophageal cancer, and oropharyngeal cancer 

patients (see Appendix F). Four of these ongoing studies are multi-site and are being conducted 

at MD Anderson Cancer Center, Massachusetts General Hospital, and the University of 

Pennsylvania as well as other centers in the US and abroad. Many have estimated completion 

dates in the next 2 years. A review of their protocols suggests the potential for some 

improvement in toxicity measurement. For example, NCT01512589, a Phase III randomized trial 

of proton beam therapy versus IMRT for esophageal cancer, has a planned outcome of total 

toxicity burden. However, it is not clear they will fully address all missing outcomes.

58
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

CONCLUSION

Despite the common claim that the advantage of proton beam therapy is self-evident, 

comparative studies have not demonstrated any common clinical situations in which proton beam 

therapy has a measurable advantage over photon radiotherapy modalities on meaningful longterm health outcomes, but have uncovered low-strength evidence of the potential for increased 

late toxicity compared with IMRT and 3D-CRT for breast, esophageal, prostate, and spinal cord 

glioma cancers. Existing comparative studies have numerous methodological deficiencies that 

limited our confidence in their findings and their findings may have limited applicability across 

all US proton beam facilities. Although numerous randomized controlled trials are underway that 

carry the promise of improved toxicity measurement, it is unclear whether they will fully address 

gaps in evidence on other important outcomes including recurrence, ability to deliver planned 

chemotherapy and radiation regimens, functional capacity, overall severe late toxicity, and 

secondary malignancies. Because this is still a rapidly evolving field, with ongoing efforts to 

improve techniques and reduce costs, this review may need more frequent updating to keep upto-date with emerging research. 

59
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

REFERENCES

1. Ollendorf DA, Colby JA, Pearson SD. Proton beam therapy: Final evidence report. 

Institute for Clinical and Economic Review 2014. 

2. Galland-Girodet S, Pashtan I, MacDonald SM, et al. Long-term cosmetic outcomes and 

toxicities of proton beam therapy compared with photon-based 3-dimensional conformal 

accelerated partial-breast irradiation: A phase 1 trial. Int J Radiat Oncol Biol Phys. 2014. 

3. Brown AP, Barney CL, Grosshans DR, et al. Proton beam craniospinal irradiation 

reduces acute toxicity for adults with medulloblastoma. Int J Radiat Oncol Biol Phys. 

2013;86(2):277-284. 

4. Gomez DR, Tucker SL, Martel MK, et al. Predictors of high-grade esophagitis after 

definitive three-dimensional conformal therapy, intensity-modulated radiation therapy, or 

proton beam therapy for non-small cell lung cancer. Int J Radiat Oncol Biol Phys. 

2012;84(4):1010-1016. 

5. Gray PJ, Paly JJ, Yeap BY, et al. Patient-reported outcomes after 3-dimensional 

conformal, intensity-modulated, or proton beam radiotherapy for localized prostate 

cancer. Cancer. 2013;119(9):1729-1735. 

6. Hoppe B, Michalski J, Mendenhall NP, et al. Comparative effectiveness study of patientreported outcomes after proton therapy or intensity-modulated radiotherapy for prostate 

cancer. Cancer. 2014;120(7):1076-1082. 

7. Yu JB, Soulos PR, Herrin J, et al. Proton versus intensity-modulated radiotherapy for 

prostate cancer: patterns of care and early toxicity. J Natl Cancer Inst. 2013;105(1):25-

32. 

8. Fang P, Mick R, Deville C, et al. A case-matched study of toxicity outcomes after proton 

therapy and intensity-modulated radiation therapy for prostate cancer. Cancer. 2014. 

9. Kim S, Shen S, Moore DF, et al. Late gastrointestinal toxicities following radiation 

therapy for prostate cancer. Eur Urol. 2011;60(5):908-916. 

10. Sheets NC, Goldin GH, Meyer AM, et al. Intensity-modulated radiation therapy, proton 

therapy, or conformal radiation therapy and morbidity and disease control in localized 

prostate cancer. JAMA. 2012;307(15):1611-1620. 

11. Coen JJ, Zietman AL, Rossi CJ, et al. Comparison of high-dose proton radiotherapy and 

brachytherapy in localized prostate cancer: A case-matched analysis. Int J Radiat Oncol 

Biol Phys. 2012;82(1):e25-31. 

12. Duttenhaver JR, Shipley WU, Perrone T, et al. Protons or megavoltage X-rays as boost 

therapy for patients irradiated for localized prostatic carcinoma. An early phase I/II 

comparison. Cancer. 1983;51(9):1599-1604. 

60
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

13. Galbraith ME, Ramirez JM, Pedro LW. Quality of life, health outcomes, and identity for 

patients with prostate cancer in five different treatment groups. Oncol Nurs Forum. 

2001;28(3):551-560. 

14. Shipley WU, Verhey LJ, Munzenrider JE, et al. Advanced prostate cancer: the results of a 

randomized comparative trial of high dose irradiation boosting with conformal protons 

compared with conventional dose irradiation using photons alone. Int J Radiat Oncol Biol 

Phys. 1995;32(1):3-12. 

15. Kahn J, Loeffler JS, Niemierko A, Chiocca EA, Batchelor T, Chakravarti A. Long-term 

outcomes of patients with spinal cord gliomas treated by modern conformal radiation 

techniques. Int J Radiat Oncol Biol Phys. 2011;81(1):232-238. 

16. Chung CS, Yock TI, Nelson K, Xu Y, Keating NL, Tarbell NJ. Incidence of second 

malignancies among patients treated with proton versus photon radiation. Int J Radiat 

Oncol Biol Phys. 2013;87(1):46-52. 

17. Mizumoto M, Okumura T, Ishikawa E, et al. Reirradiation for recurrent malignant brain 

tumor with radiotherapy or proton beam therapy. Technical considerations based on 

experience at a single institution. Strahlenther Onkol. 2013;189(8):656-663. 

18. Otsuka M, Ohara K, Takada Y, et al. Radiation therapy for intrahepatic recurrence after 

hepatectomy for hepatocellular carcinoma. Int J Clin Oncol. 2003;8(3):151-155. 

19. Wang L, Hayes S, Paskalev K, et al. Dosimetric comparison of stereotactic body 

radiotherapy using 4D CT and multiphase CT images for treatment planning of lung 

cancer: evaluation of the impact on daily dose coverage. Radiother Oncol. 

2009;91(3):314-324. 

20. Lee JA, Kim CY, Yang DS, et al. Four-dimensional computed tomography based 

respiratory-gated radiotherapy with respiratory guidance system: analysis of respiratory 

signals and dosimetric comparison. Biomed Res Int. 2014. 

21. Lin H, Lu H, Shu L, et al. Dosimetric study of a respiratory gating technique based on 

four-dimensional computed tomography in non-small-cell lung cancer. J Radiat Res. 

2014;55(3):583-588. 

22. Trikalinos TA, Terasawa T, Ip S, Raman G, Lau J. Particle beam radiation therapies for 

cancer. Agency for Healthcare Research and Quality 2009. 

23. Patel RR, Arthur DW. The emergence of advanced brachytherapy techniques for 

common malignancies. Hematol Oncol Clin North Am. 2006;20(1):97-118. 

24. Proton Beam Radiation Therapy (Overview). ECRI Institute 2013. 

25. Efstathiou JA, Gray PJ, Zietman AL. Proton beam therapy and localised prostate cancer: 

current status and controversies. Br J Cancer. 2013;108(6):1225-1230. 

61
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

26. Mitin T, Zietman AL. Promise and pitfalls of heavy-particle therapy. J Clin Oncol.

2014;32(26):2855-2863.

27. Proton Beam Therapy. ASTRO Model Policies 2014.

28. Jarosek S, Elliott S, Virnig BA. Proton beam radiotherapy in the U.S. Medicare

population: growth in use between 2006 and 2009: Data Points # 10. Data Points

Publication Series 2011.

29. Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration's tool for

assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

30. McDonagh MS, Jonas DE, Gartlehner G, et al. Methods for the drug effectiveness review

project. BMC Med Res Methodol. 2012;12:140.

31. Shea BJ, Hamel C, Wells GA, et al. AMSTAR is a reliable and valid measurement tool to

assess the methodological quality of systematic reviews. J Clin Epidemiol.

2009;62(10):1013-1020.

32. Berkman ND, Lohr KN, Ansari M, et al. Grading the strength of a body of evidence

when assessing health care interventions for the Effective Health Care Program of the

Agency for Healthcare Research and Quality: An update methods guide for effectiveness

and comparative effectiveness reviews. Agency for Healthcare Research and Quality

2013. 

33. McCurdy M, Bergsma DP, Hyun E, et al. The role of lung lobes in radiation pneumonitis

and radiation-induced inflammation in the lung: A retrospective study. J Radiat Oncol.

2013;2(2):203-208.

34. Sejpal S, Komaki R, Tsao A, et al. Early findings on toxicity of proton beam therapy with

concurrent chemotherapy for nonsmall cell lung cancer. Cancer. 2011;117(13):3004-

3013. 

35. Solares CA, Fakhri S, Batra PS, Lee J, Lanza DC. Transnasal endoscopic resection of

lesions of the clivus: a preliminary report. Laryngoscope. 2005;115(11):1917-1922.

36. Wang J, Wei C, Tucker SL, et al. Predictors of postoperative complications after

trimodality therapy for esophageal cancer. Int J Radiat Oncol Biol Phys. 2013;86(5):885-

891. 

37. Arvold ND, Lessell S, Bussiere M, et al. Visual outcome and tumor control after

conformal radiotherapy for patients with optic nerve sheath meningioma. Int J Radiat

Oncol Biol Phys. 2009;75(4):1166-1172.

38. Hocht S, Wachtlin J, Bechrakis NE, et al. Proton or photon irradiation for hemangiomas

of the choroid? A retrospective comparison. Int J Radiat Oncol Biol Phys.

2006;66(2):345-351.

62
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

39. Chakravarti A, Spiro IJ, Hug EB, Mankin HJ, Efird JT, Suit HD. Megavoltage radiation 

therapy for axial and inoperable giant-cell tumor of bone. J Bone Joint Surg Am. 

1999;81(11):1566-1573. 

40. Hug EB, Devries A, Thornton AF, et al. Management of atypical and malignant 

meningiomas: role of high-dose, 3D-conformal radiation therapy. J Neurooncol. 

2000;48(2):151-160. 

41. Blue Cross Blue Shield Association. Proton beam therapy for prostate cancer. Executive 

Summary. Technol Eval Cent Assess Program 2011;25(10):1-4. 

42. Blue Cross Blue Shield Association. Proton beam therapy for non-small-cell lung cancer. 

Executive Summary. Technol Eval Cent Assess Program 2011;25(7):1-3. 

43. Proton beam therapy for cancers of the brain, head, neck, and skull base. ECRI Institute

2013. 

44. Intensity-modulated radiation therapy compared with proton beam radiation therapy for 

treating prostate cancer. ECRI Institute 2014. 

45. Allen A, Pawlicki T, al. E. An evidence based review of proton beam therapy: The report 

of ASTRO's emerging technology committee. Radiother Oncol. 2012:8-11. 

46. Amichetti M, Amelio D, Cianchetti M, Enrici RM, Minniti G. A systematic review of 

proton therapy in the treatment of chondrosarcoma of the skull base. Neurosurg Rev. 

2010;33(2):155-165. 

47. Cianchetti M, Amichetti M. Sinonasal malignancies and charged particle radiation 

treatment: a systematic literature review. Int J Otolaryngol. 2012. 

48. De Ruysscher D, Mark Lodge M, Jones B, et al. Charged particles in radiotherapy: a 5-

year update of a systematic review. Radiother Oncol. 2012;103(1):5-7. 

49. Grimm P, Billiet I, Bostwick D, et al. Comparative analysis of prostate-specific antigen 

free survival outcomes for patients with low, intermediate and high risk prostate cancer 

treatment by radical therapy. Results from the Prostate Cancer Results Study Group. BJU 

Int. 2012;109 Suppl 1:22-29. 

50. Grutters JP, Kessels AG, Pijls-Johannesma M, De Ruysscher D, Joore MA, Lambin P. 

Comparison of the effectiveness of radiotherapy with photons, protons and carbon-ions 

for non-small cell lung cancer: a meta-analysis. Radiother Oncol. 2010;95(1):32-40. 

51. Patel SH, Wang Z, Wong WW, et al. Charged particle therapy versus photon therapy for 

paranasal sinus and nasal cavity malignant diseases: A systematic review and metaanalysis. Lancet Oncol. 2014;15(9):1027-1038. 

52. Ramaekers BL, Pijls-Johannesma M, Joore MA, et al. Systematic review and metaanalysis of radiotherapy in various head and neck cancers: Comparing photons, carbonions and protons. Cancer Treat Rev. 2011;37(3):185-201. 

63
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

53. Sun F, Oyesanmi O, Fontanarosa J, Reston J, Guzzo T, Schoelles K. Therapies for 

clinically localized prostate cancer: Update of a 2008 systematic review. Agency for 

Healthcare Research and Quality 2014. 

54. Walsh J. Proton therapy for prostate cancer. California Technology Assessment Forum 

2012. 

55. Bush DA, Do S, Lum S, et al. Partial breast radiation therapy with proton beam: 5-year 

results with cosmetic outcomes. Int J Radiat Oncol Biol Phys. 2014. 

56. Darby SC, Ewertz M, McGale P, et al. Risk of ischemic heart disease in women after 

radiotherapy for breast cancer. N Engl J Med. 2013;368(11):987-998. 

57. Romesser PB, Cahlon O, Scher E, et al. PBRT results in reduced acute toxicity compared 

with IMRT when treating the ipsilateral neck in HNC. Paper presented at: Particle 

Therapy Cooperative Group North America; October 27-29, 2014; Houston, Texas. 

58. Chang JY, Komaki R, Lu C, et al. Phase 2 study of high-dose proton therapy with 

concurrent chemotherapy for unresectable stage III nonsmall cell lung cancer. Cancer. 

2011;117(20):4707-4713. 

59. Chang JY, Komaki R, Wen HY, et al. Toxicity and patterns of failure of adaptive/ablative 

proton therapy for early-stage, medically inoperable non-small cell lung cancer. Int J 

Radiat Oncol Biol Phys. 2011;80(5):1350-1357. 

60. Hoppe BS, Flampouri S, Henderson RH, et al. Proton therapy with concurrent 

chemotherapy for non-small-cell lung cancer: Technique and early results. Clin Lung 

Cancer. 2012;13(5):352-358. 

61. Iwata H, Demizu Y, Fujii O, et al. Long-term outcome of proton therapy and carbon-ion 

therapy for large (T2a-T2bN0M0) non-small-cell lung cancer. J Thorac Oncol. 

2013;8(6):726-735. 

62. Iwata H, Murakami M, Demizu Y, et al. High-dose proton therapy and carbon-ion 

therapy for stage I nonsmall cell lung cancer. Cancer. 2010;116(10):2476-2485. 

63. Nakayama H, Sugahara S, Tokita M, et al. Proton beam therapy for patients with 

medically inoperable stage I non-small-cell lung cancer at the university of tsukuba. Int J 

Radiat Oncol Biol Phys. 2010;78(2):467-471. 

64. Oshiro Y, Mizumoto M, Okumura T, et al. Results of proton beam therapy without 

concurrent chemotherapy for patients with unresectable stage III non-small cell lung 

cancer. J Thorac Oncol. 2012;7(2):370-375. 

65. Ohri N, Dicker AP, Showalter TN. Late toxicity rates following definitive radiotherapy 

for prostate cancer. Can J Urol. 2012;19(4):6373-6380. 

66. Zietman AL, Bae K, Slater JD, et al. Randomized trial comparing conventional-dose with 

high-dose conformal radiation therapy in early-stage adenocarcinoma of the prostate: 

64
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

long-term results from proton radiation oncology group/american college of radiology 

95-09. J Clin Oncol. 2010;28(7):1106-1111. 

67. Deville C, Ben-Josef E, Vapiwala N. Radiation therapy modalities for prostate cancer. 

JAMA. 2012;308(5):451. 

68. Jacobs BL, Zhang Y, Hollenbeck BK. Radiation therapy modalities for prostate cancer. 

JAMA. 2012;308(5):450. 

69. Mendenhall NP, Schild S, Slater J. Radiation therapy modalities for prostate cancer. 

JAMA. 2012;308(5):450-451. 

70. Zietman AL, DeSilvio ML, Slater JD, et al. Comparison of conventional-dose vs highdose conformal radiation therapy in clinically localized adenocarcinoma of the prostate: a 

randomized controlled trial. JAMA. 2005;294(10):1233-1239. 

71. Jabbari S, Weinberg VK, Shinohara K, et al. Equivalent biochemical control and 

improved prostate-specific antigen nadir after permanent prostate seed implant 

brachytherapy versus high-dose three-dimensional conformal radiotherapy and high-dose 

conformal proton beam radiotherapy boost. Int J Radiat Oncol Biol Phys. 2010;76(1):36-

42. 

72. Rossi C. Conformal proton beam radiation therapy for prostate cancer: concepts and 

clinical results. Community Oncology. 2007;4(4):235-240. 

73. Galalae RM, Martinez A, Nuernberg N, et al. Hypofractionated conformal HDR 

brachytherapy in hormone naive men with localized prostate cancer. Is escalation to very 

high biologically equivalent dose beneficial in all prognostic risk groups? Strahlenther 

Onkol. 2006;182(3):135-141. 

74. Bekelman JE, Schultheiss T, Berrington De Gonzalez A. Subsequent malignancies after 

photon versus proton radiation therapy. Int J Radiat Oncol Biol Phys. 2013;87(1):10-12. 

75. DeLaney TF, Liebsch NJ, Pedlow FX, et al. Long-term results of Phase II study of high 

dose photon/proton radiotherapy in the management of spine chordomas, 

chondrosarcomas, and other sarcomas. J Surg Oncol. 2014;110(2):115-122. 

76. Lin R, Slater JD, Yonemoto LT, et al. Nasopharyngeal carcinoma: repeat treatment with 

conformal proton therapy--dose-volume histogram analysis. Radiology. 1999;213(2):489-

494. 

77. Abei M, Okumura T, Fukuda K, et al. A phase I study on combined therapy with protonbeam radiotherapy and in situ tumor vaccination for locally advanced recurrent 

hepatocellular carcinoma. Radiat Oncol. 2013;8:239. 

78. McAvoy SA, Ciura KT, Rineer JM, et al. Feasibility of proton beam therapy for 

reirradiation of locoregionally recurrent non-small cell lung cancer. Radiother Oncol. 

2013;109(1):38-44. 

65
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

79. Hojo H, Zenda S, Akimoto T, et al. Impact of early radiological response evaluation on 

radiotherapeutic outcomes in the patients with nasal cavity and paranasal sinus 

malignancies. J Radiat Res. 2012;53(5):704-709. 

80. Mizumoto M, Okumura T, Hashimoto T, et al. Proton beam therapy for hepatocellular 

carcinoma: a comparison of three treatment protocols. Int J Radiat Oncol Biol Phys. 

2011;81(4):1039-1045. 

81. Liu HH, Balter P, Tutt T, et al. Assessing respiration-induced tumor motion and internal 

target volume using four-dimensional computed tomography for radiotherapy of lung 

cancer. Int J Radiat Oncol Biol Phys. 2007;68(2):531-540. 

82. Goitein M, Cox JD. Should randomized clinical trials be required for proton 

radiotherapy? J Clin Oncol. 2008;26(2):175-176. 

83. Glatstein E, Glick J, Kaiser L, Hahn SM. Should randomized clinical trials be required 

for proton radiotherapy? An alternative view. J Clin Oncol. 2008;26(15):2438-2439. 

84. Brada M, Pijls-Johannesma M, De Ruysscher D. Proton therapy in clinical practice: 

current clinical evidence. J Clin Oncol. 2007;25(8):965-970. 

66
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

APPENDIX A. PROTON BEAM THERAPY COVERAGE POLICIES

Organization Title Last 

Review

Policy

Aetna Clinical Policy 

Bulletin: 

Proton Beam 

and Neutron 

Beam 

Radiotherapy

8/8/2014 Aetna considers proton beam radiotherapy medically necessary in any of the following 

radiosensitive tumors: A. Chordomas or chondrosarcomas arising at the base of the skull or 

cervical spine without distant metastases; or B. Malignancies in children (21 years of age and 

younger); or C. Uveal melanomas confined to the globe (ie, not distant metastases) (the uvea is 

comprised of the iris, ciliary body, and choroid [the vascular middle coat of the eye]). 

Aetna considers proton beam radiotherapy for treatment of prostate cancer not medically 

necessary for individuals with localized prostate cancer because it has not been proven to be 

more effective than other radiotherapy modalities for this indication. Proton beam therapy for 

metastatic prostate cancer is considered experimental and investigational.

Aetna considers proton beam radiotherapy experimental and investigational for all other 

indications.

American 

Society for 

Radiation 

Oncology 

(ASTRO)

Model Policy: 

Proton Beam 

Therapy

5/20/2014 The Model Policy lists the following disease sites that support the use of PBT: (1) Ocular 

tumors, including intraocular melanomas; (2) Tumors that approach or are located at the base of 

skull, including but not limited to chordoma and chondrosarcomas; (3) Primary or metastatic 

tumors of the spine where the spinal cord tolerance may be exceeded with conventional 

treatment or where the spinal cord has previously been irradiated; (4) Primary hepatocellular 

cancer treated in a hypofracitonated regimen; (5) Primary or benign solid tumors in children 

treated with curative intent and occasional palliative treatment of childhood tumors when at least 

one of the four criteria noted above apply; (6) Patients with genitive syndromes making total 

volume of radiation minimization crucial such as but not limited to NF-1 patients and 

retinoblastoma patients.

All other indications...are suitable for Coverage with Evidence Development (CED). Radiation 

therapy for patients treated under the CED paradigm should be covered by the insurance carrier 

as long as the patient is enrolled in either an IRB-approved clinical trial or in a multi-institutional 

patient registry adhering to Medicare requirements for CED. At this time, no indications are 

deemed inappropriate for CED and therefore...includes various systems such as, but not limited 

to, the following: head and neck malignancies, thoracic malignancies, abdominal malignancies, 

pelvic malignancies including genitourinary, gynecologic, and gastrointestinal carcinomas.

67
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Organization Title Last 

Review

Policy

Anthem Proton Beam 

Radiation 

Therapy 

Medical Policy

5/15/2014 All Conditions other than Localized Prostate Cancer

Proton beam radiation therapy, with or without stereotactic techniques, is considered medically 

necessary for the following conditions: (1) As primary therapy for melanoma of the uveal tract 

(iris, choroid, or ciliary body) involving tumors of up to 24 mm in largest diameter and 14 mm in 

height, and with no evidence of metastasis or extrascleral extension; (2) As postoperative 

therapy for individuals who have undergone biopsy or partial resection of a chordoma or lowgrade (I or II) chondrosarcoma of the basisphenoid region (for example, skull-base chordoma or 

chondrosarcoma) or cervical spine and have residual, localized tumor without evidence of 

metastasis; (3) Pituitary adenoma when conventional stereotactic radiation is not an available 

option; (4) Intracranial arteriovenous malformation (AVM) not amenable to surgical excision or 

other conventional forms of treatment; (5) Central nervous system (CNS) lesions including but 

not limited to, primary or metastatic CNS malignancies or arteriovenous malformations, adjacent 

to critical structures such as the optic nerve, brain stem or spinal cord. 

Proton beam radiation therapy is considered not medically necessary for the treatment of 

choroidal neovascularization secondary to age-related macular degeneration (AMD).

Proton beam radiation therapy is considered investigational and not medically necessary for all 

other indications not meeting the criteria above. 

Localized Prostate Cancer

Proton beam radiation therapy is considered medically necessary for the initial monotherapy 

radiation treatment of localized prostate cancer.

The use of proton beam radiation as dose escalation therapy, in conjunction with stereotactic 

radiosurgery, IMRT, three-dimensional conformal radiation therapy (3D CRT), or brachytherapy 

for the treatment of localized prostate cancer is considered investigational and not medically 

necessary.

Proton beam radiation therapy is considered investigational and not medically necessary for the 

treatment of prostate cancer for all other indications not meeting the criteria above.

68
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Organization Title Last 

Review

Policy

BlueCross 

BlueShield of 

North Carolina

Corporate 

Medical Policy 

Charged 

Particle 

Radiotherapy 

(Proton or 

Helium Ion)

6/2014 Charged particle irradiation with proton or helium ion beams may be considered medically 

necessary for the following clinical indications: (1) primary therapy for melanoma of the uveal 

tract (iris, choroid or ciliary body), with no evidence of metastasis or extrascleral extension, and 

with tumors up to 24 millimeters in largest diameter and 14 millimeters in height; (2) postoperative therapy (with or without conventional high energy X-rays) in patients who have 

undergone biopsy or a partial resection of chordoma or low grade (I or II) chondrosarcoma of the 

basisphenoid region (skull-base chordoma or chondrosarcoma) or cervical spine. Patients 

eligible for this treatment have residual localized tumor without evidence of metastasis; (3) In 

the treatment of pediatric central nervous system (CNS) tumors.

Charged particle irradiation with proton beams using standard treatment doses is considered not 

medically necessary and therefore non -covered in patients with clinically localized prostate 

cancer, because the clinical outcomes with this treatment have not been shown to be superior to 

other approaches including intensity modulated radiation therapy (IMRT) or conformal radiation 

therapy. 

Charged particle irradiation is considered investigational for all other indications not addressed 

above under, When Charged Particle Radiotherapy is covered

BlueCross 

BlueShield of 

California

ChargedParticles 

(Proton or 

Helium) 

Radiation 

Therapy 

Medical Policy

10/28/2013 Charged-particle irradiation with proton or helium ion beams may be considered medically 

necessary and a covered benefit in any of the following clinical situations: (1) Primary therapy 

for melanoma of the uveal tract (iris, choroid, or ciliary body) and both of the following: a. No 

evidence of metastasis or extrascleral extension, b. Tumor size up to 24 millimeters in largest 

diameter and 14 millimeters in height; (2) Postoperative therapy (with or without conventional 

high-energy x-rays) for residual localized tumor without metastasis in patients who have 

undergone biopsy or partial resection of one of the following: a. Chordoma, b. Low-grade (I or 

II) chondrosarcoma of the basisphenoid region (eg, skull-base chordoma or chondrosarcoma) or 

cervical spine; (3) Treatment of pediatric central nervous system tumors.

Charged-particle irradiation with proton or helium beams is generally not a covered service for 

prostate cancer (clinically localized prostate cancer) because it is not cost-effective.

Other applications of charged-particle irradiation with proton beams are considered 

investigational.

69
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Organization Title Last 

Review

Policy

CIGNA Proton Beam 

Therapy for 

Intracranial 

and Skull Base 

Tumors

Unknown CIGNA HealthCare covers proton beam therapy as medically necessary for the treatment of 

intracranial and skull base tumors when radiation therapy is indicated and recommended by the 

treating physician. 

CIGNA Proton Beam 

Therapy for 

Ocular 

Melanoma, 

Ocular 

Hemangiomas 

and Macular 

Degeneration

Unknown CIGNA HealthCare covers proton beam therapy as medically necessary for the treatment of 

melanoma of the uveal tract (ie, iris, ciliary body and choroid). 

CIGNA HealthCare does not cover proton beam therapy for choroidal hemangiomas or macular 

degeneration, because it is considered experimental, investigational or unproven. 

CIGNA Proton Beam 

Therapy for 

Prostate 

Cancer

Unknown CIGNA HealthCare considers proton beam therapy to be clinically equivalent to conventional 

external beam radiation therapy for the treatment of localized prostate cancer (ie, cancer that is 

confined to the prostate), but does not consider it to be clinically superior to conventional 

external beam radiation therapy. 

Coverage for proton beam therapy for the treatment of localized prostate cancer may depend 

upon the applicable health benefit plan definition of medical necessity. Many health benefit 

plans administered by CIGNA HealthCare contain definitions of medical necessity which 

include a cost comparison component. Because proton beam therapy for the treatment of prostate 

cancer is significantly more expensive than conventional external beam radiation therapy but is 

not clinically superior, it is considered not medically necessary under those plans. For health 

benefit plans which contain definitions of medical necessity that do not include a cost 

comparison component, proton beam therapy may be covered as medically necessary for the 

treatment of localized prostate cancer (ie, cancer that is confined to the prostate).

CIGNA Proton Beam 

Therapy for 

Lung Cancer

Unknown Cigna does not cover proton beam therapy for the treatment of lung cancer because it is 

considered experimental, investigational or unproven. 

70
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Organization Title Last 

Review

Policy

HealthPartners Proton Beam 

Radiation 

Therapy

7/2014 Indications that are covered: (1) Melanoma of the uveal tract that is not amenable to surgical 

excision or other conventional forms of treatment; (2) Chordomas or chondrosarcomas arising at 

the base of the skull or along the axial skeleton without distant metastases; (3) Pituitary 

neoplasms; (4) Other central nervous system tumors located near vital structures; (5) Salivary 

gland tumors. 

Indications not covered: (1) Prostate cancer; (2) Hepatocellular cancer; (3) Lung cancer; (4) 

Bladder cancer; (5) Breast cancer; (6) Esophageal cancer; (7) Cervical cancer; (8) Age-related 

maculardegeneration (AMD); (9) Chorical hemangiomas; (10) Non-uveal melanoma; (11) 

Parotid gland tumor; (12) Colon cancer; (13) Kidney cancer; (14) Pancreatic cancer; (15) Rectal 

cancer; (16) Soft tissue sarcomas.

Medica Proton Beam 

Radiation 

Therapy

6/2013 Proton beam radiation therapy is indicated for individuals with conditions not amenable to 

surgical excision or other conventional forms of treatment AND have one of the following 

diagnoses: A. Chordomas or chondrosarcomas arising at the base of the skull or along the axial 

skeleton without distant metastasis; B. Pediatric central nervous system tumors adjacent to vital 

structures (eg optic nerve, spinal cord); C. Melanoma of the uveal tract (iris, choroid, ciliary 

body) without extrascleral extension and with no evidence of metastasis.

Proton beam radiation therapy for hepatocellular cancer, prostate cancer or non-small cell lung 

cancer (NSCLC) is investigative and therefore not covered.

71
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Organization Title Last 

Review

Policy

Regence ChargedParticle 

(Proton or 

Helium Ion) 

Radiation 

Therapy

6/2014 Charged-particle irradiation with proton or helium ion beams may be considered medically 

necessary in the following clinical situations: A. Primary therapy for melanoma of the uveal tract 

(iris, choroid, or ciliary body), with no evidence of metastasis or extrascleral extension, and with 

tumors up to 24 mm in largest diameter and 14 mm in height; B. Postoperative therapy (with or 

without conventional high-energy x-rays) in patients who have undergone biopsy or partial 

resection of the chordoma or low-grade (I or II) chondrosarcoma of the basisphenoid region 

(skull-base chordoma or chondrosarcoma) or cervical spine. Patients eligible for this treatment 

have residual localized tumor without evidence of metastasis.; C. In the treatment of pediatric 

(less than 21 years of age) central nervous system tumors and retinoblastoma.

Charged-particle irradiation with proton beams is considered not medically necessary in patients 

with clinically localized prostate cancer.

Other applications of charged-particle irradiation are considered investigational.

United 

Healthcare

Medical 

Policy: Proton 

Beam 

Radiation 

Therapy

9/2014 Proton beam radiation therapy is proven and medically necessary for the following indications: 

(1) Intracranial arteriovenous malformations (AVMs); (2) Ocular tumors, including 

intraocular/uveal melanoma (includes the iris, ciliary body and choroid); (3) Skull-based tumors 

(eg, chordomas or chondrosarcomas). 

Proton beam radiation therapy is unproven and not medically necessary for treating ALL other 

indications.

72
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Organization Title Last 

Review

Policy

Wellmark Proton Beam 

Radiation 

Therapy

1/2014 Proton beam therapy may be considered medically necessary for the following conditions: (1) 

Primary therapy for melanoma of the uveal tract (ie iris, choroid, or ciliary body), with no 

evidence of metastasis or extrascleral extension and with tumors up to 24mm in largest diameter 

and 14mm in height; (2) Post operative therapy (with or without conventional high energy x-ray) 

in patients who have under gone biopsy or partial resection of the chordoma or low grade (I or 

II) chondrosarcoma of the basisphenoid region (skull-base chordoma or chondrosarcoma) or 

cervical spine. Patients eligible for this treatment have residual localized tumor without evidence 

of metastasis. (3) Osteosarcoma for patients with unresectable or incompletely resected 

osteosarcoma; (4) Central nervous system tumors (ie tumors within the skull, skull based tumors 

and spinal cord tumors); (5) Intracranial arteriovenous malformations, small lesions when 

surgery may be associated with increased risk based on anatomic location or feeding vessel 

anatomy; (6) Soft tissue sarcoma. Postoperative use in retroperitoneal soft tissue sarcoma not 

amenable to other radiotherapy (eg, IMRT, stereotactic body radiotherapy) in persons who have 

not received preoperative radiotherapy. 

Proton beam therapy as a treatment for prostate cancer is considered not medically necessary 

because the clinical outcomes with this treatment have not been shown to be superior or more 

effective than the other radiotherapy modalities for this indication.

Proton beam radiation therapy is considered investigational for all other indications not meeting 

the criteria above.

73
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Organization Population Title Year Statement

Alberta Health 

Services, 

Cancer Care

All 

diagnoses 

Proton beam 

radiation therapy

2013 “Adult tumour sites that may be considered for referral for out-of-country proton 

beam radiotherapy include: 

a. The following CNS tumours or lesions: arteriovenous 

malformations, benign meningioma, neuromas, craniopharyngioma, 

CNS germ cell tumours, and low grade gliomas 

b. The following non-CNS tumours: sarcoma including chordoma and 

chondrosarcoma, lymphoma in patients under the age of 30 years, 

and paranasal sinus and nasal cavity tumours. 

Adult, pediatric, and adolescent patients with ocular melanomas requiring proton 

beam radiotherapy should be sent to the TRIUMF Proton Treatment Facility in 

Vancouver, British Columbia (BC) for treatment. 

Members of the working group do not currently recommend that patients with 

prostate cancer, non-small cell lung cancer, or most lymphomas be referred for 

proton beam radiotherapy, due to an insufficient evidence base. However, individual 

patient cases should be discussed by the multidisciplinary team during a Tumour 

Board meeting.”

American 

College of 

Radiology

Stage T1 

and T2 

prostate 

cancer

ACR 

Appropriateness 

Criteria® 

definitive 

external-beam 

irradiation in 

stage T1 and T2 

prostate cancer.

2013 “There are only limited data comparing proton beam therapy to other methods of 

irradiation or to radical prostatectomy for treating stage T1 and T2 prostate cancer. 

Further studies are needed to clearly define its role for such treatment.”

American 

College of 

Radiology

N2 nonsmall cell 

lung cancer 

ACR 

Appropriateness 

Criteria® 

induction and 

adjuvant therapy 

for N2 non–

small-cell lung 

cancer.

2013 “The utility of intensity-modulated radiation therapy (IMRT) or protons to 

potentially further reduce normal tissue toxicity remains to be explored.” 

APPENDIX B. CLINICAL PRACTICE GUIDELINES IN OTHER ORGANIZATIONS 

74
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Organization Population Title Year Statement

American 

College of 

Radiology

Lymphoma ACR 

Appropriateness 

Criteria® 

localized nodal 

indolent 

lymphoma.

2013 “Advanced radiation techniques, such as IMRT and proton therapy, may be 

considered depending on the clinical scenario and whether an improvement in the 

therapeutic ratio is expected.”

American 

College of 

Radiology

Cervical 

cancer

ACR 

Appropriateness 

Criteria® 

advanced 

cervical cancer.

2012 “The combined use of imaging, advanced radiotherapeutic modalities, and 

chemotherapy has led to better treatment for cancer of the cervix.” 

Particle therapy is included as one of the external beam radiotherapies.

American 

College of 

Radiology

Prostate 

cancer

ACR 

Appropriateness 

Criteria® 

external beam 

radiation therapy 

treatment 

planning for 

clinically 

localized prostate 

cancer.

2011 Proton beam therapy is “usually appropriate” (rating of 7) for treating prostate 

cancer.

American 

College of 

Radiology

Breast 

cancer

ACR 

Appropriateness 

Criteria® locally 

advanced breast 

cancer.

2011 “Maximal cardiac sparing achieved through proton therapy has the potential to 

decrease [common treatment-related toxicities].”

American 

College of 

Radiology

Brain cancer ACR 

Appropriateness 

Criteria® preirradiation 

evaluation and 

management of 

brain metastases.

2011 Not mentioned

75
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Organization Population Title Year Statement

American 

College of 

Radiology

Head/neck 

cancer

ACR 

Appropriateness 

Criteria® 

retreatment of 

recurrent head 

and neck cancer 

after prior 

definitive 

radiation.

2010 “Experience with nasopharyngeal retreatment has included combinations of 

nasopharyngectomy, chemotherapy, external beam radiation therapy (EBRT), 

brachytherapy, intraoperative radiotherapy, hyperthermia, radiosurgery, and proton 

therapy.” 

BlueCross 

BlueShield

Prostate 

cancer

Proton Beam 

Therapy for 

Prostate Cancer

2014 “Based on the above, proton beam therapy as a boost to photon external-beam 

radiotherapy or proton beam therapy without photon external-beam radiotherapy in 

the treatment of prostate cancer does not meet the TEC criteria.”

BlueCross 

BlueShield

Non-small 

cell lung 

cancer

Proton Beam 

Therapy for 

Prostate Cancer

2014 “Proton beam radiotherapy for treatment of non-small cell lung cancer at any stage 

or for recurrent non-small cell lung cancer does not meet the TEC criteria.”

National 

Comprehensive 

Cancer 

Network

Prostate 

Cancer

NCCN 

Guidelines 

Version 1.2015 

Prostate Cancer

2014 “An ongoing prospective randomized trial is accruing patients and comparing 

prostate proton therapy to prostate IMRT. The NCCN panel believes there is no clear 

evidence supporting a benefit or decrement to proton therapy over IMRT for either 

treatment efficacy or long-term toxicity. Conventionally fractionated prostate proton 

therapy can be considered a reasonable alternative to X-ray based regimens at clinics 

with appropriate technology, physics, and clinical expertise.”

76
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Concept Search Terms Comments

All Cancers neoplasm* OR cancer* OR 

carcinom*

Key words in title/abstract

Proton Beam Therapy proton OR "proton beam" OR 

"proton beam therapy" OR proton* 

OR proton* therap* OR 

protontherap*

Key words in title/abstract

"Protons"[Mesh] OR "Proton 

Therapy"[Mesh]

MeSH terms 

Limits AND Humans[Mesh] AND 

English[lang] 

Removes non-English 

language articles and animal 

studies

NOT (Comment[ptyp] OR 

Letter[ptyp] OR Review[ptyp]) 

Removes publication types 

that are not studies. (this is 

more sensitive than restricting 

the search to RCTs)

NOT ("Proton Pump 

Inhibitors"[Mesh] OR "Proton 

Pump Inhibitors" [Pharmacological 

Action])

Removes Proton Pump 

Inhibitor Studies 

NOT AND (infant[MeSH] OR 

child[MeSH] OR 

adolescent[MeSH])

Removes Children from 

search (safer than limiting to 

adults only)

Total Results = 1,964

APPENDIX C. SEARCH STRATEGIES

MEDLINE® via PubMed® searched on December 10, 2014 

77
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Cochrane Central Register of Controlled Trials via OVID searched on December 10, 2014 

Step Term Results

1 exp Protons/ 106

2 proton.mp. 2,172

3 proton beam.mp. 37

4 proton beam therapy.mp. 7

5 exp Proton Therapy/ 3

6 proton*.mp. 2,239

7 proton$ therap$.mp. 19

8 protontherap$.mp. 0

9 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 2,239

10 (neoplasm* or cancer* or carcinoma*).mp. 68,072

11 9 and 10 169

12 limit 11 to english language 146

13 Proton Pump Inhibitors/ 811

14 12 not 13 111

15 limit 14 to (editorial or letter or "review") 0

16 14 not 15 111

Term Results

neoplasm* or cancer* or carcinoma* | proton therapy | Adult, Senior 50

ClinicalTrials.gov searched on December 10, 2014 

78
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

MEDLINE® via PubMed® searched on December 10, 2014 with special emphasis on 

selected journals without any limitations for cancer type for proton beam therapy

Concept Search Terms Comments

Proton Beam 

Therapy 

proton[tiab]

AND 

irradiation OR radiation OR radiotherap* OR therap* OR 

beam[tiab]

Keywords in title or 

abstract

OR

"Proton Therapy"[Mesh] OR "Protons/therapeutic 

use"[Mesh]

Medical Subject terms

Selected Journals "Acta oncologica (Stockholm, Sweden)"[Journal]

OR "American journal of clinical oncology"[Journal]

OR "BMC cancer"[Journal]) OR "Cancer"[Journal] 

OR "Cancer journal (Sudbury, Mass.)"[Journal]

OR "International journal of radiation oncology, biology, 

physics"[Journal] 

OR "J Clin Oncol"[Journal] 

OR "J Surg Oncol"[Journal] 

OR "JAMA : the journal of the American Medical 

Association"[Journal] 

OR "J Thorac Oncol"[Journal] 

OR "Jpn J Clin Oncol"[Journal] 

OR "Radiat Oncol"[Journal] 

OR “Radiother Oncol” [Journal]

OR "Strahlenther Onkol"[Journal] 

OR "Urology"[Journal]

Top journals from 

previous search

Total Results = 927

79
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Citation Reason for Exclusion

Demizu Y, Murakami M, Miyawaki D, et al. Analysis of Vision loss 

caused by radiation-induced optic neuropathy after particle therapy for 

head-and-neck and skull-base tumors adjacent to optic nerves. Int J 

Radiat Oncol Biol Phys. Dec 1 2009;75(5):1487-1492.

Ineligible comparator: carbon 

ion therapy 

Desjardins L, Lumbroso-Le Rouic L, Levy-Gabriel C, et al. Combined 

proton beam radiotherapy and transpupillary thermotherapy for large 

uveal melanomas: a randomized study of 151 patients. Ophthalmic Res. 

2006;38(5):255-260.

Ineligible population: uveal 

melanoma patients

Fujii O, Demizu Y, Hashimoto N, et al. A retrospective comparison of 

proton therapy and carbon ion therapy for stage I non-small cell lung 

cancer. Radiother Oncol. Oct 2013;109(1):32-37.

Ineligible comparator: carbon 

ion therapy 

Gragoudas ES, Lane AM, Regan S, et al. A randomized controlled trial 

of varying radiation doses in the treatment of choroidal melanoma. 

Arch Ophthalmol. Jun 2000;118(6):773-778.

Ineligible population: uveal 

melanoma patients

Jabbari S, Weinberg VK, Shinohara K, et al. Equivalent biochemical 

control and improved prostate-specific antigen nadir after permanent 

prostate seed implant brachytherapy versus high-dose threedimensional conformal radiotherapy and high-dose conformal proton 

beam radiotherapy boost. Int J Radiat Oncol Biol Phys. Jan 1 

2010;76(1):36-42.

Ineligible outcomes: interval to 

reach PSA nadir, PSA level, 

biological no evidence of 

disease

Komatsu S, Fukumoto T, Demizu Y, et al. Clinical results and risk 

factors of proton and carbon ion therapy for hepatocellular carcinoma. 

Cancer. Nov 1 2011;117(21):4890-4904.

Ineligible comparator: carbon 

ion therapy 

Lopez Guerra JL, Gomez DR, Zhuang Y, et al. Changes in pulmonary 

function after three-dimensional conformal radiotherapy, intensitymodulated radiotherapy, or proton beam therapy for non-small-cell 

lung cancer. Int J Radiat Oncol Biol Phys. Jul 15 2012;83(4):e537-543.

Ineligible outcome: decline in 

lung diffusing capacity for 

carbon monoxide 

Matsuzaki Y, Osuga T, Chiba T, et al. New, effective treatment using 

proton irradiation for unresectable hepatocellular carcinoma. Intern 

Med. Apr 1995;34(4):302-304.

Ineligible comparator: proton 

beam therapy plus 

chemotherapy

Miyawaki D, Murakami M, Demizu Y, et al. Brain injury after proton 

therapy or carbon ion therapy for head-and-neck cancer and skull base 

tumors. Int J Radiat Oncol Biol Phys. Oct 1 2009;75(2):378-384.

Ineligible comparator: carbon 

ion therapy 

Park L, Delaney TF, Liebsch NJ, et al. Sacral chordomas: Impact of 

high-dose proton/photon-beam radiation therapy combined with or 

without surgery for primary versus recurrent tumor. Int J Radiat Oncol 

Biol Phys. Aug 1 2006;65(5):1514-1521.

Ineligible comparator: proton 

beam therapy plus photon 

therapy and surgery

Sethi RV, Shih HA, Yeap BY, et al. Second nonocular tumors among 

survivors of retinoblastoma treated with contemporary photon and 

proton radiotherapy. Cancer. Jan 1 2014;120(1):126-133.

Ineligible population: pediatric 

and retinoblastoma 

Tokuuye K, Akine Y, Kagei K, et al. Proton therapy for head and neck 

malignancies at Tsukuba. Strahlenther Onkol. Feb 2004;180(2):96-101. 

Ineligible comparator: proton 

beam therapy plus photon 

therapy

APPENDIX D. ICER COMPARATIVE STUDIES EXCLUDED AT 

FULL-TEXT LEVEL

80
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Arvold 

2009 

US

“The decision 

to treat 

patients with 

photons 

versus 

protons 

versus a 

combination 

of both was 

made by the 

treating 

radiation 

oncologist 

and was 

based 

primarily on 

treatment 

machine 

availability.”

No.

12% (3/25) 

loss to 

follow-up. 

Yes. Yes. Yes. No.

Differences in 

tumor size at 

baseline (2.25 

mL, 3.63 mL, 

and 4.15 mL). 

No.

42 months 

for photon 

therapy, 78 

months for 

combination 

therapy, and 

12.5 months 

for proton 

therapy. 

Poor.

APPENDIX E. QUALITY ASSESSMENT OF PRIMARY STUDIES 

Observational Studies

81
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Brown 

2013 

US

Unclear how 

patients were 

selected for 

each group. 

Unclear. Yes. Yes. Yes.

Unknown 

whether

outcome 

ascertainers 

were blinded, 

outcomes are 

not subjective.

Not adequate.

Only performed 

for hematologic 

toxicity 

endpoints. 

No.

26.3 months 

(PBT) vs 

57.1 months 

(photon) 

Poor.

Chakravarti 

1999 

US

Unclear how 

patients were 

selected for 

each group. 

N/A Yes. Yes. Yes. No.

Differences in 

age at baseline 

(23.5 yrs vs 

52.5 yrs for 

combined and 

photon only 

groups, 

respectively).

Yes. Poor.

82
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Chung 

2013 

US

Yes.

Patients 

compared 

with 

historical 

results, but 

matched.

Yes.

27% 

excluded 

from proton 

cohort 

because they 

were lost to 

follow-up 

and it is 

plausible 

that their 

loss is 

associated 

with second 

cancer 

events.

Yes. Yes. No.

Method of 

data collection 

differed 

between 

groups. PBT: 

“data 

abstracted 

from 

pathology 

reports, 

operative notes 

and clinic visit 

notes ... 

Patients were 

also contacted 

by mail and 

scripted 

telephone calls 

to obtain 

data.” Photon 

group: based 

on diagnosis 

codes from 

SEER 

database.

Age at 

treatment, sex, 

primary tumor 

site, and year of 

treatment.

No information 

on dose of RT 

or 

chemotherapy 

in either group. 

No. Poor.

83
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Coen 

2012 

US

Yes.

Patients 

compared 

with 

historical 

results, but 

matched.

N/A Yes. Yes. Unclear. No, but 

matched on T 

stage, Gleason 

score, PSA 

level, and age.

Yes. Fair.

Duttenhaver

1983 

US

Unclear how 

patients were 

selected for 

each group. 

No. Yes. Yes. Yes.

While 

outcome 

ascertainers 

were 

apparently not 

blinded, 

outcomes are 

not subjective.

No. Did not 

control for 

potentially 

confounding 

effects of 

differentiation 

which could 

lead to better 

survival, but 

had an 

unknown effect 

on local control.

XRT: 55% had 

poorly 

differentiated 

tumors 

XRT+PBT: 

35% had poorly 

differentiated 

tumors

Yes. Local 

control: 

Fair.

Other

outcomes: 

Poor. 

84
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Fang

2014 

US

Allocation to 

PBT or 

IMRT was 

based on 

suitability for 

PBT as 

determined 

by triage 

committee, 

patient desire, 

machine 

availability, 

and insurance 

coverage.

N/A Yes. Yes. Unknown 

whether 

outcome 

assessors were 

blinded 

Yes.

Matched on 

prostate cancer 

risk group, age 

at diagnosis, 

prior GI or GU 

disorders. 

Potential 

confounders 

included ADT, 

hypertension, 

hemorrhoids, 

diabetes, 

Eastern 

Cooperative 

Oncology 

Group 

performance 

status, IPSS 

score and 

Bowel 

Symptom 

Score.

No. 

Shorted F/U 

time for the 

PBT group 

compared 

with IMRT 

group (29 

months vs 47 

months) 

Fair.

85
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Galbraith 

2001 

US

Yes. No.

17% attrition 

at 18 

months. 

Yes. Yes. Yes. No.

Age and stage 

assessed as 

covariates, but 

were not 

significant.

Yes. Fair.

GallandGirodet 

2014 

US

Unclear how 

patients were 

selected for 

each group. 

No.

2% loss to 

follow-up 

Yes. Yes. Yes.

While 

outcome 

ascertainers 

were 

apparently not 

blinded, 

outcomes are 

not subjective.

No.

No significant 

differences in 

patient or tumor 

characteristics 

between the two 

groups. 

Yes. Fair.

Gomez 

2012 

US

No.

PBT patients

compared 

with 

historical 

results.

No. Yes. Yes. Yes. Yes.

Age, gender, 

smoking status, 

stage, and 

histology did 

not predict 

severe 

esophagitis.

Yes. Fair.

86
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Gray 

2013 

US

No.

PBT patients 

compared 

with 

historical 

results.

No. Yes. Yes. Unclear. No.

Did not control 

for time or sitespecific 

variables.

Yes. Poor.

Hocht 

2006 

Germany

Patients 

received 

protons once 

they were 

made 

available in 

Germany in 

1998.

N/A Yes. No. Unclear. Controlled for 

differential 

follow-up time, 

tumor 

characteristics 

differed at

baseline.

Yes. Fair.

Hoppe 

2014 

US

No.

PBT patients 

compared 

with 

historical 

results.

No. Yes. Yes. Unclear. Yes.

Controlled for 

age, prostate 

size, ADT use, 

baseline QOL, 

but not for time 

or site-specific 

variables.

Yes. Poor.

87
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Hug 

2000 

US

“Patients

were selected 

for protons or 

photons 

based on the 

specific dose 

conformity 

advantages of 

protons for 

certain 

anatomic 

locations and 

for high 

degrees of 

target 

irregularity.”

No. Yes. Yes. Yes. Unknown.

No baseline 

patient 

characteristics 

were given by 

treatment 

group. 

Yes. Poor.

Kahn 

2011 

US

Unclear how 

patients were 

selected for 

each group. 

No. Yes. Yes. Yes.

Unknown 

whether 

outcome 

ascertainers 

were blinded, 

outcomes are 

not subjective.

Performed for 

survival but not 

local control 

outcome. 

Yes. Poor.

88
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Kim 

2011 

US

Yes. N/A Yes. Yes. Yes. Yes.

Controlled for 

diagnosis year, 

comorbidity, 

age group, 

clinical stage, 

SEER region, 

race, marital 

status, poverty, 

cancer grade.

Yes. Fair.

McCurdy 

2013 

US

Unclear how 

patients were 

selected for 

each group.

No. Yes. Yes. Yes. No. Yes. Fair.

Mizumoto

2013 

Japan

Treatment 

modality was 

chosen based 

on the 

location, 

distribution, 

and size of 

recurrent 

tumor. 

No. Yes. Yes. Yes. No.

Initial radiation 

dose, age, 

tumor volume 

differed 

between the 

reirradiation 

treatment 

groups.

Yes. Poor.

89
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Otsuka 

2003 

Japan

Patients were 

selected for 

each group 

based on 

availability of 

PBT system.

No. No. No. Unclear. No.

X-ray group 

older (60 vs 56 

years) and had 

larger tumors 

(3.6 vs 2.7 cm) 

at baseline.

Yes. Poor.

Sejpal 

2011 

US

No.

PBT patients 

compared 

with 

historical 

results.

No. Yes. Yes. Yes. Performed 

stratification of 

harms, but not 

multivariate 

regression. 

Didn’t control 

for temporal 

trends or 

differences at 

baseline (age, 

race, disease 

stage, tumor 

volume, 

adjuvant 

chemotherapy, 

time since 

treatment)

Yes. Poor.

90
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Country

Non-biased 

selection?

High overall 

loss to 

follow-up or 

differential 

loss to

follow-up?

Outcomes 

prespecified 

and 

defined?

Ascertainment 

techniques 

adequately 

described?

Non-biased 

and adequate 

ascertainment 

methods?

Statistical 

analysis of 

potential 

confounders?

Adequate 

duration of 

follow-up?

Overall 

quality 

rating

Sheets 

2012 

US

Yes. N/A Yes. Yes. Yes. Yes.

Propensity 

score 

weighting.

Yes. Fair.

Solares 

2005 

US

Unclear how 

patients were 

selected for 

each group. 

No. Yes. No. Unclear. No.

No baseline 

demographic 

information 

given by patient 

group.

Yes. Poor.

Wang 

2013 

US

Unclear how 

patients were 

selected for 

each group.

No. Yes. Yes. Yes. Yes. Yes. Fair.

Yu 

2013 

US

Yes. N/A Yes. Yes. Yes. No, but 

matched on age, 

race, residence, 

comorbidity, 

receipt of ADT, 

prior influenza 

vaccination, or 

prior visit to a 

primary care 

physician.

Yes. Fair.

91
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Randomized Controlled Trial

Author

Year

Adequate 

sequence 

generation?

Adequate 

allocation 

concealment?

Blinding of 

participants, 

personnel and 

outcome 

assessors?

Incomplete 

outcome data 

adequately 

addressed?

Study reports 

free of 

suggestion of 

outcome 

reporting bias?

Study free of 

other sources 

of bias? Risk of bias?

Shipley

1995 

US

Unknown. Unknown. Unknown.

Outcomes are 

not subjective.

No.

6.4% (13/202) 

did not 

complete 

planned 

protocol, 

excluded from 

analysis.

Yes. Yes. Medium.

92
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Title

Trial Sponsor 

Location(s) 

Identifier

Design Treatment/ 

Comparators 

Patient Population Primary 

Outcomes

Estimated 

Completion 

Date

A Bayesian Randomized Trial of 

Image-Guided Adaptive Conformal 

Photon vs Proton Therapy, With 

Concurrent Chemotherapy, for Locally 

Advanced Non-Small Cell Lung 

Carcinoma: Treatment Related 

Pneumonitis and Locoregional 

Recurrence 

MD Anderson Cancer Center

MD Anderson Cancer Center, 

Massachusetts General Hospital

USA 

NCT00915005 

RCT PBT (74 Gy) 

PBT (66 Gy) 

Photon therapy 

N=250 

18-85 years 

Unresected, locoregionally 

advanced NSCLC (stage II-IIIb) 

w/out evidence of hematogenous 

metastases 

Suitable for concurrent 

chemoradiation therapy 

FEV1 ≥ 1 liter 

Time to treatment 

failure

June 2015 

APPENDIX F. ONGOING CLINICAL TRIALS

93
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Title

Trial Sponsor 

Location(s) 

Identifier

Design Treatment/ 

Comparators 

Patient Population Primary 

Outcomes

Estimated 

Completion 

Date

Prostate Advanced Radiation 

Technologies Investigating Quality of 

Life (PARTIQoL): A Phase III 

Randomized Clinical Trial of Proton 

Therapy vs IMRT for Low or 

Intermediate Risk Prostate Cancer

 

Massachusetts General Hospital

Massachusetts General Hospital, 

University of Pennsylvania

USA 

NCT01617161 

RCT PBT 

IMRT 

N=400 

≥18 years 

Histologically confirmed 

adenocarcinoma of the prostate 

Clinical stages T1c-T2b 

Efficacy of PBT 

vs IMRT

QOL

Long-term 

survival

January 2016 

Randomized Comparison of Proton and 

Carbon Ion Radiotherapy With 

Advanced Photon Radiotherapy in 

Skull Base Meningiomas: The 

PINOCCHIO Trial 

University Hospital Heidelberg

University Hospital of Heidelberg

Germany 

NCT01795300 

RCT PBT 

Carbon ion therapy 

Hypo- fractionated 

photon therapy 

Conventional 

photon therapy 

N=80 

≥18 years 

Histologically or imaging 

confirmed skull base meningioma 

Macroscopic tumor, Simpson 

grade 4 or 5 

Karnofsky score ≥60 

Toxicity (graded 

after one year)

Overall survival

Progression-free 

survival

QOL

February 

2016 

94
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Title

Trial Sponsor 

Location(s) 

Identifier

Design Treatment/ 

Comparators 

Patient Population Primary 

Outcomes

Estimated 

Completion 

Date

Randomized Phase II Study Comparing 

Stereotactic Body Radiotherapy 

(SBRT) With Stereotactic Body Proton 

Therapy (SBPT) for Centrally Located 

Stage I, Selected Stage II and Recurrent 

Non-Small Cell Lung Cancer 

MD Anderson Cancer Center

MD Anderson Cancer Center

USA 

NCT01511081 

RCT SBPT 

SBRT 

N=120 

≥18 years 

Histological confirmation or 

clinically diagnosed primary 

NSCLC 

Centrally located stage I or 

selective stage II primary tumors 

Isolated recurrent disease 

Zubrod status = 0-2 

SBRT and SBPT 

related toxicity 

Treatment 

response 

August 2016 

A Prospective Phase II Randomized 

Trial to Compare Intensity Modulated 

Proton Radiotherapy (IMPT) vs 

Intensity Modulated Radiotherapy 

(IMRT) for Newly Diagnosed 

Glioblastoma (WHO Grade IV)

MD Anderson Cancer Center

MD Anderson Cancer Center

USA 

NCT01854554 

RCT IMPT 

IMRT 

N=80 

≥18 years 

Histological diagnosis of 

glioblastoma or gliosarcoma 

(WHO grade IV) adapted RPA 

class III, IV or V 

Mini Mental Status Exam score 

≥21 

Karnofsky score ≥70 

Time to cognitive 

failure

May 2017 

95
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Title

Trial Sponsor 

Location(s) 

Identifier

Design Treatment/ 

Comparators 

Patient Population Primary 

Outcomes

Estimated 

Completion 

Date

Phase III Randomized Trial of Proton 

Beam Therapy Versus IntensityModulated Radiation Therapy for the 

Treatment of Esophageal Cancer 

MD Anderson Cancer Center

MD Anderson Cancer Center

USA

 

NCT01512589 

RCT PBT 

IMRT 

N=180 

≥18 years 

Histologically confirmed 

adenocarcinoma or squamous cell 

carcinoma of the cervical or 

thoracic esophagus or 

gastroesophageal junction or 

cardia of stomach 

Karnofsky score ≥60 

ECOG criteria = 0, 1, or 2 

Progression-free 

survival 

Total toxicity 

burden (TTB) 

April 2018 

Phase III Randomized Trial Comparing 

Overall Survival After Photon Versus 

Proton Chemoradiotherapy for 

Inoperable Stage II-IIIB NSCLC

Radiation Therapy Oncology Group 

University of Florida, MD Anderson 

Cancer Center

USA

NCT01993810 

RCT PBT + 

chemotherapy 

Photon therapy + 

chemotherapy 

N=560

≥18 years 

Histologically or cytologically 

proven NSCLC 

Patients w/non-operable disease 

or refuse surgery 

Clinical stage TII, TIIIA, TIIIB 

Zubrod status = 0-1 

FEV1 ≥ 1 liter 

Overall survival 

Progression-free 

survival

Adverse events

QOL

Changes in 

pulmonary 

function 

December 

2020 

96
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Title

Trial Sponsor 

Location(s) 

Identifier

Design Treatment/ 

Comparators 

Patient Population Primary 

Outcomes

Estimated 

Completion 

Date

Phase II/III Randomized Trial of 

Intensity-Modulated Proton Beam 

Therapy (IMPT) Versus IntensityModulated Photon Therapy (IMRT) for 

the Treatment of Oropharyngeal Cancer 

of the Head and Neck Cancer

MD Anderson Cancer Center

MD Anderson Cancer Center

USA 

NCT01893307 

RCT IMPT 

IMRT 

N=360 

≥18 years 

Histologically documented 

squamous cell carcinoma of the 

oropharynx 

ECOG criteria = 0, 1, or 2 

Rates and 

severity of late 

grade 3-5 toxicity 

between IMRT 

and IMPT

August 2023 

Randomized Phase II Trial of 

Hypofractionated Dose-Escalated 

Photon IMRT or Proton Beam Therapy 

Versus Conventional Photon Irradiation 

With Concomitant and Adjuvant 

Temozolomide in Patients With Newly 

Diagnosed Glioblastoma 

NRG Oncology

NRG Oncology, Cadence Cancer 

Center

USA

NCT02179086

RCT 3-D conformal 

radiation therapy 

IMRT

Photon beam 

radiation therapy

Proton beam 

radiation therapy

Temozolomide 

N=576 

≥18 years

MRI of the brain performed 

postoperatively within 72 hours 

of resection; enhancing tumor 

must have a maximal diameter of 

5 cm

GBM tumor located in the 

supratentorial compartment only

Histologically proven diagnosis 

of glioblastoma (WHO grade IV)

Karnofsky performance status 

≥70

Overall survival 

Progression-free 

survival

Incidence of 

treatment-related 

toxicity

Change in 

cognitive and 

neurocognitive 

function 

May 2019

97
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Was an 

‘a priori’ 

design 

provided?

Was there 

duplicate 

study 

selection and 

data 

extraction?

Was a 

comprehensive 

literature 

search 

performed?

Was the 

status of 

publication 

(ie grey 

literature) 

used as an 

inclusion 

criterion?

Was a list 

of studies 

(included 

and 

excluded) 

provided?

Were the 

characteristics 

of the included 

studies 

provided?

Was the 

scientific 

quality of the 

included 

studies 

assessed and 

documented?

Was the 

scientific 

quality of the 

included 

studies used 

appropriately 

in formulating 

conclusions?

Were the

methods used 

to combine 

the findings of 

studies 

appropriate?

Was the 

likelihood 

of 

publication 

bias 

assessed?

Was the 

conflict 

of 

interest 

stated?

Amichetti 

2010

Can’t 

answer. 

No 

protocol 

number 

noted.

Can’t answer.

Duplicate 

study 

selection, but 

unknown data 

abstraction.

No.

Only PubMed® 

was searched.

Yes.

Restricted to 

English 

language.

No. Yes. No. No. Not applicable.No. No.

De 

Ruysscher 

2012

Can’t 

answer.

Can’t answer. Yes. No. No. Yes. Assessed but 

not 

documented.

No. Not applicable.No. Yes.

Cianchetti 

2012

Can’t 

answer.

Can’t answer. Yes. Yes.

Restricted to 

English 

language.

No.

Did not 

provide a 

list of 

excluded 

studies.

Yes. No. No. Not applicable.No. No.

Grimm

2012

Can’t 

answer.

Can’t answer. Yes. Can’t 

answer.

A list of 

included,

but not 

excluded, 

studies was 

provided.

Yes. No. No. No. Did not 

control for 

confounding 

between noncomparative 

studies.

No. Yes.

APPENDIX G. QUALITY ASSESSMENT OF SYSTEMATIC REVIEWS 

98
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Was an 

‘a priori’ 

design 

provided?

Was there 

duplicate 

study 

selection and 

data 

extraction?

Was a 

comprehensive 

literature 

search 

performed?

Was the 

status of 

publication 

(ie grey 

literature) 

used as an 

inclusion 

criterion?

Was a list 

of studies 

(included 

and 

excluded) 

provided?

Were the 

characteristics 

of the included 

studies 

provided?

Was the 

scientific 

quality of the 

included 

studies 

assessed and 

documented?

Was the 

scientific 

quality of the 

included 

studies used 

appropriately 

in formulating 

conclusions?

Were the

methods used 

to combine 

the findings of 

studies 

appropriate?

Was the 

likelihood 

of 

publication 

bias 

assessed?

Was the 

conflict 

of 

interest 

stated?

Grutters 

2010

Can’t 

answer.

Can’t answer.

Duplicate 

data 

abstraction, 

but unknown 

study 

selection.

Yes. Yes.

Restricted to 

English 

language.

No.

Did not 

provide a 

list of 

excluded 

studies.

Yes. No. No. Yes. No. No.

Ollendorf

2014

(ICER)

Can’t 

answer.

No 

protocol 

number 

noted.

No. Yes.

“The electronic 

databases we 

searched... 

included 

MEDLINE, 

EMBASE, and 

The Cochrane 

Library... for 

health 

technology 

assessments, 

systematic 

reviews, and 

primary 

studies.”

Yes.

Restricted to 

English 

language, 

but didn’t 

tell us 

whether they 

included 

grey lit.

Yes.

Did not 

provide a 

list of 

excluded 

studies, 

gave 

PRISMA 

flow chart

Yes. Assessed but 

not 

documented.

Yes. Not applicable.Yes. Yes.

No 

statement 

on author 

conflict of 

interest.

99
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author

Year

Was an 

‘a priori’ 

design 

provided?

Was there 

duplicate 

study 

selection and 

data 

extraction?

Was a 

comprehensive 

literature 

search 

performed?

Was the 

status of 

publication 

(ie grey 

literature) 

used as an 

inclusion 

criterion?

Was a list 

of studies 

(included 

and 

excluded) 

provided?

Were the 

characteristics 

of the included 

studies 

provided?

Was the 

scientific 

quality of the 

included 

studies 

assessed and 

documented?

Was the 

scientific 

quality of the 

included 

studies used 

appropriately 

in formulating 

conclusions?

Were the

methods used 

to combine 

the findings of 

studies 

appropriate?

Was the 

likelihood 

of 

publication 

bias 

assessed?

Was the 

conflict 

of 

interest 

stated?

Patel

2014

Yes. Yes. Yes. Can’t 

answer.

Yes.

Did not 

provide a 

list of 

excluded 

studies, 

gave 

PRISMA 

flow chart

Yes. Yes. No. Yes. No. Yes.

Ramaekers 

2011 

Can’t 

answer.

Can’t answer.

Duplicate 

data 

abstraction, 

but unknown 

study 

selection.

No.

Only PubMed® 

was searched.

Yes.

Restricted to 

English 

language.

Yes. Yes. No. No. Yes. No. Yes.

Walsh 2012

(CTAF)

Can’t 

answer.

Can’t answer. Yes. Yes.

Restricted 

to English 

language, 

no grey 

literature 

search.

No.

Did not 

provide a 

list of 

excluded 

studies.

Yes. No. No. Not applicable.No. No.

100
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author Year

Study Design

Study Site

Intervention

Comparator 

Follow-up

Sample Size

Patient 

characteristics

Treatment 

protocol 

Outcomes assessed

Main findings

Harms Quality

Duttenhaver 1983

Prospective cohort 

Massachusetts 

General Hospital

XRT plus 

PBT

XRT only

5 years

XRT plus PBT

N=64 

Age: 67.7 

(mean)

Race: unknown

Clinical stage: 

T1: 3%, T2: 

25%, T3: 59%, 

T4: 13%

XRT only

N=116 

Age: 67.7 

(mean)

Race: unknown

Clinical stage: 

T1: 4%, T2: 

30%, T3: 56%, 

T4: 10%

XRT plus PBT

Initial 50 Gy 

given using 

standard 

external beam

Total dose: 70-

76.5 CGE 

(mean: 74 

CGE)

XRT only

Pelvis treated 

to 50 Gy in 5 

1⁄2 weeks, 5 

fractions per 

week, 1.8 Gy 

fractions, 

initial dose 

followed by a 

boost to 

prostatic tumor 

volume 

Total dose: 60-

68.4 Gy 

(mean: 67 Gy) 

Survival

Did not differ 

between XRT plus 

PBT and XRT 

groups 

Disease-free 

survival

Did not differ 

between XRT plus 

PBT and XRT 

groups 

Clinical local 

recurrence-free 

survival

“As yet there is no 

observed 

improvement in 

local control in the 

XRT plus PRT 

group despite an 8-

10% increase in 

radiation dose, 

when compared to 

the XRT group.”

Mild dysuria/increased frequency

XRT plus PBT: 11% 

XRT: 10%

Hematuria

XRT plus PBT: 8% 

XRT: 10%

Benign structure 

XRT plus PBT: 5% 

XRT: 4%

Mild proctitis

XRT plus PBT: 8% 

XRT: 11%

Moderate proctitis 

XRT plus PBT: 13% 

XRT: 5%

Severe proctitis

XRT plus PBT: 0% 

XRT: 1%

Poor

APPENDIX H. DATA ABSTRACTION OF COMPARATIVE STUDIES NOT INCLUDED IN 

THE ICER REVIEW

101
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author Year

Study Design

Study Site

Intervention

Comparator 

Follow-up

Sample Size

Patient 

characteristics

Treatment 

protocol 

Outcomes assessed

Main findings

Harms Quality

Fang

2014 

Retrospective 

cohort 

University of 

Pennsylvania 

(PBT: 01/10-12/12; 

IMRT: 07/09-

12/12)

PBT

IMRT

PBT median: 

47 months 

(5-65) 

IMRT 

median: 29 

months (5-

50)

PBT

N=94

IMRT

N=94

PBT and 

IMRT: 79.2 

Gy delivered 

in 44 fractions

Clinician-reported grade ≥ 2 GI 

toxicity: 4.3% (PBT) vs 13.8% 

(IMRT); OR (PBT vs IMRT): 0.27 

(95% CI, 0.06-1.24) 

Clinician-reported grade ≥ 2 GU 

toxicity:21.3% (PBT) vs 28.7% 

(IMRT); OR (PBT vs IMRT): 0.69 

(95% CI, 0.32-1.51) 

Fair

Galland-Grirodet 

2014 

Prospective phase 

1 nonrandomized 

trial

Massachusetts 

General Hospital

(10/03—04/06)

PBT

Photonbased 3DAPBI

Median: 82.5 

months 

Range: 1.6—

103.8 

months 

PBT

N=19

Age: 63 

(median) 

Race: unknown

Grade: 1: 53%, 

2: 32%, 3: 15% 

Photon-based 

3D-APBI

N=79

Age: 60 

(median) 

Race: unknown

Grade: 1: 46%, 

2: 46%, 3: 8% 

PBT

Dose: 32 Gy in 

8 fractions 

Photon-based 

3D-APBI

Dose: 32 Gy in 

8 fractions 

7-year cumulative 

incidence of local 

failure rate

PBT: 11%

Photon: 4% (P=.22) 

5-year skin color change

PBT: 44%

Photon: 2% (P≤0.0001) 

Patchy atrophy 

PBT: 50%

Photon: 5% (P≤0.0001) 

“At 7 years, physician assessments 

of skin color change (P=.02) and 

late skin toxicity (P=.029) were 

significantly worse in the PBT 

group. Telangiectasia >4 cm2 was 

observed for 38.5% of the PBT 

group as compared with 4% of the 

photon-based group (P=.0013).”

“There was no difference between 

the treatment groups in 

noncutaneous toxicities, including 

breast pain, breast edema, and rib 

tenderness, at either 5 or 7 years.”

Fair

102
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

 

Author Year

Study Design

Study Site

Intervention

Comparator 

Follow-up

Sample Size

Patient 

characteristics

Treatment 

protocol 

Outcomes assessed

Main findings

Harms Quality

Mizumoto

2013 

Retrospective 

cohort 

University of 

Tsukuba 

(01/05-09/10) 

PBT

Traditional 

radiotherapy 

and 

stereotactic 

radiotherapy 

Median 

follow-up 

for 

survivors: 

19.4 months 

PBT

N=8

Age: 51 

(median) 

Initial dose: 64.1 

Gy

Tumor volume: 

70.4 cc 

RT

N=8

Age: 41 

(median) 

Initial dose: 57.1 

Gy

Tumor volume: 

70.9 cc 

SRT

N=10

Age: 64.5 

(median) 

Initial dose: 58.9 

Gy

Tumor volume: 

11.5 cc

PBT

Mean 

reirradiation 

dose: 39.5 Gy 

RT

Mean 

reirradiation 

dose: 43.7 Gy 

SRT

Mean 

reirradaition 

dose: 41.7 Gy 

Overall survival

PBT: 19.4 months 

RT: 5.15 months

SRT:11.6 months 

Mortality after a 

median of 11.6 

months:62.5% 

(conventional 

photon), 50% 

(SRT), 62.5% 

(PBT); P>.99

Local 

recurrence:25% 

(conventional 

photon), 30% 

(SRT), 25% (PBT); 

P>.99 

Poor

103
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

104  

APPENDIX I. PEER REVIEW COMMENT DISPOSITION TABLE 

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

1. Are the objectives, scope, and methods for this review clearly described?

1 1 Yes (no comments)

2 2 Yes (no comments)

3 3 Yes (no comments)

4 4 Yes (no comments)

5 5 Yes (no comments)

6 6 Yes (no comments)

7 7 Yes (no comments)

8 8 Yes (no comments)

2. Is there any indication of bias in our synthesis of the evidence?

1 1 No (no comments)

2 2 No (no comments)

3 3 No (no comments)

4 4 No (no comments)

5 5 No (no comments)

6 6 No (no comments)

7 7 No (no comments)

8 8 Yes: The actual title, "Effectiveness and Harms of proton 

irradiation treatment" seems to imply there is something bad 

with proton beam RT vs other forms of radiation therapy. While 

there have been studies reported with possibly worse than 

expected toxicities, these are entirely related to technique vs

proton beam itself. For example, the partial breast proton beam 

series toxicity was related to the use of a single-beam, not 

protons itself. The more contemporary series from Loma Linda 

We changed the title of our review to “Comparative 

Effectiveness of Proton Irradiation Treatment”
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

105  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

published earlier this year (D. Bush, IJROBP 2014) showed this 

quite nicely and was supported by the accompanying editorial 

from E. Strom. This theme continues with prostate, where the 

series from Shipley, published in the 1990's, used single 

perineal fields, an outdated technique.

I think re-titling this, "Proton Beam Therapy: An Updated 

Review" would be more appropriate and less inflammatory.

3. Are there any published or unpublished studies that we may have overlooked?

1 1 No (no comments)

2 2 No (no comments)

3 3 Yes: Systematic reviews of the past 5 years were included. 

Although it will not add value to the data and will not influence 

the overall conclusion, below three more reviews to add: 

• van de Water TA, Bijl HP, Schilstra C, Pijls-Johannesma M, 

Langendijk JA. The potential benefit of radiotherapy with 

protons in head and neck cancer with respect to normal tissue 

sparing: a systematic review of literature. Oncologist. 

2011;16(3):366-77.Review. 

• Pijls-Johannesma M, Grutters JP, Verhaegen F, Lambin P, De 

Ruysscher D. Do we have enough evidence to implement 

particle therapy as standard treatment in lung cancer? A 

systematic literature review. Oncologist. 2010;15(1). Review. 

• Combs SE, Laperriere N, Brada M. Clinical controversies: 

proton radiation therapy for brain and skull base tumors. Semin 

Radiat Oncol 2013r;23(2):120-6. Review.

Thank you for suggesting these additional systematic 

reviews. We excluded the 2011 van de Water et al. 

publication because, as a review of in silico planning 

studies, it did not include any outcomes of interest for 

our review.

We excluded the 2010 Pijls-Johannesma et al. 

publication because none of the included PBT studies 

included comparison groups.

We excluded the 2013 Combs et al. publication 

because this systematic review included noncomparative studies and did not perform any pooling of 

studies or meta-analysis.

4 4 Yes - Grimm et al., 2012, mentioned in my review Thank you for bringing this study to our attention. We 

added a discussion of Grimm et al. 2012 in the Prostate 

Cancer section.

5 5 Yes: Talcott JA et al, JAMA 303(11):1046-53, 2010.

Fang P, et al. Cancer 2014 Epub, PMID: 25423899

Thank you for these suggestions. We excluded the 

2010 Talcott et al publication because we could not 

isolate the effect of PBT since it compared two 

different doses of combined proton beam and photon 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

106  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

therapies.

Fang: Added

6 6 No (no comments) N/A

7 7 Yes: Please see my written report N/A

8 8 Yes: Receny partial breast RT series (Bush et al, PMID: 

25084608)(Strom et al, PMID: 25304946)

Bush 2014 is already included. Strom 2014 is an 

editorial that provides interpretation of the differences 

between Galland-Girodet 2014 and Bush 2014; which 

are consistent with ours.

4. Additional suggestions or comments can be provided below. If applicable, please indicate the page and line numbers from the draft 

report.

1 4 Pg. 1 line 10 “Dosimetric advantages are not theoretical. I 

would rephrase: "because of the physical properties of the 

proton beam, which can translate into dosimetric advantages for 

certain clinical situation..." 

Changed to “appealing dosimetric characteristics”

2 4 Pg. 2 line 14 “It is critical to note that this trial was only for the 

partial-breast irradiation, which means only patients with Stage 

I cancer were eligible, and the partial breast concept is to 

eliminate the dose to lungs and heart. Therefore there is not 

much advantage for protons in the partial breast RT. 

HOWEVER, proton beam is a very interesting concept for 

patients with LEFT breast locally advanced cancer (Stages II 

and III), where the entire left chest wall needs to be irradiates.

A regular 3D photon treatment exposes too much heart, with a

7.5% risk of coronary events for each Gray of RT to the heart. 

Protons can significantly spare the dose to the heart in these 

clinical scenarios. For patients with Left locally advanced breast 

cancer, proton beam therapy should be considered, if a regular

3D plan cannot achieve a heart mean dose of less than 5 Gy.”

We clarified that the patients in this trial were being 

treated for stage I breast cancer.

3 4 Pg. 2 line 38 “This dose (54.6 vs 52.9) is to the primary disease 

in the brain. Yes, protons allow for a very modest dose 

escalation to the primary disease, and it is not yet clear whether 

1.7 Gy of difference leads to improved survival. However, 

We thank the reviewer for these insights. No change 

needed. 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

107  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

protons allow for much less dose to the healthy brain 

parenchyma, and to Right and Left cochleas. This could lead to 

improved hearing and cognitive function with a long follow-up. 

In terms of esophagitis and nausea/vomiting - this comes from 

the toxicity of the spinal cord irradiation for medulloblastoma. 

The dose is the same with photons and with protons, but there is 

much less exit dose into the body cavity with all internal 

organs, which leads to better acute tolerance of treatment, and 

theoretically decreased risk of secondary malignancies in these 

organs, when patients are followed for longer than 15 years.”

4 4 Pg. 3 line 4 “Rephrase this: There is also insufficient evidence 

to draw conclusions of proton beam -based stereotactic ablative 

therapy for early stage lung cancer, in comparison to photonbased stereotactic ablative therapy.”

We rephrased this sentence: “There is also insufficient 

evidence to draw conclusions about proton-based 

stereotactic ablative therapy for early-stage lung cancer 

compared with photon-based stereotactic ablative 

therapy.”

5 4 Pg. 3 line 23 “Survival difference is known - there is NO 

survival difference. In prostate cancer, one cannot find a 

difference in survival between two treatment modalities - it 

takes over decades for prostate cancer failure to translate into 

survival difference. All studies of IMRT vs protons have used 

the same radiation dose, therefore the only question that was 

ever asked in these studies - is the quality of life and toxicity.”

Deleted “but unknown survival” for IMRT and 3DCRT comparisons. We take your point that calling out 

the lack of evidence on survival may lack relevance 

here given that finding a difference in survival at 5 

years may not be plausible given the natural history of 

the disease.

6 4 Pg. 3 line 58 “It is critical to mention that the main 

disadvantage of this study was a very low number of patients 

analyzed, therefore any outcome is likely to be influenced by 

chance alone. Moreover, proton beam was not associated on a 

univariate analysis (table 4), therefore it is more likely that 

selection of patients for proton beam was influenced more by 

worse pathology and size of the tumor, inability to resect the 

tumor, etc, rather than the choice of therapy itself.”

We agree with the reviewer that the strength of the 

evidence is low due to the mentioned methodological 

limitations and have described these in the full results 

section. No change needed to the Executive Summary. 

7 4 Pg. 4 line 7 “Why you have not included a special section on 

chordoma and chondrosarcoma of the spine and base of skull? 

These are pathologies that SHOULD only be treated with 

We did not identify any comparative studies in 

chordoma or chondrosarcoma patients so did not 

include a separate section for these conditions. In our 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

108  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

protons at this moment, as there is no data that photon-based 

therapy is equivalent to results achieved with protons.”

introduction we cite a VA Memorandum that 

acknowledges the improved safety of radiation delivery 

to sacral and base of skull chordomas. 

8 4 Pg. 4 line 13 “Evaluation of recurrent tumors is very patientspecific. If patient has a locally recurrent tumor with no 

evidence of metastatic disease, and re-irradiation offers a 

second chance of cure, then attention should be drawn to the 

cumulative dose to the organs, when previous and present

treatment plans are combined. If dosimetrically, the physician 

cannot achieve a dose high enough for tumor control, without 

damaging normal structures, based on their radiation tolerance, 

and a physician can show that a proton beam therapy can 

improve this, then there is an indication for proton beam 

therapy.”

We appreciate the reviewer’s rationale for the role of 

proton beam therapy in recurrent tumors. We do not 

think that Mizumoto 2013 and Otsuka 2003 are the last 

word on proton beam for recurrent tumors and would 

strongly encourage this as an area for future research 

efforts. 

9 4 Pg. 4 line 39 “This is not true. A SEER-based publication 

showed fewer secondary malignancies for kids treated with 

proton vs photons (Chung et al., 2013). Therefore when one 

applies the same criterias for young adults (young men with 

seminoma, young women with breast cancer or lymphoma), the 

same principle applies to this population - less normal tissue 

exposed to overall radiation translates into fewer secondary 

malignancies. This cannot be applied to elderly population, as it 

takes at least 15 to 20 years to develop secondary malignancy 

after RT, but for young adults this is very applicable.”

Yes, Chung 2013 is included in our review. But it is 

retrospective, not prospective, so this statement is 

correct and relevant. But, we have added a section 

about its findings to the Executive Summary. 

10 4 Pg. 9 line 32 “you mean genetic?” We rephrased this sentence to more clearly reflect 

ASTRO’s model policy: “The American Society for 

Radiation Oncology supports the use of PBT for ocular 

tumors, tumors that approach or are located at the base 

of the skull, certain tumors of the spine, primary 

hepatocellular cancer treated in a hypofractionated 

regimen, solid tumors in children treated with curative 

intent, and in patients with genetic syndromes for 

whom minimizing the total volume of radiation is 

11 4 Pg. 9 line 33 “Hepatocellular carcinoma has nothing to do with 

the genetic syndromes. It is all about how much of normal liver 

receives radiation therapy. At least 700 cc of normal liver 

should be completely spared of any radiation dose in order for 

treatment to be safe and for patients not to develop radiationinduced liver disease (RILD). This means that with proton 

beams, larger tumors can be safely irradiated, in comparison to 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

109  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

photon-based treatment.” crucial.”

12 4 Pg. 10 line 19 “An additional expense that comes with proton 

beam therapy is the need to create brass apertures for each beam 

used on an individual patient. This requires special facilities, 

specially trained personnel. The cost of brass material is very 

high.”

Thank you for this information. We added the 

following sentence: “Additional expenses associated 

with PBT is the creation of individual brass apertures 

for each beam, requiring special facilities and 

personnel.”

13 4 Pg. 15 line 53 “Massachusetts General Hospital and MD 

Anderson. The prostate randomized trial was started at Mass 

General Hospital, the second institution that joined the trial is 

UPenn. Lung Cancer was a collaborative work between MGH 

and MDACC.”

We altered Appendix G and updated this sentence: 

“These ongoing studies are being conducted at the 

University of Texas MD Anderson Cancer Center, 

Massachusetts General Hospital and the University of 

Pennsylvania as well as other centers in the US and 

abroad.”

14 4 Pg. 26 line 4 “partial breast treatment for early stage breast 

cancer. It is very crucial to distinguish whole breast treatment or 

chest wall treatment with radiation therapy for advanced breast 

cancer vs partial breast treatment for patients with Stage I breast 

cancer.”

We altered the title of table 4: “Strength of evidence 

(SOE) for comparative benefits and harms of PBT or 

photons or mixed photons/electrons in partial breast 

treatment for early-stage breast cancer”

15 4 Pg. 27 line 16 “Exactly! The effectiveness of both treatments

was the same. When patients die from the spinal cord glioma, 

they die from the local disease progression. The fact that the 

local recurrence rates were the same puts the multi-variable 

analysis under suspicion, that it inappropriately adjusted for 

large difference in age and made proton beam therapy look 

horrible, whereas on univariate analysis there was no 

difference. I am worried that this inappropriate analysis will 

make proton beam look so much worse for the spinal cord 

gliomas, where in reality the effectiveness dose per dose is the 

same.”

Yes, we also have low confidence in the stability of this 

estimate for the reasons you’ve mentioned, including 

the insufficient sample size and the inconsistency 

between the univariate and multivariate analyses. For 

these reasons, we have rated the strength of this 

evidence as low. 

16 4 Pg. 28 line 48 “Poorly worded. Effectiveness of combination of 

protons with photons vs photons alone?”

We rephrased this sentence: “We identified one poorquality retrospective comparative cohort that evaluated 

the clinical effectiveness of combination proton and 

photon treatment versus photon-only treatment in giant 

cell tumor of bone.”
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

110  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

17 4 Pg. 31 line 41 “Proton-based Ablative RT vs photon-based 

ablative RT for early stage NSCLC.”

We rephrased this heading: “Proton-based Ablative RT 

versus photon-based ablative RT for early stage 

NSCLC”

18 4 Pg. 32 line 30 “for locally advanced NSCLC” Added to this heading: “PBT versus IMRT or 3D-CRT 

for locally advanced NSCLC”

19 4 Pg. 39 line 38 “I am not sure this is relevant now. Almost 

everyone agrees that IMRT is superior to 3D-CRT for highdose RT to prostate (over 70 Gy), therefore no one is treating 

patients with localized disease without IMRT.”

Thank you for this comment. As the scope of our 

review includes both conventional and state-of-the art 

therapies, we included this evidence for the sake of 

completeness

20 4 Pg. 41 line 18 “There is a meta-analysis of various treatment 

modalities for prostate cancer which indicates that 

brachytherapy is superior to all forms of external beam RT, 

including proton beam therapy. Grimm et al., BJUI 2012. If you 

look at the names of co-authors, you will see that these are all 

leaders of the prostate cancer treatment in radiation oncology. I 

think it is important to incorporate this study.”

Thank you for bringing this study to our attention. We 

added a discussion of Grimm et al. 2012 in the Prostate 

Cancer section.

21 4 Pg. 41 line 38 “The "boost" was only a few fractions at the very 

end of the treatment. The main question of this randomized trial 

at MGH was to determine whether increasing the dose beyond 

70 Gy, what was considered standard back then with 3D-CRT 

would improve the outcomes, not the protons. But since the trail 

was done at MGH where protons are available, the study used 

protons for the last five fractions. So this study is really about 

79.2 Gy vs 70 Gy of radiation therapy, not about photons alone 

vs photons + protons.”

We altered our discussion of both Shipley et al. 1995 

and Duttenhaver et al 1983 to emphasize that these 

studies used a “boost” of protons in addition to photon 

therapy and that this combined group received a larger 

overall dose compared with the group that received 

photon treatment only.

22 4 Pg. 44 line 32 “p<0.01” Added.

23 4 Pg. 44 line 41 “These patients were treated for non-metastatic 

disease, using standard treatments doses, therefore we can be 

sure that the doses were comparable, if not exactly the same, 

diagnosis for diagnosis.”

We disagree. Although patients were matched by 

treatment site and histology in an attempt to control for 

irradiation volume, we still cannot be completely sure 

radiation dose and field size were comparable. 

24 4 Pg. 44 line 60 “Yes!!! This is the MAIN criticism and argument 

against this article. We do not anticipate to see 2nd malignancy 

Agreed. 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

111  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

during the RT within the first 5 years, more likely to develop 

these malignancies in 15 years or longer.”

25 4 Pg. 45 line 37 “This is a situation where there will be no data. It 

is similar to taking very difficult surgical cases and trying to 

determine which technique led to improved outcomes. Based on 

the amount of disease, prior radiation therapy plan, patient's 

anatomy, physician should be able to decide how best to 

approach the treatment - and if proton beam allows for less dose 

to previously irradiated organs, then protons will be superior. 

Similar to open vs laparoscopic surgery - surgeon decides at the 

time of surgery which technique is necessary for best 

visualization/access, etc, and that in theory should be left to the 

physician to decide and will never be analyzed in literature.”

It is precisely because this is a particularly difficult to 

treat population that this is a worthwhile question to 

ask. Knowing whether patients with recurrences are 

more or less likely to benefit from proton therapy 

compared with other alternatives could be particularly 

useful when, in general, recurrent tumors may be less 

sensitive to all treatments. And, in fact, there are 

already studies that have evaluated the comparative 

effectiveness of proton therapy in recurrent tumors. 

26 4 Pg. 46 line 53 “It is incorrect to say that proton therapy is less 

beneficial in patients with recurrent tumors. The correct 

statement is: ‘Any treatment form is less effective in the setting 

of disease recurrence.’ This is primarily driven by two factors: 

often patients have metastatic disease at the time of recurrence, 

and if the tumor comes back, it is more likely to be less 

sensitive to treatment effect. Therefore physicians always tell 

patients that the effectiveness of treatment is lower in the 

setting of disease recurrence. Don't state that this is specific to 

proton beam therapy.”

Thank you for this information. We replaced this 

sentence with: “Any treatment form is less effective in 

the setting of disease recurrence, including proton 

therapy.”

27 4 Pg. 47 line 7 “Have all the confidence in the world with this 

statement! It has absolutely nothing to do with protons! The 

same can be shown for brachytherapy, surgery, external beam 

RT, chemotherapy, you name it! Cancer at the time of 

recurrence is always worse than at the time of the new 

diagnosis.”

Agreed, our point is not about protons. It is that the

reduced benefit of protons for recurrent vs primary 

tumors seen in these studies is likely not entirely due to 

the inherently worse prognosis for recurrences, given 

the potential for confounding due to differences in 

radiation dosage, years of treatment and patient age. 

28 4 Pg. 47 line 20 “What this means is that experience with proton 

beam is more important than the experience with photon 

therapy. A fresh proton beam facility with minimal experience 

We thank the reviewer for this insight. No change 

needed. 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

112  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

is likely to have initially worse outcomes than a proton beam 

facility with 20 years of experience. It is similar to 

neurosurgeons who are fresh out of residency programs, vs 

experienced surgeons.”

29 4 Pg. 51 line 13 “again, chordoma and chondrosarcoma have only 

successfully been treated with protons, so it is up to IMRTbased treatment studies to prove IMRT is just as good. Uveal 

melanoma is also can be treated with brachytherapy or proton 

beam therapy, IMRT-based therapy is not used for uveal 

melanoma.”

We did not identify any comparative studies in 

chordoma or chondrosarcoma patients so did not 

include a separate section for these conditions. In our 

introduction we cite a VA Memorandum that 

acknowledges the improved safety of radiation delivery 

to sacral and base of skull chordomas.

We altered this sentence: “For all cancer sites and 

types, except for ocular and pediatric cancers which 

were not reviewed here...”

30 4 Pg. 51 line 40 “again, only in the setting of PARTIAL BREAST 

IRRADIATION. This should be stated very clearly, that this 

does not apply to whole breast RT situations.”

We rephrased the beginning of this sentence: 

“Increased harms include that (1) for partial breast 

irradiation, various skin toxicities are more common 

with PBT than with 3D-CRT when PBT is delivered in 

single-fields...”

31 4 Pg. 51 line 46 “again, would not use this study to claim that 

proton beam causes more death.”

Agree that Kahn 2011 has major deficiencies and 

provides only low-strength evidence. We added the 

low-strength evidence qualifier to our statement about 

Kahn’s 2011 findings. 

32 4 Pg. 53 line 53 “Not true.” We altered Appendix G and updated this sentence: 

“Many of these ongoing studies are multi-site and are 

being conducted at MD Anderson Cancer Center, 

Massachusetts General Hospital, and the University of 

Pennsylvania as well as other centers in the US and 

abroad.”

33 4 Pg. 54 line 19 “I think the conclusion is pretty nihilistic. A 

current randomized study of protons vs IMRT for prostate 

cancer as done at MGH and UPenn is actually a very good 

study, well designed, with all biases controlled, that will show 

Changed to: “Although numerous randomized 

controlled trials are underway that carry the promise of 

improved toxicity measurement, it is unclear whether 

they will fully address gaps in evidence on other 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

113  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

whether there is any difference in rectal toxicity.” important outcomes including recurrence, ability to 

deliver planned chemotherapy and radiation regimens, 

functional capacity, overall severe late toxicity, and 

secondary malignancies” 

34 1 Table 1, page 3, Harms column. For each entry, make explicit 

which is the reference arm. For IMRT on lines 23-27, it appears 

proton beam is compared to IMRT (in that order, so PB is the 

reference) and proton beam has higher GI toxicity with HR = 

3.32. However, in lines 29-35, the order appears to be reversed 

(IMRT vs PB) with IMRT now the reference and with RR of 

less than 1, so proton beam is again more toxicity. This change 

in reference is prone to confusion. This is repeated in Table 2, 

page 5.

Thank you for this suggestion. We clarified the 

reference group for every measure of association listed 

in a table.

35 2 Conclusions stated within the tables, such as Table 15, are 

occasionally unclear. Though the accompanying text clearly 

attributes relative benefits or harms to the appropriate radiation 

modality, table entries are less clear.

We labeled all outcomes by treatment group in tables 

throughout the report.

36 3 This report represents a review on the clinical evidence of 

proton therapy as compared to conventional/state of the art 

photon therapy. This systematic review was very well 

performed, including a risk of bias analysis. Methodological I 

have nothing to add.

Systematic reviews of the past 5 years were included. Although 

it will not add value to the data and will not influence the 

overall conclusion, below three more reviews to add: 

• van de Water TA, Bijl HP, Schilstra C, Pijls-Johannesma M, 

Langendijk JA. The potential benefit of radiotherapy with

protons in head and neck cancer with respect to normal tissue 

sparing: a systematic review of literature. Oncologist. 

2011;16(3):366-77.Review. 

• Pijls-Johannesma M, Grutters JP, Verhaegen F, Lambin P, De 

Ruysscher D. Do we have enough evidence to implement 

particle therapy as standard treatment in lung cancer? A 

Thank you for suggesting these additional systematic 

reviews. 

We excluded the 2011 van de Water et al. publication 

because, as a review of in silico planning studies, it did 

not include any outcomes of interest for our review.

We excluded the 2010 Pijls-Johannesma et al. 

publication because none of the included PBT studies 

included comparison groups.

We excluded the 2013 Combs et al. publication 

because this systematic review included noncomparative studies and did not perform any pooling of 

studies or meta-analyses.
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

114  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

systematic literature review. Oncologist. 2010;15(1). Review. 

• Combs SE, Laperriere N, Brada M. Clinical controversies: 

proton radiation therapy for brain and skull base tumors. Semin 

Radiat Oncol 2013r;23(2):120-6. Review.

Although disappointedly, mainly due to lack of evidence, I fully 

agree with the overall conclusions that was drawn by the 

authors. I have no additional comments.

37 5 Overall, this is an outstanding article and demonstrates that the 

authors have an impressive insight into radiation treatment and 

outcomes. 

Thank you. 

38 5 Please see two suggested articles (Talcott, Fang), which add to 

the literature (and perhaps level of overall evidence) on 

comparative patient-reported outcomes of proton vs photon for 

prostate cancer.

We excluded the 2010 Talcott et al publication because 

we could not isolate the effect of PBT since it 

compared two different doses of combined proton 

beam and photon therapies.

Added Fang 2014

39 5 In the lung sections, it may be worthwhile for the authors to 

examine whether IMRT is deemed “standard of care” for this 

disease. I think many insurers do not cover IMRT routinely for 

lung cancer radiation treatment. This consideration provides 

important practical insight to the reader of this review, in terms 

of the proton vs 3DCRT (standard) and proton vs IMRT (not 

standard) comparisons.

Defining the standard of care is outside of the scope of 

this review as it may vary across health systems and 

over time and may incorporate other information (eg, 

clinical expert input, cost, adherence) 

40 5 For prostate cancer, it is important to note that the randomized 

trial of PBT+photon vs photon alone used different doses in the 

2 arms. This is an important detail that was left out of the 

review. Some would interpret the data from this trial to suggest 

that proton can be used to safely increase dose of prostate 

cancer radiation without increasing toxicity. It is also important 

to note that the photon used in this trial was not IMRT. So this 

trial does not directly provide evidence on the toxicity outcomes 

of PBT vs IMRT for prostate cancer.

We amended the discussion of PBT+photon versus 

photon alone in the prostate cancer section to include 

information on the delivered dose in each treatment 

group of the three included studies.

41 5 Table 4 and other tables: it would be helpful to label the 

outcomes. For example “7-year cumulative local recurrence: 

Thank you for the suggestion. We labeled all outcomes 

by treatment group in tables throughout the report.
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

115  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

11% vs 4%” – which is PBT and which is photon? I can give a 

good guess based on the data, but labeling this in the Tables 

would be even better.

42 5 Page 38, the significant space used for criticisms of the Sheets 

et al study seems way out of proportion with the rest of the 

document, and suggests bias by the authors. As the authors 

note, all of the literature comparing PBT to photons have 

significant flaws – but the criticisms of this one study took up 

almost an entire page of space and seems unusual. The language 

of “widely criticized” to describe this study seems especially 

harsh, especially given the context that the 3 cited letters-to-the 

editor all came from physicians from proton centers, who may 

have inherent bias and financial conflicts of interest.

To provide full disclosure, I am one of the authors of that study 

and am able to provide further insight on the limitations of the 

study. The reason that critiques were not all fully answered was 

because of the significant JAMA word count limitations for 

author responses.

The criticism of using surrogate measures and data from few 

proton institutions – apply to all SEER-Medicare studies 

(include Kim et al, and Yu et al), not just the Sheets study. 

Frankly, until 2006, there were only 2-3 proton centers treating 

prostate cancer in total.

I believe it is inappropriate to directly translate results from the 

Sheets study to institutional series. The institutional series 

reported % of patients with toxicity attributed to radiation 

treatment. The Sheets (and Yu and Kim) studies examined % of 

patients who had a diagnosed GI or GU morbidity, or had GI or 

GU procedures after radiation treatment. These are very 

different outcomes, and one cannot be calculated to compare to 

the other. Patients can have GI or GU diagnosed morbidity or 

procedures not related to radiation treatment, which are counted 

We streamlined our discussion of the Kim 2011 and 

Sheets 2012 weaknesses to focus on the similar and 

well-accepted problems in both with exposure and 

outcome assessment methods: 

“GI toxicity at 4-5 years. Two fair-quality, populationbased retrospective cohort studies of Medicare claims 

data linked to the Surveillance Epidemiology and End 

Results (SEER) database provide low-strength 

evidence of an increased risk of late GI toxicity at 4-5 

years (Table 16).9,10 The first of the SEER-database 

studies, by Kim et al, included patients diagnosed with 

early-stage localized prostate cancer between 1992 and 

2005 and used ICD-9 or CPT procedure codes to assess 

grade 3 to 4 bleeding, ulceration, fistula, stricture, and 

colostomy that developed at least 6 months after 

diagnosis and required intervention.9 The second of the 

SEER-database studies, by Sheets et al, assessed risk of 

unspecified GI morbidity-related procedures (including 

colonoscopy) and diagnoses that occurred at least 12 

months after diagnosis in patients diagnosed with earlystage prostate cancer between 2000 and 2007.10 The 

main limitations of both studies include their high 

potential for exposure and outcome misclassification 

biases.67-69 Regarding exposure ascertainment, risk of 

bias was high because dose and field size specifics 

were unknown; therefore, the increased risk of late GI 

toxicity with PBT may have been entirely due to higher 

doses. Regarding outcome ascertainment, risk of bias 

was high because of the questionable reliability of 

using surrogate procedure (including colonoscopy) and 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

116  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

as events in the SEER/Medicare studies – with unclear 

attribution to radiation treatment, but with the assumption that 

underlying events unrelated to treatment should be similar 

between PBT and IMRT patients. There is an underlying 

“baseline” event rate in these elderly patients (many elderly 

patients have GI or GU problems anyway). But differences 

found in event rates between PBT and IMRT patients are 

possibly due to treatment outcome differences. If there is bias, 

the bias should favor PBT patients who are usually younger, 

healthier, and can travel longer distances to receive treatment –

and therefore may have lower baseline event rates than photon 

patients. 

The criticism about photon with lower dose than PBT: this 

applies to all 3 SEER/Medicare studies. Further, the Sheets 

study directly addresses the common claim by proton center 

physicians that proton treatment can deliver a higher dose to the 

prostate safely, therefore increase cure rate compared to IMRT. 

In fact, if indeed PBT patients received a modestly higher dose 

than IMRT in the Sheets study, the results do not support these 

claims – there was no higher cure rate, and morbidity was 

higher.

diagnosis code-based measures to detect the actual 

clinical events of interest. Also, both studies may suffer 

from potential confounding by study site since likely a 

high majority of proton patients were treated at the

single Loma Linda study site, whereas IMRT patients 

were likely treated at a variety of sites.”

43 5 Applicability section, page 52: “more or less experience.” It is 

unlikely that newer proton facilities would have “more” 

experience compared to the facilities that have been treating 

patients for many years/decades.

We changed this sentence to read: “First, the majority 

of the proton beam treatment groups came from one of 

3 proton facilities that are among the oldest in 

operation (Loma Linda, MD Anderson, or 

Massachusetts General). It is unclear whether the 

patient outcomes of these centers would generalize to 

other facilities with less experience treating patients 

and that may have difference standards of care.”

44 5 Page 53: It is worth noting that some dosimetric comparison 

studies show that PBT can have worse dose distribution 

compared to IMRT. For example, in prostate cancer, PBT vs

IMRT delivers higher doses to the femoral heads, and more 

Changed to: “Other clinicians and experts in radiation 

oncology question whether PBT’s dosimetric 

characteristics translate into measurable clinical 

benefits or increased survival for patients.”
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

117  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

high dose to the bladder. The statement of “obvious superior 

dose distribution” is not always true.

45 6 This was a well written review of the evidence for the treatment 

of adult cancers with proton radiotherapy. The authors have 

done an outstanding job collecting all the comparative studies.

Thank you. 

46 6 Page 3, Table 1. “IMRT” Comparator: “Harms” column: The 

Yu et al. (ref 11) did not find increased late GI toxicity. Also, in 

the “Benefits” column for the “IMRT” Comparator: the Yu et 

al. (ref 11) paper found transiently lower GU toxicity for 0-6 

months after treatment.

Moved Yu 2013 citation to statement about similar 

acute toxicity at beginning of sentence; added evidence 

on transient GU advantage at 6 months.

47 6 Page 4: Rather than simply saying “hemangiomas”, which can 

occur anywhere, I would specify “uveal hemangiomas”, which 

given their location in the eye there is much more impetus to 

use proton beam radiotherapy. I would replace “hemangiomas” 

throughout the whole review with “uveal hemangiomas.”

We made this change throughout the report.

48 6 Page 5: Table 2: Prostate row “-vs IMRT”: the Yu et al. study 

found a transiently lower GU toxicity for 0-6 months after 

treatment but no long term differences. This is a potential 

advantage. 

For the disadvantage row “-PBT + photon vs photon alone:” the 

comparison is a little difficult to make as the “PBT+photon” 

was a higher dose than the photon alone. The study cited 

(Shipley et al) was testing PBT as a boost therapy. This would 

be clearer if the row reads “-PBT Boost + Photon vs photon 

alone”. Furthermore, the Shipley et al study used much older 

radiotherapy techniques, and so should not be indicative of 

current outcomes and is of limited relevance. The same can be 

said for Duttenhaver et al.

Added evidence on transient GU advantage at 6 months

We altered our discussion of both Shipley et al. 1995 

and Duttenhaver et al 1983 to emphasize that these 

studies used a “boost” of protons in addition to photon 

therapy and that this combined group received a larger 

overall dose compared with the group that received 

photon treatment only.

49 6 Page 9 – line 60: “with 9 additional centers under construction.” 

I believe 10 centers are under construction (Mayo is 

constructing 2 centers in different locations).

We changed the number of centers under construction 

to 10.

50 6 Page 22: Row “Esophageal”. The Wang paper seems to indicate It is difficult to compare these two studies because they 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

118  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

that pulmonary complications are improved with PBT, but the 

McCurdy reports the opposite – that PBT is associated with 

greater complications – specifically pneumonitis. Given this 

conflicting data – should not the evidence considered 

inadequate?

report very different outcomes: Wang reports a 30-day 

broad composite outcome of any pulmonary 

complication and McCurdy reports a 3-month specific 

outcome of pneumonitis. We evaluated them separately 

for this reason. 

51 6 Page 23: Line 28: GI toxicity was not worse in the Yu 2013 

study, but in the Kim 2011 study. Also, the Yu 2013 study 

showed transient genitourinary benefits for proton beam (as 

noted above in comments 1 and 3)

Moved Yu 2013 citation to statement about similar 

acute toxicity at beginning of sentence; added evidence 

on transient GU advantage at 6 months.

52 6 Page 24: Line 11: again, the “PBT + Photon” would be clearer 

if it was “PBT boost + photon”. Though there were higher 

rectal bleeding with PBT boost, it was a higher dose of 

radiation (75.6 CGE vs 67.2 Gy). Duttenhaver et al also used a 

higher dose for the proton arm. This is also noted above in 

comment 3.

We altered our discussion of both Shipley et al. 1995 

and Duttenhaver et al 1983 to emphasize that these 

studies used a “boost” of protons in addition to photon 

therapy and that this combined group received a larger 

overall dose compared with the group that received 

photon treatment only.

53 6 Page 26: General comment re: CNS cancers. There is some 

evidence / consensus in the medulloblastoma literature that for 

pediatric cancers there is long term benefit in terms of IQ, etc. 

I’m not sure whether this is important for the VA ESP.

Thank you for this information, but reviewer is right 

that pediatric outcomes are not relevant to this review. 

54 6 Page 28: “Giant cell tumor” – would be better to entitle this 

section “Giant cell tumor of the bone.”

We made this change throughout the report.

55 6 Page 30: “Hemangioma” – would be better to entitle this 

section “uveal hemangioma.”

We made this change throughout the report.

56 6 Page 41: Line 34: Again, it should be emphasized that PBT and 

photon therapy was to a greater dose than photon therapy alone 

in the Shipley study. It also used older technique and is so dated 

that the findings are not helpful to the technology assessment of 

PBT in current practice.

We amended the discussion of PBT+photon versus 

photon alone in the prostate cancer section to include 

information on the delivered dose in each treatment 

group as well as noting that the radiotherapy techniques 

used in Duttenhaver 1983 and Shipley 1995 are 

outdated.

57 6 Page 49: Line 31: “Giant cell tumor” again would be better to 

write as “Giant cell tumor of the bone”

We made this change throughout the report.

58 6 Page 49: Line 37: “Hemangioma” again would be better as We made this change throughout the report.
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

119  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

“Uveal Hemangioma”

59 7 Overall this is a very well-researched and written summary 

report and reasonably current. As the report outlines, 4 key 

strategies were utilized, a literature review, a summary of 

clinical trials, review of coverage policies by various insurance 

companies, and review of prior summary reports, and this report 

clearly highlights one major limitation of any such exercise, 

that this field is extremely dynamic and fluid, and therefore, 

obsolence is a very rapid phenomenon, rendering most prior 

reports significantly out-of-date.

Thank you. 

60 7 As key examples of this rapid obsolence, while the report was 

being commissioned, several new pieces of data were presented 

at the PTCOG-NA meeting in October 2014, and most of these 

have not made it into the current report.

We thank the reviewer for this information. We 

reviewed the abstract book from the October 2014 

PTCOG-NA meeting and identified one comparative 

study on head and neck cancer which we now include 

in that section of the report (Abstract #33, P.B. 

Romesser, O. Cahlon, E. Scher, Y. Zhou, T. Leven, R. 

Wong, N. Riaz, S. McBride, N.Y. Lee).

61 7 A second important example of the rapidly shifting field, is the 

update to the NCCN guidelines; for example, the 2015 Prostate 

Cancer NCCN guidelines now include the following phrase: 

“conventionally fractionated prostate proton therapy can be 

considered a reasonable alternative to x-ray based regimens at 

clinics with appropriate technology, physics, and clinical 

expertise.” (The 2014 version provided as follows: “proton 

therapy is not recommended for routine use at this time. 

Research hasn’t shown proton beams to be the same or better 

for treating prostate cancer than conventional external beams.”) 

This is a very clear example of how within 1 year, there is a 

dramatic change even in NCCN coverage language relative to 

protons. Further examples of this significant and rapid change 

in recommendations will be highlighted in disease-specific 

categories.

Thank you for this information. We added the most 

recent guidelines issues by the NCCN regarding PBT 

for prostate cancer: “An ongoing prospective 

randomized trial is accruing patients and comparing 

prostate proton therapy to prostate IMRT. The NCCN 

panel believes there is no clear evidence supporting a 

benefit or decrement to proton therapy over IMRT for 

either treatment efficacy or long-term toxicity. 

Conventionally fractionated prostate proton therapy can 

be considered a reasonable alternative to X-ray based 

regimens at clinics with appropriate technology, 

physics, and clinical expertise.”

62 7 Pencil Beam Scanning (PBS) Intensity Modulated Proton We discuss PBS in the Introduction, but added a 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

120  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

Therapy (IMPT): In the last 3+ years, a dramatic technological 

change has occurred in this field, ie the rapid and almost 

immediate adaptation of PBS IMPT over the historic spread-out 

Bragg peak (SOBP) approach; this is widely, and almost 

universally recognized as having multiple advantages such as 

superior dosimetry, and more importantly the robust ability to 

expand the reach of proton therapy into various organ sites and 

disease categories, not historically treated with SOBP because 

of technical limitations; this major technological advance 

potentially has huge clinical implications and the development 

and consequences of this aspect have largely been ignored in 

this report; going forward, significant data are expected to 

emerge for PBS, and therefore, a focus on this technology is 

crucial for any such report.

mention of the lack of studies on the newer PBT 

delivery systems and methods (ie, pencil beam 

scanning) to the ‘Gaps in Evidence’ section of the 

Discussion

63 7 Breast Cancer: This disease has historically not been the focus 

of proton therapy, primarily because prior to approximately 3+ 

years ago, the technology did not permit easy delivery of whole 

breast and regional nodal irradiation, a scenario that has 

changed dramatically with the advent of PBS IMPT. Therefore, 

it is now technically feasible to expand the role of proton 

therapy to much beyond partial breast irradiation, which is what 

was primarily discussed in the VA report. Two key clinical 

facts have also created a “perfect storm”. There are now 2 

randomized clinical trials that underscore the fact that in several 

subcategories, regional nodal irradiation, especially internal 

mammary (IM) irradiation improves overall survival, making 

this an integral component of future therapeutic approaches 

(Budach, 2013). An unfortunate consequence of this is the 

guaranteed increase in cardiac dose, relative to treatment 

planning that avoids the internal mammary nodal chain. A 

seminal long-term cohort study published in the NEJM in 2013 

by Darby et al, very convincingly established that the adverse 

cardiac outcome risk level is directly proportional to the mean 

heart radiation dose, and therefore, although IM irradiation is 

Added: “With the advent of intensity-modulated proton 

therapy (IMPT), it may now be more feasible to expand 

the role of PBT beyond partial breast irradiation. 

However, we found no IMPT for whole breast or nodal 

irradiation. It will be important to consider whether 

IMPT can improve on the 7.4%-per-gray rate for major 

coronary events within the first 5 years observed in a 

population-based case-control study of 2168 women 

who underwent external radiotherapy for breast cancer 

between 1958 and 2001 in Sweden and Denmark. 56”
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

121  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

now the new standard, women receiving this will be expected to 

have a categorical increase in late adverse cardiac events, some 

delayed by 10+ years, as a consequence of increased mean heart 

dose. Numerous dosimetric studies in the literature have 

demonstrated the categorical, and unquestioned superiority of 

proton beam therapy over photon therapy (something that is 

analogous to an in-silico randomized trial) in terms of 

decreasing cardiac dose. Further, as more and more cardiotoxic 

systemic therapies, such as Adriamycin, and anti-her2 agents 

are combined in the management of non-metastatic breast 

cancer, the risk to these women of developing late cardiac 

toxicities should not be underestimated; in this context, relying 

on the low level evidence regarding the MGH partial breast 

irradiation papers is not an appropriate approach to making a 

determination regarding the development of VA 

recommendations for whole breast and nodal irradiation, 

especially because a short-sighted approach could result in 

subjecting a large number of women with breast cancer to late 

cardiotoxicity which may not become apparent for a decade or 

more, and in addition to the negative clinical consequences of 

this approach, the global cost of managing these cardiac 

toxicities would most likely exceed the up-front investment in 

treating such women with proton therapy. The VA should 

therefore seriously consider either launching its own trial, or 

whole-heartedly supporting the randomized and nonrandomized efforts currently in development to test proton vs 

photon therapy for breast cancer, at least for women who need 

whole breast and/or chest wall irradiation, especially in the 

event that they also need IM irradiation, and also for those 

women who have other underlying cardiac risk factors. This is 

an absolutely key healthcare consideration, and a major focus of 

the future VA effort, in cognizance of the dramatic increase of 

women as a proportion of our veterans.

64 7 Prostate Cancer: The biochemical progression free survival Yes, we agree that dose considerations are key and we 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

122  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

improves as function of dose, and modern conventionally 

fractionated approaches requires doses in the order of 78+ Gy.

are hopeful they are being better addressed in ongoing 

studies. The dose in the new Fang 2014 study, for 

example, was 79.2 Gy. 

7 Prostate Cancer: Retrospective comparisons of databases, most 

notably the SEER database suggests that the disease-related 

clinical outcomes are comparable for photon versus proton 

treated patients, and some toxicities might in fact be higher in 

frequency in the proton cohort. These retrospective 

comparisons, in fact, are seriously flawed; radiation toxicities 

are a function, among other things, of total dose. In the 

comparative studies that have been referenced, no dosecorrection was applied; in fact most proton patients were treated 

to higher doses. Further, most of the proton patients received a 

combination of photons and protons, and the reported toxicities 

are largely artefactual because of the fact that the photon treated 

patients were not on any trials, whereas several of the proton 

treated patients were, with the trial requiring a more intense 

level of scrutiny, including the use of endoscopies, one of the 

endpoints reported on by the Shelby study.

Yes we fully agree with this reviewer’s criticism and 

have discussed in detail their high potential for 

exposure and outcome misclassification biases

65 7 Prostate Cancer: The very large University of Florida

experience with protons which reports (in a non-randomized 

context, and therefore subject to bias) some of the best clinical 

outcomes for prostate cancer following radiotherapy, ever 

reported (Hoppe et al, IJROBP, 2014).

We clarified with the reviewer that he is referencing: 

Mendenhall NP, Hoppe BS, Romaine NC, et al. Fiveyear outcomes from 3 prospective trials of imageguided proton therapy for prostate cancer. Int J Radiat 

Oncol Biol Phys. 2014;88(3):596-602. This is a 

noncomparative study in which all patients received 

proton beam therapy. We had already been aware of 

this study, but did not discuss its findings because it did 

not compare outcomes with any other radiation 

modality. Although its results are promising, as the 

study authors themselves admit, it does not address 

remaining concerns about comparative effectiveness. 

66 7 Prostate Cancer: The elegant very recently reported casematched study of protons versus photons for prostate cancer Thank you for this information. We added Fang 2014 

to our analysis.
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

123  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

from the University of Pennsylvania, which does not show the 

increase in rectal/GI toxicities and colonoscopies attributed to 

the SEER study (Fang 2014).

67 7 Prostate Cancer: The very significant comparative effectiveness 

study of patient-reported outcomes after proton therapy or 

intensity-modulated radiotherapy for prostate cancer (Hoppe et 

al, cancer 2014) which compared patient-reported quality-oflife (QOL) outcomes after proton therapy (PT) and intensitymodulated radiation therapy (IMRT) for prostate cancer, 

performed on prospectively collected QOL data using the 

Expanded Prostate Cancer Index Composite (EPIC) 

questionnaire. PT was delivered to 1243 men at doses from 76 

to 82 Gy. IMRT was delivered to 204 men who were included 

in the Prostate Cancer Outcomes and Satisfaction with 

Treatment Quality Assessment (PROSTQA) study in doses 

from 75.6 Gy to 79.4 Gy. No differences were observed in 

summary score changes for bowel, urinary incontinence, 

urinary irritative/obstructive, and sexual domains between the 2 

cohorts. However, more men who received IMRT reported 

moderate/big problems with rectal urgency (P = 0.02) and 

frequent bowel movements (P = 0.05) than men who received 

PT. These outcomes highlight the need for further comparative 

studies of PT and IMRT.

We agree that the Hoppe et al study published in 

Cancer in 2014 provides important information on 

QOL outcomes for prostate cancer patients treated with 

IMRT or PBT. This study has already been included in 

our discussion of IMRT vs PBT for prostate cancer.

68 7 Prostate Cancer: Proton therapy has become widely accepted as 

a key standard of care by thousands of men with prostate 

cancer, especially when it comes to assessing patient 

satisfaction, and reporting patient-based outcomes; in fact the 

quality of data for proton therapy far exceeds anything that is 

available for photon therapy in this context, especially when 

one recognizes the “power of numbers”. As an example, at the 

National proton Conference in April 2014, the National 

Association of Proton Therapy (NAPT) released a patientcentered report on the outcomes following proton therapy for 

Thank you for this information. Since we did not 

include patient satisfaction as an outcome and the 

report that you reference did not compare prostate 

cancer patients that received photon modalities, this 

study does not meet our inclusion criteria.
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

124  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

prostate cancer in almost 4000 men treated at 12 different 

proton centers, with the following key observations:

1. 96% have recommended proton therapy to others

2. 98% rated their proton therapy experience as “excellent” or 

“good” 

3. 98% believed they made the best treatment decision for 

themselves

4. 96% were “satisfied” or “extremely satisfied” with their 

proton therapy treatment

5. 85% reported that their quality of life was “better than” or 

“the same as” before their treatment. 

Collectively, these data very strongly suggest that modern 

proton therapy, especially PBS, deserves to be seriously 

incorporated and evaluated as an option for men with prostate 

cancer, either in the context of a prospective longitudinal multiinstitutional cohort study, or a randomized trial, with the VA 

being a key player in either or both efforts, given the extremely 

high patient satisfaction ratings for this therapy.

69 7 Lung Cancer: For NSCLC, the factual elements in play are a 

categorical dose-response relationship favoring higher dose 

radiotherapy in terms of improved loco-regional control, but the 

review also categorically demonstrates that the major US trial 

seeking a survival benefit from this dose-escalation strategy 

failed; however, there are several important observations that 

are missed in this top-line conclusion:

a. There is already a previous, randomized phase III trial from 

China that shows a categorical survival improvement with 

higher doses (Chen et al., 2013).

b. More detailed analysis of the RTOG trial demonstrates that 

the likely cause of “endpoint failure” was excess toxicities 

associated with higher doses, especially higher cardiac and 

pulmonary doses (Cox et al., 2012).

c. An analysis of technique in the RTOG trial shows that when 

“tissue-sparing” IMRT techniques were employed (in 

We agree that these studies do not provide conclusive 

evidence on the comparative effectiveness of PBT and 

look forward to evidence from additional ongoing 

RCTs
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

125  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

comparison to 3-D techniques), patient reported QOL (using the 

FACT-L TOI tool) was superior, and translated to a survival 

benefit, underscoring the enormous significance of decreasing 

cardiac and lung dose, something that PBS has been shown in 

several dosimetric studies to achieve readily.

At this point, several randomized trials are evaluating the 

photon versus proton question in various stages and clinical 

presentation of NSCLC, and the VA would be ideally 

positioned to contribute to this in a major way.

70 7 Esophageal Cancer: The review appropriately summarizes the 

data, in particular highlighting the key observation of decreased 

mortality in the context of trimodality therapy, in favor of 

proton therapy, underscoring the need and value in supporting 

such an approach, ideally in a prospective multi-institutional 

registry trial, or perhaps even through a prospective randomized 

trial.

Thank you for confirming our findings. 

71 7 Medulloblastoma: This is an uncommon disease in the adult 

context; however, the data that exist show a very dramatic 

reduction in several toxicities, and given the very dramatic 

toxicities from photon therapy for large field irradiation, 

especially craniospinal, this should be considered as a default 

indication in favor of proton therapy.

Thank you for confirming our findings. 

72 7 Head and Neck Cancer: Emerging data from the ongoing 

randomized MDACC trial very strongly favor consideration of 

PBS IMPT in several forms of H/N cancer. The extent of 

reduction in radiation dose to several H/N substructures is very 

dramatic, and emerging data regarding acute toxicities and cost 

effectiveness lean toward IMPT. Given the relatively high 

incidence of this diagnosis in the VA patient population, a 

major opportunity exists to support the ongoing trials and 

categorically establish the value in this disease.

We look forward to the final results of the MDACC 

trial. 

73 7 Ocular Tumors: Although not reviewed, PBT is an obvious 

choice for these tumors.

Agreed. It was not reviewed because the VA already 

covers PBT for ocular melanomas. 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

126  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

74 7 Skull base and Brain Tumors: For several tumors in this 

category, such as chordoma, chondrosarcoma, and possibly 

craniopharyngioma, low grade glioma, meningioma, etc., PBS 

IMPT is the logical therapy; for example, the high radiation 

doses necessary to control chordoma simply cannot be 

delivered with photon therapy; meningiomas represented a 

logical target for PBS IMPT, given that radiation the the 

meningeal surface is one of the few known causes of this 

condition, and photons would irradiate a dramatically larger 

volume of meninges, etc.

Agreed. At least some of these tumors are already 

covered by the VA. 

75 7 Combination therapy approaches: Increasingly, in several 

malignancies, combination chemoradiotherapy is becoming the 

defacto standard. This combination is frequently associated 

with profound myelotoxicity and two pieces of data have 

recently become very clear:

a. Proton therapy decrease myelotoxicity in comparison to 

photon therapy.

b. Treatment-induced lymphopenia is associated with inferior 

survival in several disease types, and inadvertent irradiation of 

the circulating lymphocyte compartment is a major causative 

factor in this. PBT would very likely diminish this.

The additional recognition in the last 2+ years that upregulation of effector T cells can dramatically improve cancer 

outcomes in several malignancies underscores the major need to 

use therapies with the lowest likelihood of treatment-associated 

lymphopenia, and PBS would be much better suited for this 

purpose, compared to IMRT.

We confirmed with the reviewer that the data sources 

supporting his statement (a) about decreased 

myelotoxicity with proton vs photon are: Komaki R. 

Reduction of bone marrow suppression for patients 

with stage III NSCLC treated by protons and 

chemotherapy compared with IMRT and 

chemotherapy. The Particle Therapy Cooperative 

Group (PTCOG) 47 Conference; May 19-24 2008; 

Jacksonville, FL and Krause M, Baumann M. [Reduced 

acute toxicity for adults with medulloblastoma treated 

with proton beam craniospinal irradiation]. 

Strahlenther Onkol. 2014;190(1):111-112. As a 

conference abstract, Komaki 2008 lacks adequate detail 

about methodology to fully assess its internal validity 

(eg, patient selection and outcome ascertainment 

methods, baseline characteristics, statistical methods, N 

analyzed). Krause 2014 is in German and our methods 

do not include translation of non-English articles.

76 7 Conclusion: The purpose of this report is to synthesize the 

evidence supporting the role/value of proton therapy, 

specifically in terms of assisting the VA in making further 

recommendations as to how to approach this modality. My 

summary comments should provide strong rationale for 

Thank you. 
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

127  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

pursuing this in a logical, organized, and structured manner, and 

if done smartly, without a dramatic escalation in cost-of-care.

77 8 Overall, I think there is confusion in what is toxicity due to 

proton beam itself vs proton beam techniques. Proton beam is a 

relatively young technology. Although centers have been using 

this technology for 2+ decades, the overall number is low (just 

2 - Loma Linda and MGH). Of the 14 currently operating 

proton centers, 12 have been treating for less than 10 years. As 

a result, techiques are still evolving. In contrast, there are 

probably ~3000 RT centers in operation today. If you look at 

the metric of "proton years" vs "IMRT years", it's probably 150 

years vs 30,000 years. This is not meant to be a soapbox 

defense of proton beam, but rather an explanation of what 

should be considered by the reviewers.

Added to conclusion: “Because this is still a rapidly 

evolving field, with ongoing efforts to improve 

techniques and reduce costs, this review may need 

more frequent updating to keep up-to-date with 

emerging research” 

78 8 I think in lung cancer, the Sejpal MDA manuscript, along with 

other manuscripts from MDA, indicate that much higher doses 

of RT can be delivered with proton beam vs either 3D 

conformal RT or IMRT and result in much lower complication 

rates.

Yes, we agree that Sejpal found lower Grade 3 

esophagitis for PBT vs 3D-CRT and IMRT in a 

subgroup of 202 patients treated with concurrent 

chemotherapy. 

79 8 For prostate cancer, there are now 6 dose escalation studies. All 

6 showed that higher doses lead to higher biochemical control 

rates. But 5 of 6 studies showed increased GI complication risks 

with dose escalation. The 6th study used proton beam for the 

final RT boost (PROG 9509, Zietman JCO 2010) and showed 

the same complication rates in the low and high dose arms. In 

looking at these 6 studies together, it is very reasonable to 

conclude that the only method in which dose escalation in 

prostate cancer has been shown to feasible WITHOUT an 

increase in GI toxicity is with the use of proton beam. 

Yes, this is a reasonable inference to make based on 

comparing findings from high vs low dose comparisons 

of proton beam to high vs low dose comparisons of 

non-proton beam modalities; yet, that didn’t translate 

into an actual GI toxicity advantage based on direct 

evidence from a head-to-head comparison of proton 

beam vs IMRT in the newest Fang 2014 study. Added 

dose escalation studies to the report. 

80 8 If the Sheets article is to be used, then the comparison of the 6 

dose-escalation studies should also be used. The Sheets article 

has multiple flaws, not the least of which was the use of a 

Yes, we agree that the Sheets article has multiple flaws 

which we have described in detail and have cited the 

JAMA letters to the editor.
Comparative Effectiveness of Proton Irradiation Treatment Evidence-based Synthesis Program

128  

Comment 

#

Reviewer 

#

REVIEWER COMMENT RESPONSE

surrogate marker (colonoscopcies) to quantify GI morbidity. No 

one knows the indications for ordering the colonoscopies. The 

letter to the editor from Mendenhall, Shields, and Slater in 

JAMA outlined the flaws in the Sheets article.

Also added comparison of dose escalation studies: 

“Some dose-escalation studies have found increased GI 

complication risks with increased dose of photons,65

but not with protons.66 But when compared head-tohead in prostate cancer, proton beam has only 

transiently reduced risk of 6-month acute GU toxicity 

versus IMRT, but may increase risk of late GI toxicity 

after 4-5 years.”

81 8 Finally, secondary malignancies may well be lower with the use 

of proton beam RT. The article from Chung et al only examined 

one side, if protons had a higher rate of radiation induced 

malignanices than X-rays. They found a lower rate of second 

cancers in the matched populations.

Yes, Chung’s finding can seem promising, but it 

provided insufficient evidence to draw any conclusions 

at this time due to its high potential for unmeasured 

confounding due to higher missing data but superior 

outcome ascertainment methods in the proton group 

and unknown radiation dose.