# Referring Parkinson's disease patients for deep brain stimulation: a RAND/UCLA appropriateness study

Elena Moro, Michael Schüpbach, Tobias Wächter, Niels Allert, Roberto Eleopra, Christopher R. Honey, Mauricio Rueda, Mya C. Schiess, Yasushi Shimo, Peter Valkovic, et al.

<figure><img src="image_1.png" /></figure>

## To cite this version:

Elena Moro, Michael Schüpbach, Tobias Wächter, Niels Allert, Roberto Eleopra, et al. Referring Parkinson's disease patients for deep brain stimulation: a RAND/UCLA appropriateness study. Journal of Neurology, 2016, 263 (1), pp.112-119. 10.1007/s00415-015-7942-x. hal-01224805

HAL Id: hal-01224805

https://hal.sorbonne-universite.fr/hal-01224805

Submitted on 5 Nov 2015

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

ORIGINAL COMMUNICATION

# Referring Parkinson's disease patients for deep brain stimulation: a RAND/UCLA appropriateness study

Elena Moro<fnref n="1" /> · Michael Schüpbach<fnref n="2" />,<fnref n="3" /> · Tobias Wächter<fnref n="4" />,<fnref n="5" /> · Niels Allert<fnref n="6" /> ·

Roberto Eleopra<fnref n="7" /> · Christopher R. Honey<fnref n="8" /> · Mauricio Rueda<fnref n="9" /> · Mya C. Schiess<fnref n="10" /> ·

Yasushi Shimo<fnref n="11" /> · Peter Valkovic<fnref n="12" /> · Alan Whone<fnref n="13" /> · Herman Stoevelaar<fnref n="14" />

Received: 4 August 2015 / Revised: 4 October 2015 / Accepted: 13 October 2015
© The Author(s) 2015. This article is published with open access at Springerlink.com

**Abstract** In 2005, a European expert panel developed and validated an electronic tool to support the appropriate referral of patients with Parkinson's disease (PD) for the consideration of deep brain stimulation (DBS). Since new evidence has become available over the last decade an update of the tool is necessary. A world-wide expert panel (71 neurologists and 11 neurosurgeons) used the RAND/UCLA Appropriateness Method to assess the appropriateness of referral for 1296 scenarios (9-point scale). Scenarios were permutations of 8 clinical variables relevant to the decision of referral. Appropriateness of referral was calculated on the basis of the median score and the extent of agreement. Compared to 2005, the impact of clinical variables on the appropriateness of referral was similar for severity of on-off fluctuations, dyskinesias and refractory tremor (positive

association, *p* < 0.001), and cognitive impairment (negative association, *p* < 0.001). A relatively stronger negative impact was seen for levodopa-unresponsive gait and balance disturbances as well as older age, the latter most likely due to a higher cut-off value (75 versus 70 years in the previous study). The impact of PD duration on the appropriateness of referral was less pronounced than in 2005. The contribution of the newly included variable ‘non-motor side effects of anti-PD medication’ was very modest. Based on these results the panel produced new recommendations on the appropriateness of referral for the evaluation of DBS in PD patients. Differences from the previous study reflect the new clinical evidence, particularly related to the use of DBS in an earlier stage of PD. The validation of the updated recommendations is in progress.

<footnote>☒ Herman Stoevelaar
herman.stoevelaar@ismar.com</footnote>

<footnote n="1">¹ Division of Neurology, CHU of Grenoble, Joseph Fourier University, INSERM U836, Grenoble, France</footnote>

<footnote n="2">² Department of Neurology, Movement Disorders Center, Bern University Hospital and University of Bern, Bern, Switzerland</footnote>

<footnote n="3">³ Assistance-Publique Hôpitaux de Paris; Centre d'Investigation Clinique 9503, Institut du Cerveau et de la Moelle épinière; Département de Neurologie, Université Pierre et Marie Curie, Paris 6 et INSERM, CHU Pitié-Salpêtrière, Paris, France</footnote>

<footnote n="4">⁴ Department of Neurology and Hertie Institute for Clinical Brain Research, University of Tübingen, Tübingen, Germany</footnote>

<footnote n="5">⁵ Center of Rehabilitation, Bad Gögging, Germany</footnote>

<footnote n="6">⁶ Neurological Rehabilitation Center Godeshoehe, Bonn, Germany</footnote>

<footnote n="7">⁷ Department of Neurology, Santa Maria della Misericordia University Hospital, Udine, Italy</footnote>

<footnote n="8">⁸ Division of Neurosurgery, University of British Columbia, Vancouver, Canada</footnote>

<footnote n="9">⁹ Departamento de Neurociencias, Fundación Cardiovascular de Colombia, Bucaramanga, Colombia</footnote>

<footnote n="10">¹⁰ Department of Neurology, University of Texas Medical School at Houston, Texas, USA</footnote>

<footnote n="11">¹¹ Department of Neurology, Juntendo University School of Medicine, Tokyo, Japan</footnote>

<footnote n="12">¹² 2nd Department of Neurology, Comenius University Faculty of Medicine and University Hospital Bratislava, Bratislava, Slovakia</footnote>

<footnote n="13">¹³ Department of Neurology, Fenchay Hospital, Bristol, UK</footnote>

<footnote n="14">¹⁴ Centre for Decision Analysis and Support, Ismar Healthcare, Lier, Belgium</footnote>
**Keywords** Parkinson's disease · Deep brain stimulation · RAND/UCLA appropriateness method · Referral · Consultation

## Introduction

Deep brain stimulation (DBS) is an established treatment for well-selected patients with Parkinson's disease (PD) [1, 2]. Patient eligibility for DBS is determined by rigorous and standardised evaluation in specialised surgical movement disorder centres. However, patient pre-selection by non-specialised neurologists or other physicians is often not optimal due to lack of criteria or guidelines that are easy to use in daily practice. This may lead to inappropriate referrals [3, 4] but also to under-referrals if potentially eligible candidates are not given the opportunity to be evaluated in a specialised centre. A recent Swedish survey has revealed that three quarters of patients with advanced PD were not even informed of the possibility of having DBS therapy by their neurologists [5].

In 2005 a European expert panel used the RAND/UCLA Appropriateness Method (RUAM) [6] to develop patient-specific referral criteria for DBS consideration in patients with PD [7]. These criteria were subsequently embedded in an online decision support tool (“Stimulus”) [7]. An observational study in Germany and Spain showed that the selection rate for DBS was significantly higher for patients in which the Stimulus tool had been used compared to the unscreened population (77 versus 48 %, respectively) [8]. A recent single-centre study in the USA showed that Stimulus was superior to another tool in predicting DBS candidacy in patients with PD [9].

Over the last few years new evidence has become available on the efficacy of DBS, particularly in an earlier stage of PD [10], and with respect to non-motor fluctuations and symptoms, comprising fluctuations of mood and impulse control disorders [11]. These important data needed to be incorporated into an update of the Stimulus recommendations and the online tool. Given the world-wide application of DBS in PD, a global approach targeting at broadly applicable criteria was adopted. The aim of this study was to update the 2005 referral criteria and to develop common recommendations for DBS consideration in patients with PD.

## Methods

### Selection of panellists and construction of clinical scenarios

Similar to the European study [7], the RUAM [6, 12] was used to establish patient-specific referral criteria. An Executive Committee, consisting of three neurologists

(EM, TW, and MS) and a methodologist (HS) prepared the study design (Fig. 1). Three of the executive committee members were previously involved in the European study [7]. Panellists were recruited from a worldwide network of DBS implanting centres. The principal selection criterion was their active involvement in the selection of at least 15 patients with PD for DBS annually. Furthermore, a reasonable geographic spread was pursued. In total, 146 neurologists and 29 neurosurgeons received an invitation by email of whom 121 (69 %) responded. Of this group, 105 physicians (87 %) agreed to participate.

Clinical scenarios were based on those of the European study [7] with some adaptations and refinements. Scenarios were permutations of the values of eight variables considered relevant to the decision of referral (Table 1). Principal changes in comparison to the 2005 study were the adapted categories for age and PD duration, and the addition of the variable ‘non-motor side effects of anti-PD medication’ (Table 1). The population considered was restricted to patients who met the absolute criteria for DBS (Table 1).

### Panel process

The typical RUAM process consists of individual rating rounds and plenary face-to-face discussions [12]. The large size of our panel and the involvement of people from all over the world necessitated some practical adaptations, such as web conferences for the plenary meetings (Fig. 1).

Panellists used an online programme to individually rate the appropriateness of referral for 1296 different scenarios. The appropriateness scale ranged from 1 (extremely inappropriate) to 9 (extremely appropriate) with 5 being ‘uncertain’ or ‘equivocal’. Ratings had to be based on medical considerations only, and economic aspects had to be disregarded. Instructions to the ratings were accompanied by an overview of key peer-reviewed publications regarding DBS therapy for PD. After the ratings had been completed, data were analysed for patterns of appropriateness in relation to the clinical variables used.

A selection of 11 participants (Review Panel; “Appendix 1”) convened in Germany (November 2013) to discuss the results and to prepare a follow-up web conference for all panel members (January 2014). Thereafter, panellists were asked to review all their first round results and to adjust the values if thought to be necessary.

### Classification of appropriateness and statistical analysis

Classification of appropriateness was based on mathematical rules typically applied in RAND/UCLA
<figure><img src="image_4.png" /><figcaption>**Table 1** Definition of terms and variables/categories used for the construction of clinical scenarios</figcaption></figure>

<table><tr><td>Absolute criteria for the consideration of DBS</td></tr><tr><td>Parkinson's disease for at least 4 years</td></tr><tr><td>Presence of bothersome disease-related symptoms (motor fluctuations, dyskinesias, persisting tremor) and/or side effects related to anti-parkinsonian medication (hyperdopaminergic, anticholinergic)</td></tr><tr><td>Motor improvement with dopaminergic medication or presence of medically refractory tremor</td></tr><tr><td>Absence of medical conditions preventing surgery (e.g., terminal cancer, severe cardio-respiratory insufficiency)</td></tr><tr><td>Absence of ongoing severe, medically resistant neuropsychiatric diseases (e.g., severe depression, severe cognitive impairment)</td></tr><tr><td>Referral</td></tr><tr><td>Referral for the detailed evaluation of DBS therapy in patients with PD</td></tr><tr><td>Irrespective of the target (subthalamic nucleus, globus pallidus pars interna, ventral intermediate nucleus of the thalamus)</td></tr><tr><td>Variables used for the construction of clinical scenarios</td></tr><tr><td>Age (&lt;60 years; 60–74 years; ≥75 years)<sup>a</sup></td></tr><tr><td>PD duration since onset of symptoms (4–7 years; ≥7 years)<sup>b</sup></td></tr><tr><td>Parkinsonian signs during OFF periods, despite medical treatment (no–mild; moderate; severe impact on activities of daily living)</td></tr><tr><td>Dyskinesias (no–mild; moderate; severe impact on activities of daily living)</td></tr><tr><td>Tremor unresponsive to or requiring unacceptably high doses of anti-parkinsonian medication (no–mild; moderate; severe impact on activities of daily living)</td></tr><tr><td>Levodopa-unresponsive gait and balance abnormalities (no; yes)</td></tr><tr><td>Cognitive impairment (no–mild; moderate)</td></tr><tr><td>Non-motor side effects of anti-parkinsonian medication (no–mild; moderate–severe)<sup>c</sup></td></tr></table>

<footnote>a Age categories in 2005: &lt;60 years; 60–69 years; ≥70 years</footnote>

<footnote>b PD duration categories in 2005: &lt;5 years; ≥5 years</footnote>

<footnote>c Not included as a variable in 2005</footnote>
appropriateness studies [12]. Referral was classified as
appropriate if the median score was 7–9 without dis-
agreement, and as inappropriate if the median score was
1–3 without disagreement. All other outcomes were
deemed uncertain. Disagreement was defined as the situa-
tion in which at least one-third of the panellists had scored
in each of the extreme sections of the 9-point scale (1–3
and 7–9) [12]. To correct for potential asymmetric ratings,
the disagreement calculations were checked using the
interpercentile range adjusted for symmetry (IPRAS) for-
mula that has been advised for use in large-scale panels
[12]. Logistic regression was used to determine the rela-
tionship between clinical variables and appropriateness of
referral. All statistical analyses were performed using IBM
SPSS for Windows version 22.

Results

Participants and panel process

Of the 105 physicians who agreed to participate, 82 (78 %) completed all first round ratings. This voting panel consisted of 71 neurologists and 11 neurosurgeons from 28 countries (“Appendix 1”). Ninety-four percent reported ≥5 years of experience with patient selection for DBS, and 51 % mentioned to be involved in at least 25 evaluations annually. The first online rating round was conducted in October–November 2013. Panellists needed on average 3–4 h for completing the ratings. Discussion of the results by the Review Panel and during the web conference revealed that the structure and variables were adequate, but that different perspectives had been used while doing the ratings (appropriateness of DBS versus the appropriateness of referral). Panellists were asked to review all ratings taking the perspective of appropriateness of referral as the starting point. Second round ratings were completed in April 2014.

Agreement and appropriateness

Using the RUAM classical calculation, disagreement after
the second round was found for only two out of 1296
scenarios (0.2 %). Application of the IPRAS formula
resulted in disagreement for 1.2 % of scenarios. However,
the two approaches never led to different appropriateness
outcomes.

Referral for DBS consideration was deemed inappro-
priate for 15 % of the scenarios, appropriate for 46 % of
the scenarios, and uncertain for the remaining 39 %.
Appropriateness figures by clinical variables are given in
Table 2. Logistic regression analysis (appropriate versus
uncertain/inappropriate) revealed a consistent pattern of

factors determining the appropriateness of referral (pre-
dictive value 96.5 % at a cut-off value of 0.5). Whereas the
severity of OFF symptoms, dyskinesias and tremor showed
a pronounced positive association with appropriateness, a
negative impact was found for cognitive impairment,
levodopa-unresponsive gait and balance problems, and
higher age (Table 3).

Though statistically significant, the contribution of PD duration and presence of non-motor side effects of antiparkinsonian medication was small. No meaningful interaction effects between the variables, including age and PD duration, were seen.

Differences by subgroups of raters

Neurosurgeons showed higher rates for referral consider-
ation than neurologists (61 versus 43%; *p* < 0.001); sim-
ilar figures were seen for the Review Panel (59%) versus
other participants (43%; *p* < 0.001). Variations by geo-
graphic regions could not be assessed due to the small
number of participants per area.

Online educational tool

Panel results were embedded in an online tool that allows the user to select a patient profile and to see the related panel recommendation and additional information on patient selection for DBS. The tool can be freely accessed via https://www.earlestimulus.com.

Discussion

Our study provides practical criteria to reduce inappropri-
ate referrals and to support referral of appropriate candi-
dates for DBS in patients with PD. This is particularly
relevant since there is a strategic need for physicians and
patients to improve the process quality of PD referrals for
DBS.

The previously developed Stimulus tool has been shown
to be able to considerably improve the quality of pre-se-
lection by referral neurologists [8, 9]. The current study has
not only updated the referral criteria taking into account
new evidence from recent clinical studies [10, 11], but it
has also received wide feedback for a worldwide applica-
bility by involving expert neurologists and neurosurgeons
from many countries around the world. Although some
differences by subgroups were seen, this large-scale panel
reached considerable agreement on the appropriateness of
referral for DBS consideration in patients with PD. The
relatively large area of uncertainty (39 % of indications)
was not due to opposite opinions, but to “middle-of-the-
road” ratings reflecting situations in which potential
<table><caption>**Table 2** Appropriateness of referral for DBS by clinical variables</caption><thead><tr><th>Variable</th><th>Categories</th><th>Inappropriate %</th><th>Uncertain %</th><th>Appropriate %</th><th>p value<sup>a</sup></th></tr></thead><tbody><tr><td rowspan="3">Age</td><td>&lt;60 years</td><td>2</td><td>30</td><td>68</td><td rowspan="3">&lt;0.001</td></tr><tr><td>60-74 years</td><td>3</td><td>40</td><td>57</td></tr><tr><td>≥75 years</td><td>41</td><td>47</td><td>12</td></tr><tr><td rowspan="2">PD duration</td><td>4-7 years</td><td>16</td><td>40</td><td>44</td><td rowspan="2">0.279</td></tr><tr><td>≥7 years</td><td>15</td><td>38</td><td>48</td></tr><tr><td rowspan="3">OFF symptoms</td><td>Mild</td><td>22</td><td>41</td><td>38</td><td rowspan="3">&lt;0.001</td></tr><tr><td>Moderate</td><td>15</td><td>41</td><td>45</td></tr><tr><td>Severe</td><td>10</td><td>36</td><td>55</td></tr><tr><td rowspan="3">Dyskinesias</td><td>Mild</td><td>24</td><td>39</td><td>38</td><td rowspan="3">&lt;0.001</td></tr><tr><td>Moderate</td><td>14</td><td>43</td><td>43</td></tr><tr><td>Severe</td><td>9</td><td>35</td><td>57</td></tr><tr><td rowspan="3">Tremor</td><td>No/mild</td><td>25</td><td>41</td><td>35</td><td rowspan="3">&lt;0.001</td></tr><tr><td>Moderate</td><td>15</td><td>42</td><td>43</td></tr><tr><td>Severe</td><td>7</td><td>34</td><td>59</td></tr><tr><td rowspan="2">Gait/balance problems</td><td>No</td><td>9</td><td>26</td><td>65</td><td rowspan="2">&lt;0.001</td></tr><tr><td>Yes</td><td>22</td><td>52</td><td>26</td></tr><tr><td rowspan="2">Cognitive impairment</td><td>No/mild</td><td>6</td><td>28</td><td>66</td><td rowspan="2">&lt;0.001</td></tr><tr><td>Moderate</td><td>25</td><td>50</td><td>26</td></tr><tr><td rowspan="2">Non-motor side effects</td><td>No/mild</td><td>18</td><td>39</td><td>44</td><td rowspan="2">0.085</td></tr><tr><td>Moderate-severe</td><td>13</td><td>39</td><td>48</td></tr></tbody></table>

Percentages apply to the complete set of clinical scenarios (N = 1296)

Row totals per variable are 100 %, but may slightly deviate due to round-offs

<sup>a</sup> Pearson's Chi-square test for categorical data

<table><caption>**Table 3** Determinants of the panel outcome “appropriate” versus “uncertain/inappropriate”</caption><thead><tr><th>Variable</th><th>Value</th><th>β</th><th>SE</th><th>p value</th></tr></thead><tbody><tr><td rowspan="2">Age</td><td>60-74 years</td><td>-2.64</td><td>0.41</td><td rowspan="2">&lt;0.001</td></tr><tr><td>≥75 years</td><td>-16.06</td><td>1.42</td></tr><tr><td rowspan="2">PD duration</td><td>≥7 years</td><td>1.27</td><td>0.32</td><td rowspan="2">&lt;0.001</td></tr><tr><td>Moderate</td><td>1.97</td><td>0.40</td></tr><tr><td rowspan="2">OFF symptoms</td><td>Severe</td><td>4.89</td><td>0.55</td><td rowspan="2">&lt;0.001</td></tr><tr><td>Moderate</td><td>1.53</td><td>0.38</td></tr><tr><td rowspan="2">Dyskinesias</td><td>Severe</td><td>5.47</td><td>0.59</td><td rowspan="2">&lt;0.001</td></tr><tr><td>Moderate</td><td>2.10</td><td>0.38</td></tr><tr><td rowspan="2">Refractory tremor</td><td>Severe</td><td>7.25</td><td>0.72</td><td rowspan="2">&lt;0.001</td></tr><tr><td>Moderate</td><td>-11.34</td><td>1.02</td></tr><tr><td>Levodopa-unresponsive gait/balance abnormalities</td><td>Yes</td><td>-11.34</td><td>1.02</td><td>&lt;0.001</td></tr><tr><td>Non-motor side effects of anti-parkinsonian medication</td><td>Moderate-severe</td><td>1.09</td><td>0.31</td><td>&lt;0.001</td></tr><tr><td>Cognitive impairment</td><td>Moderate</td><td>-11.60</td><td>1.04</td><td>&lt;0.001</td></tr><tr><td>Constant value</td><td></td><td>7.04</td><td>0.85</td><td></td></tr></tbody></table>

Outcomes of logistic regression analysis

Reference classes for regression: age: <60 years; PD duration: 4-7 years; OFF symptoms: mild; Dyskinesias: mild; Refractory tremor: no/mild; Levodopa-unresponsive gait/balance abnormalities: no; Non-motor side effects of anti-parkinsonian medication: no/mild; Cognitive impairment: no/mild

advantages and drawbacks of referral were considered to be equivocal. However, the appropriateness figures relate to a theoretical population, and the distribution of clinical scenarios in real-life practice may differ considerably. Moreover, some scenarios may reflect frequent

constellations whereas others may be rather “constructed” and rarely seen in daily clinical practice.

Regression analysis showed consistent and logical associations between the clinical variables and appropriateness outcomes. As no meaningful interaction effects
<figure><img src="image_5.png" /><figcaption>**Fig. 2** Panel outcomes for selected patient profiles</figcaption></figure>

were detected, the impact of the variables on appropriateness of referral for DBS was merely cumulative. That means that the sum of positive and negative factors, with their different weights, determines the appropriateness category. To illustrate this result, the panel outcomes (appropriateness category and median score) for five selected profiles are projected against the patient characteristics in Fig. 2.

The comparison with the results from our previous study [7] suggests similar patterns for the impact of the severity of OFF symptoms, dyskinesias and refractory tremor (positive association), as well as of cognitive impairment (negative association). In this current study, a relatively stronger negative impact was seen for older age and levodopa-unresponsive gait and balance disturbances. For older age, the difference is most likely to be ascribed to the higher cut-off value (75 versus 70 years in the previous study) as the general trend goes to including older patients.

With the new clinical evidence showing the benefits of DBS at an earlier stage of PD [10, 13], criteria of DBS selection concerning PD duration and severity have changed. This is reflected in the less pronounced impact of PD duration on the appropriateness of referral in our study in comparison to the view of the initial panel [7]. The panel ratings also indicate that these recent findings on the effect of early surgical treatment [10] are readily implemented by experts worldwide in spite of a controversial debate on this topic [14, 15].

As evidence became available that DBS may also be useful in improving non-motor side effects of anti-

parkinsonian medication [11, 16], we included this condition in the construction of clinical scenarios. However, its impact on the appropriateness of referral, though statistically significant in multivariate logistic regression, was very modest. This finding could reflect the insufficient distinction made by neurologists between hyperdopaminergic behavior induced by anti-PD medications (that is supposed to improve with subthalamic nucleus (STN) DBS due to reduction of medications) and impulsivity occasionally induced by STN DBS (that may occur under inappropriate STN-DBS programming) [17].

Other factors used by the panel study, including the absolute criteria for the consideration of DBS for PD (bothersome symptoms, motor improvement with dopaminergic medication or presence of medically refractory tremor, absence of medical conditions preventing surgery, absence of severe medically resistant neuropsychiatric diseases) are comparable to the initial study [7] and have not undergone substantial changes over the last decade [18, 19].

The dissemination of the panel recommendations via a quick and user-friendly online tool has already proven its usefulness in a European setting [8]. The new updated tool, established now with the involvement of a world-wide panel, may have a larger geographic reach and impact. However, the authors' personal experience with the first Stimulus tool has shown that users frequently misinterpreted an appropriate outcome as being synonymous to "eligible for DBS". Therefore, we would like to emphasise that the tool is designed to assess the appropriateness of
referral to a specialised DBS centre for further evaluation, and not for surgery itself.

The most important limitations of this study are related to the subjective nature of the panel opinions, and selection of panel members may, therefore, influence the outcomes. Indeed, new published guidelines or consensus about selection of patients for DBS are still lacking, despite the available evidence concerning efficacy of DBS on earlier PD stages and on non-motor fluctuations. However, various studies have shown that agreement between RUAM panels of similar composition, even if from different countries or continents, is usually satisfactory to good [20–23]. Comparison of our initial and current study into DBS/PD confirms on the one hand the reproducibility of the study results over time and across countries, but also suggests sensitivity to change in scientific insights.

In conclusion, patient eligibility for DBS is determined in specialised surgical centres following a rigorous and extensive evaluation that is often challenging for the patient and the family, both physically and emotionally. The use of clear and practical criteria for pre-selection by general neurologists or other physicians may reduce the number of inappropriate referrals, and may also help to avoid under-referral of potentially appropriate candidates. A large world-wide panel has produced detailed and consistent recommendations on the appropriateness of referral for the consideration of DBS in patients with PD. Differences in comparison to the initial panel study reflect new clinical evidence, particularly in relation to the use of DBS in an earlier stage of PD. The use of a simple online tool may help to disseminate the panel recommendations and to increase the quality of pre-selection for DBS. Research on validating the updated recommendations is in progress.

## Compliance with ethical standards

**Funding** This study was supported by Medtronic.

**Conflicts of interest** Elena Moro received honoraria from Medtronic for consulting service and lecturing. Michael Schüpbach received lecture fees from Medtronic and travel reimbursements from Merz Pharma, Boston Scientific, and Medtronic, received research grants from “Parkinson Schweiz”, Boston Scientific, and Medtronic and has been serving as a consultant for Medtronic, Aleva, and Lundbeck. Tobias Wächter received honoraria from Medtronic, Pharm-Allergan and Ipsen Pharma for consulting service, lecturing and travelling. Niels Allert received honoraria from Medtronic for consulting service and lecturing. Roberto Eleopra reported no conflict of interest. Christopher R. Honey received research funding from Medtronic and St. Jude’s Medical and speaking honoraria from Medtronic. Mauricio Rueda received honoraria for lecturing from Medtronic. Mya C. Schiess received educational scholarships for training fellows and honoraria for lecturing from Medtronic. Yasushi Shimo received honoraria from Medtronic for lecturing. Peter Valkovic received honoraria from Medtronic for consulting service and lecturing. Alan Whone received honoraria from Medtronic for consulting service and travelling. Herman Stoevelaar received

honoraria from Medtronic for his advice to the design of the study and for data analysis.

**Ethical standard** Mentioned studies have been approved by the appropriate ethics committee and have therefore been performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments.

**Open Access** This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

# Appendix 1

## Participants to the panel study

Niels Allert (Germany),² Vladimir Alexeyevets (Belarus), Gabriel Arango (Colombia), Keyoumars Ashkan (United Kingdom), Merek Baláž (Czech Republic), Luigi Bartolomei (Italy), Oscar Bernal (Colombia), Rupam Borgohain (India), Alfonso Castro (Spain), Roberto Ceravolo (Italy), Haibo Chen (China), Ling Chen (China), Shin-Yan Chen (Taiwan), Miguel Coelho (Portugal), Arthur Cukiert (Brazil), Carsten Eggers (Germany), Roberto Eleopra (Italy),² Amalia Ene (Romania), Alfonso Fasano (Canada), Tao Feng (China), Tom Foltynie (United Kingdom), Alexandre Francisco (Brazil), Milan Grofik (Slovakia), Chris Honey (Canada),² Wolfgang Jost (Germany), Merete Karlsbor (Denmark), Asha Kishore (India), Peter Klivenyi (Hungary), Karina Knudsen (Germany), Norbert Kovacs (Hungary), Pierre Krystkowiak (France), Max Lange (Germany), Nilton Lara (Brazil), ChongSik Lee (Korea), Xiaoguang Luo (China), Yasushi Miyagi (Japan), Claudia Moreno, (Colombia), Francesca Morgante (Italy), Elena Moro (France),¹,² Mariana Moscovich (Brazil), Stefania Nassetti (Italy), Martin Nevrlý (Czech Republic), Genko Oyama (Japan), Claudio Pacchetti (Italy), Berta Pascual (Spain), Nicola Pavese (United Kingdom), Manuela Pilleri (Italy), Walter Pirker (Austria), Monika Pötter-Nerger (Germany), Carlos Rieder (Brazil), Mayela Rodriguez (Korea), Maria Jose Rosas (Portugal), Simone Rossi (Italy), Mauricio Rueda (Colombia),² Hidemoto Saiki (Japan), Patrick Santens (Belgium), Mya Schiess (United States),² Michael Schüpbach (Switzerland),¹,² Maria Chiara Sensi (Italy), Yasushi Shimo (Japan),² Friederike Sixel-Döring (Germany), Martin Südmeyer (Germany), Chun-Hwei Tai (Taiwan), Atsushi Takeda (Japan), Gertrud Tamas (Hungary), Giorgio Tommasi (Italy), Wesley Thevathasan (Australia), Atsushi Umemura (Japan), Peter Valkovic (Slovakia),² Franco Valzania (Italy), Tobias Wächter
(Germany),<fnref n="1" />,<fnref n="2" /> Pettarusp Wadia (India), Jian Wang (China), Alan Whone (United Kingdom),<fnref n="2" /> Håkan Widner (Sweden), Tatiana Witjas (France), Karsten Witt (Germany), Qin Xiao (China) Ludvic Zrinzo (United Kingdom), and Xian-Lun Zhu (Hong Kong).

## References

1. Fox SH, Katzenschlager R, Lim SY, Ravina B, Seppi K, Coelho M, Poewe W, Rascol O, Goetz CG, Sampaio C (2011) The Movement Disorder Society evidence-based medicine review update: treatments for the motor symptoms of Parkinson's disease. Mov Disord 26(Suppl 3):S2-S41

2. Fasano A, Daniele A, Albanese A (2012) Treatment of motor and non-motor features of Parkinson's disease with deep brain stimulation. Lancet Neurol 11:429-442

3. Okun MS, Fernandez HH, Pedraza O, Misra M, Lyons KE, Pahwa R, Tarsy D, Scollins L, Corapi K, Friehs GM, Grace J, Romrell J, Foote KD (2004) Development and initial validation of a screening tool for Parkinson disease surgical candidates. Neurology 63:161-163

4. Katz M, Kilbane C, Rosengard J, Alterman RL, Tagliati M (2011) Referring patients for deep brain stimulation: an improving practice. Arch Neurol 68:1027-1032

5. Lökk J (2011) Lack of information and access to advanced treatment for Parkinson's disease patients. J Multidiscip Healthc 4:433-439

6. Brook RH, Chassin MR, Fink A, Solomon DH, Kosecoff J, Park RE (1986) A method for the detailed assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care 2:53-63

7. Moro E, Allert N, Eleopra R, Houeto JL, Phan TM, Stoevelaar H, International Study Group on Referral Criteria for DBS (2009) A decision tool to support appropriate referral for deep brain stimulation in Parkinson's disease. J Neurol 256:83-88

8. Wächter T, Mínguez-Castellanos A, Valldeoriola F, Herzog J, Stoevelaar H (2011) A tool to improve pre-selection for deep brain stimulation in patients with Parkinson's disease. J Neurol 258:641-646

9. Coleman RR, Kotagal V, Patil PG, Chou KL (2014) Validity and efficacy of screening algorithms for assessing deep brain stimulation candidacy in Parkinson disease. Mov Disord Clin Pract 1:342-347

10. Schuepbach WM, Rau J, Knudsen K, Volkmann J, Krack P, Timmermann L, Hälbig TD, Hesekamp H, Navarro SM, Meier N, Falk D, Mehdorn M, Paschen S, Maarouf M, Barbe MT, Fink GR, Kupsch A, Gruber D, Schneider GH, Seigneuret E, Kistner A, Chaynes P, Ory-Magne F, Brefel Courbon C, Vesper J, Schnitzler A, Wojtecki L, Houeto JL, Bataille B, Maltête D, Damier P, Raoul S, Sixel-Doering F, Hellwig D, Gharabaghi A, Krüger R, Pinsker MO, Amtage F, Régis JM, Witjas T, Thobois S, Mertens P, Kloss M, Hartmann A, Oertel WH, Post B, Speelman H, Agid Y, Schade-Brittinger C, Deuschl G, EARLYSTIM Study Group (2013) Neurostimulation for Parkinson's disease with early motor complications. N Engl J Med 368:610-622

11. Lhommée E, Klinger H, Thobois S, Schmitt E, Ardouin C, Bichon A, Kistner A, Fraix V, Xie J, Aya Kombo M, Chabardès S, Seigneuret E, Benabid AL, Mertens P, Polo G, Carnicella S, Quesada JL, Bosson JL, Broussolle E, Pollak P, Krack P (2012) Subthalamic stimulation in Parkinson's disease: restoring the balance of motivated behaviours. Brain 135(Pt 5):1463-1477

12. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lázaro P, Van het Loo M, McDonnell J, Vader JP, Kahan JP (2001) The RAND/UCLA appropriateness method User's manual. RAND Corporation, Santa Monica

13. Schüpbach WM, Maltête D, Houeto JL, du Montcel ST, Mallet L, Welter ML, Gargiulo M, Béhar C, Bonnet AM, Czernecki V, Pidoux B, Navarro S, Dormont D, Cornu P, Agid Y (2007) Neurosurgery at an earlier stage of Parkinson disease: a randomized, controlled trial. Neurology 68:267-271

14. Mestre TA, Espay AJ, Marras C, Eckman MH, Pollak P, Lang AE (2014) Subthalamic nucleus-deep brain stimulation for early motor complications in Parkinson's disease-the EARLYSTIM trial: early is not always better. Mov Disord 29:1751-1756

15. Schüpbach WM, Rau J, Houeto JL, Krack P, Schnitzler A, Schade-Brittinger C, Timmermann L, Deuschl G (2014) Myths and facts about the EARLYSTIM study. Mov Disord 29:1742-1750

16. Thobois S, Ardouin C, Lhommée E, Klinger H, Lagrange C, Xie J, Fraix V, Coelho Braga MC, Hassani R, Kistner A, Juphard A, Seigneuret E, Chabardès S, Mertens P, Polo G, Reilhac A, Costes N, LeBars D, Savasta M, Tremblay L, Quesada JL, Bosson JL, Benabid AL, Broussolle E, Pollak P, Krack P (2010) Non-motor dopamine withdrawal syndrome after surgery for Parkinson's disease: predictors and underlying mesolimbic denervation. Brain 133(Pt 4):1111-1127

17. Schüpbach WM (2012) Impulsivity, impulse control disorders, and subthalamic stimulation in Parkinson's disease. Basal Ganglia 2:205-209

18. Bronstein JM, Tagliati M, Alterman RL, Lozano AM, Volkmann J, Stefani A, Horak FB, Okun MS, Foote KD, Krack P, Pahwa R, Henderson JM, Hariz MI, Bakay RA, Rezai A, Marks WJ Jr, Moro E, Vitek JL, Weaver FM, Gross RE, DeLong MR (2011) Deep brain stimulation for Parkinson disease: an expert consensus and review of key issues. Arch Neurol 68:165

19. Wagle Shukla A, Okun MS (2014) Surgical treatment of Parkinson's disease: patients, targets, devices, and approaches. Neurotherapeutics 11:47-59

20. Lawson EH, Gibbons MM, Ko CY, Shekelle P (2012) The appropriateness method has acceptable reliability and validity for assessing overuse and underuse of surgical procedures. J Clin Epidemiol 65:1133-1143

21. McDonnell J, Stoevelaar HJ, Bosch JL, Kahan JP, Dutch Panels on the Appropriate Treatment of BPH; European Panels on the Appropriate Treatment of BPH (2001) The appropriateness of treatment of benign prostatic hyperplasia: a comparison of Dutch and multinational criteria. Health Policy 57:45-56

22. Quintana JM, Arostegui I, Azkarate J, Goenaga JI, Elexpe X, Letona J, Arcelay A (2000) Evaluation of explicit criteria for total hip joint replacement. J Clin Epidemiol 53:1200-1208

23. Vader JP, Burnand B, Froehlich F, Dupriez K, Larequi-Lauber T, Pache I, Dubois RW, Gonvers JJ, Brook RH (1997) Appropriateness of upper gastrointestinal endoscopy: comparison of American and Swiss criteria. Int J Qual Health Care 9:87-92

<footnote n="1">¹Member of the Executive Committee</footnote>

<footnote n="2">²Member of the Review Panel</footnote>