MEMORANDUM

TO: Michael Wroblewski

, Craig Caplan , Sabrina Ahmed

FROM: Wil Lim

, Gene Rich

, Jeff Ballou

, Bob Schmitz DATE: 3/ 2 7 /2013

SUBJECT : C omparability Peer Grouping: Refinements and Assessments

The purpose of this memo is to describe results from refinements to the comparability peer grouping (CPG) benchmarking methodology. These refinements include different (1) specialty

mix and (2) practice size categorizations. Applying these refinements, we assessed the following

aspe cts of CPG benchmarking: peer groups, benchmarks, and cost tiers. We assessed peer groups constructed from the national sample of TINs ( that is, those with at least 1 eligible professional,

or EP) . We assessed peer groups, benchmarks, and cost tiers based on the sample of 870 TINs in

a nine

- state region with at least 25 EPs

.

Key Findings

1. When practices are characterized by the share of EPs from each CMS specialty represented in the practice, peer groups of at least 30 peers with identical specialty configur ations can be formed for 93% percent of practices nationwide

.

2. Constructing peer groups based on practice size in addition to specialty mix has a relatively small impact on peer groups and benchmarks.

3. Constructing CPG benchmarks based on peer specialty mix reduces differences between practices’ per capita costs and their benchmark means relative to using a national

benchmark , on average. This reflect s an increase in peer comparability moving from a national peer group to CPG peer groups

.

4. Medical group practi ces dominated by specialties that are thought to be disproportionately represented among outliers when using a national benchmark appear much less likely to be categorized as outliers under CPG.

5. Although results relating to benchmarks and cost tiers are in ferred from the 870

- TIN sample, these are potentially conservative estimates of the ability of CPG to compute benchmarks from more comparable peers. This is because larger sample s will generally increase the number of peers with any given specialty configu ration

.MEMO TO: Michael Wroblewski, Craig Caplan, Sabrina Ahmed

FROM: Wil Lim, Gene Rich, Jeff Ballou, Bob Schmitz

DATE: 3/27/2013

PAGE: 2

Description of Refinements

To determine peers, for each practice we computed the percentage of EPs in each specialty,

which we then classified into one of six categorical percentage groupings : 0 % , 1

- 20%, 21

- 40%,

41

- 60%, 61

- 80%, and 81

- 100%. Each pr actice’s profile is then determined by the percentage grouping for each specialty category. Note that in two of the categorizations we tested, we

restricted profiling to be based on physicians only. We tested the following three specialty categorizations:

1. “Part B Drug Specialties ”

a. Allergy

- immunology; infectious disease; rheumatology; hematolog y; medical oncology; hematology/ oncology

b. Other physician specialties

c. Non

- physicians

2. “All CM S Specialties”

a. Each of the CMS physician specialties

3. “15 Specialties”

a. Primary c are p hysicians (consisting of g eneral p ractice, f amily p ractice, i nternal

m edicine

, and g eriatric m edicine)

b. E mergency m edicine

c. C ardiology

d. G eneral s urgery

e. P sychiatry

f. A nesthesiology

g. O rthopedic s urgery h. N eurology

i. D iagnostic r adiology j. G astroenterology

k. H em atology/ o nc ology ( includes h ematology/ o ncology, h ematology, o ncology)

l. Urology

m. O bstetrics / gynecology

n. Pediatrics

o. Other ( includes all other physician specialties)

Similarly, we tested s tratifying practices based on practice size. We tested the following practice size categorizations:

1. National sample

a. 1+ EP s b. 1

- 24 EPs ; 25

- 99 EPs ; 100+ EP sMEMO TO: Michael Wroblewski, Craig Caplan, Sabrina Ahmed

FROM: Wil Lim, Gene Rich, Jeff Ballou, Bob Schmitz

DATE: 3/27/2013

PAGE: 3

c. 1

- 24 EPs ; 25

- 49 EPs ; 50

- 99 EPs ; 100+ EP s

2. 870

- TIN sample

a. 25+ EP s b. 25

- 99 EPs ; 100+ EP s

c. 25

- 49 EPs ; 50

- 99 EPs ; 100+ EP s

CPG Peer Groups

Table 1 shows , for each specialty and practice categorization, the distribution of comparability levels required to attain a peer group of 30 practices (inclusive of the practice itself) . In general, the finer the categorization, the more levels necessary to complete a peer group of at least 3 0 practice s. Of the 870 TINs that were categorized b ased on Part B Drug

S pecialties and regardless of practice size, 88 percent ( 766 ) had at least 30 peers in their same

“ bucket ”, where a bucket is defined as having the same specialty percentage categorizations

. No practices categorized based on 15 S pecialt ies categories or A ll CMS S pecialt ies categories had at

leas t 30 peers in their same bucket. I t is important to note that, by definition, coarser specialty categori zations will always have more peers in the same bucket. Since the Part B Drug Specialties categorization is coarser then the 15 Specialt ies and All CMS Specialt ies categorizations, it inherently has more practices in each bucket than the others. Thus, this metric

is of limited use for assessing the relative superiority of one specialty categorization over another

.

Table 1 also shows the corresponding peer groups constructed from the national sample of

TINs. Nearly all practices that were categorized based on Part B Drug S pecialties and regardless

of practice size had at least 30 peers in their same bucket , and 96 percent of practices that were categorized based on the 15 S pecialt ies categories and regardless of practice size had at least 30 peers in their same bucket. For the A ll CMS Specialties categories, this percentage was 93

percent

.

Table 1 also shows that these “ completion percentages” vary by practice size. Using the All

CMS Specialties categories to peer group, 94 percent of practices with 1

- 24 EPs, 34 percent of practices wi th 25

- 99 EPs, and 7 percent of practices with 100+ EPs had at least 30 peers in their

same bucket. There are two main reasons for this. First, there are many more small practices than large practices

— 97 percent of practices have fewer than 25 EPs. Second, smaller practices have inherently less varied specialty mixes than larger, often multi

- specialty, practices. The latter has two implications: small and large practices tend not to be peer

- grouped together , and large practices have fewer peers that are very similar in specialty mix

.

In summary, we would be able to construct peer groups consisting of other practices in the

same bucket for over 90 percent of practices , even when using the A ll CMS SpecialtiesMEMO TO: Michael Wroblewski, Craig Caplan, Sabrina Ahmed

FROM: Wil Lim, Gene Rich, Jeff Ballou, Bob Schmitz

DATE: 3/27/2013

PAGE: 4

categories. Moreove r
, stratifying also by practice size when creating peer groups has a relatively

small effect.

Benchmarks : CPG vs. National Average

One useful metric for assess ing the comparab i l ity of practices to their peer groups is the average per capita cost difference between practices and their benchmark ( peer group performance) . For example, dermatologists have substantially lower per capita costs than other specialties on average because the patients they see tend to be healthier in a way that might not be fully captured by HCC scores . However, if we were instead to compare a practice having a large proportion of dermatologists to other practices also having a large proportion of dermatologists, we expect the difference in per capita costs to be lower. Furthermore, we expect

that these differenc es would be driven more by actual variation in performance, since the types of patients seen are likely to be more similar . Thus, as we improve comparability of peer groups,

we expect deviations from benchmarks to diminish as we reduce the component due to treating different types of patients and retain differences due to actual performance.

Table 2 shows that per capita cost differences between practices and their peer groups is smaller, and in many cases substantially so, when benchmarking against a comp arable peer group versus benchmarking against the national average. On average, the difference is relatively unchanged when creating peer groups based on the Part B Drug Specialties categories versus using the national average. However, w hen creating peer groups based on the 15 Specialties and All CMS Specialties categories, these differences are reduced by 39 percent and 59 percent, respectively. These differences are consistent regardless of whether we additionally use practice size categories to construc t peer groups. Note that while creating peer groups based on Part B Drug Specialties categories has little impact overall, it does improve comparisons for the specialties for whom Part B drugs represent a significant part of their revenues, which we

addres s in the next section.

These findings suggest that constructing more comparable peer groups based on specialty mix substantially reduces the noise in measur ing relative cost performance

.

Benchmarks for Groups Dominated by a Single Physician Specialty

While the previous table shows that CPG creates more comparable peer groups overall and

reduces the noise in assess ing relative performance, it is useful to assess its impact for certain specialties, particularly those disproportionately represented among outliers when using a

national benchmark . To do this, we measure per capita cost differences between practices and their benchmarks for practices where a single specialty accounts for at least 60 percent of the physicians in a practice . These results are d isplayed in Table 3 . Consider , for example, the practices composed of at least 60 percent of dermatologists. As expected, their per capita costs are more than $4,000 lower than the national average. However, by benchmarking against moreMEMO TO: Michael Wroblewski, Craig Caplan, Sabrina Ahmed

FROM: Wil Lim, Gene Rich, Jeff Ballou, Bob Schmitz

DATE: 3/27/2013

PAGE: 5

comparable practice s, par ticularly based on 15 Specialties categories or All CMS Specialties categories, this difference is reduced by more than one third. Conversely , for practices with a large percentage of physicians specializing in hematology/oncology, their per capita c osts are nearly $ 5 ,000 higher than the national average. In this case, CPG reduces the per capita cost differences by more than 75 percent for all specialty categorization methods. Note that the Part B Drug Specialties categorization also performs well in this case because hematology/oncology physicians are included among specialties with a high proportion of Part B Drug revenue.

While the number of practices with a high percentage of any given specialty is low, the findings in this section are consistent across several specialties, suggesting that these are not

random aberrations. In addition, in Table 3b, we display results from repeating the exercise for practices with various percent ages of hematology/oncology physicians . We find that , relative to using a national benchmark, CPG reduces the average absolute difference between per capita costs for practices with hematology/oncology physicians and their peer group means, regardless of the hematology/oncology percentage

.

These findings highlight that in creating more comparable peer groups, CPG can reduce the average absolute deviation from the peer group benchmark , relative to the national benchmark, for practices that consist of physicians with particular specialties

.

Cost Tiering

Ultimately, bench marks are used to compute cost scores for assigning practices to low, average, or high cost tiers. Specifically, practices are assigned to tiers based on whether their cost

scores fall at least one standard deviation from the mean cost score and are also s tatistically significantly different from the mean cost score. We address this here, with two caveats. First, we consider only overall per capita costs and not a composite of overall and condition specific per capita costs. Because overall per capita costs are highly correlated with condition specific per capita costs, we expect this to have a minimal impact on these findings. Second, we construct

tier s based on only the one standard deviation from the mean criterion. That is, we do not

consider statistical significance. While doing so would lead to more parsimonious tier assignments, we expect this to have a minimal impact as previous analyses on the 870 TIN sample suggest that less than 10 percent of one standard deviation outliers are not statistically

di fferent from the mean

.

Table 4 shows the distribution of the practices in which a single physician specialty accounts for at least 60 percent of physicians into low, average, and high cost tiers for various

benchmarks. Although the samples are small, the y show CPG mov ing some dermatologist dominated practices that would be considered low by national benchmarking into the average tier. Conversely, CPG move d some hematology/oncology practices from high cost tiers into average tiers (the TIN tiered as high using the All CMS Specialties categorization is near the threshold separating high cost from average cost

, at 1.06 standard deviations above the mean )

.MEMO TO: Michael Wroblewski, Craig Caplan, Sabrina Ahmed

FROM: Wil Lim, Gene Rich, Jeff Ballou, Bob Schmitz

DATE: 3/27/2013

PAGE: 6

Again, this is consistent with CPG reducing the average absolute deviation from the peer group

benchmark , relative to using a national benchmark, for these specialties.

It is also useful to know overall the distribution of practices into tiers using CPG versus the

national benchmark. Tables 5 compare the assignment of practices to tiers using CPG versus the

national benchmark. In general, while CPG brings in TINs previously classified as high or low (due at least in part to the 60 plus percent of a specialty cases discussed previously) , the methodology tends to produce more outliers. However, this likely ref lects CPG improving benchmarking

. Because there are large outliers when using the national average as a benchmark,

this causes the standard deviation to be skewed upward , resulting in fewe r high and low categorizations. However, as CPG reduces deviations f rom benchmarks, particularly for extreme outliers, the standard deviation shrinks, resulting in more outliers. This explains why CPG using

the A ll CMS S pecialties categorization regardless of size produces 33 percent outliers and the

national benchmark onl y produces 14 percent. Also , keep in mind that additionally applying the statistical significance criterion will lead to slightly more parsimonious tiering.

Thus, c onsistent with finding that CPG improves benchmarking by shrinking the average absolute deviation from the peer group benchmark , CPG appears to improve cost tiering relative to using a national benchmark. P ractices which tend to be disproportionately represented as outliers when using the national benchmark are less likely to be classified as such under CPG.

Discussion

These refinements addressed two important concerns about CPG : whether most peer groups would be constructed from peer groups with identical specialty categorical

- percentage distributions and whether it would reduce the bias faced by certain specialties when using the national benchmark. In both cases, the response is “yes”. It should also be emphasized that CPG

creates these benchmarks by forming peer groups at the medical group practice level

, and not relying on the performance of individual physicians , addressing another concern by stakeholders

.

One concern about the Part B Drug Specialties and 15 Specialties categorization s is the somewhat arbitrary profiling of practices. While the purpose of the Part B Drug Specialties categorization in particular was clear in addressing the two concerns above, it is useful to know that these were addressed by using the national sample and all CMS physician specialties

, without the need to target specific ones

.

It should also be noted that the 870 TIN sample produces conservative estimates for the performance of CPG. U sing the national sampl e, versus a smaller sample, will likely result in more comparable peer groups from having more peers for any given specialty distribution. The results here suggest that these more comparable peer groups will result in further improvement to

b enchmarking.MEMO TO: Michael Wroblewski, Craig Caplan, Sabrina Ahmed

FROM: Wil Lim, Gene Rich, Jeff Ballou, Bob Schmitz

DATE: 3/27/2013

PAGE: 7

A useful next step for testing any benchmarking methodology is to conduct a split sample test, which we are conducting for CPG and national benchmarking. In this test, beneficiaries for each group are randomly divided into two samples, A and B. Using each sample, groups are tiered into low, average, and high. If a method were to accurately assess groups’ true performance , the tiers should vary little between the two samples.

Lastly, the potential for improved benchmarking to produce more outl iers is an important consideration to keep in mind when testing different ways of tiering. Appropriately adjusting tiering approaches in light of improved benchmarks will be important for assessing relative performance of practices and achieving other poli cy objectives, such as determining the suitable

percentage of outliers.

cc: Mai Hubbard; Mary Laschober; Jessica Heeringa