CONFIDENTIAL

Designing a Probability Sample to Produce a Large Number of Key Estimates

Phillip S. Kott

Darryl V. Creel

www.rti.org

RTI International is a trade name of Research Triangle Institute. RTI and the RTI logo are U.S. registered trademarks of Research Triangle Institute.

CONFIDENTIAL

Comparing Testing Volume by Lab Type from 2018 Medicare Claims with Testing Volume Reported to CMS 2016 (Voluntarily Reported)
| Type of Lab          | Testing Volume (in millions)         |                                               |                                                            |                                                                              |
|----------------------|--------------------------------------|-----------------------------------------------|------------------------------------------------------------|------------------------------------------------------------------------------|
|                      | Medicare Testing Volume - 2018 (RTI) | Percent of Total Medicare Volume - 2018 (RTI) | Private Payer Testing Volume Reported to CMS -  2016 (CMS) | Percent of Total Private Payer Testing Volume Reported to CMS – 2016 (CMS)   |
| Independent lab      | 210                                  | 49                                            | 224                                                        | 90                                                                           |
| Hospital lab         | 122                                  | 28                                            | 2                                                          | 1                                                                            |
| Physician-office lab | 94                                   | 22                                            | 19                                                         | 8                                                                            |
| Other                | 4                                    | 1                                             | 3                                                          | 1                                                                            |
| Total                | 430                                  | 100                                           | 248                                                        | 100                                                                          |

2

CONFIDENTIAL

Maximal Probability Proportional-to-Size (MPPS) is a Version of Maximal Brewer Selection

International Conference on Establishment Surveys, II June 2000, Buffalo, New York 

THE THEORY AND PRACTICE OF MAXIMAL BREWER SELECTION WITH POISSON PRN SAMPLING 

Phillip S. Kott And Jeffrey T. Bailey, National Agricultural Statistics Service 

Phillip S. Kott, NASS, Room 305, 3251 Old Lee Hwy, Fairfax, VA 22030, USA pkott@nass.usda.gov 

K.R.W. Brewer suggests that when estimating the total of a single item for which there is control (auxiliary) data, one employ a ratio or regression estimator and draw the sample using probabilities proportional to the control values raised to a power between 1/2 and 1. Brewer's sample selection scheme can be expanded to multiple targets by drawing overlapping Poisson samples for a number of items simultaneously using permanent random numbers (PRN's). We can call the result an example of "Maximal Brewer Selection" (MBS). This paper develops the theory behind MBS and the calibration estimator rendering it practical. It goes on to describe how this estimation strategy is being used at the National Agricultural Statistics Service. 

Key Words: Model; Model-assisted; Randomization; Calibration; Delete-a-group jackknife 

3

icespap2c8.pdf (usda.gov)

CONFIDENTIAL

Example MPPS Probabilities of Selection (POS), Target Sample Size is Two 

4
| Lab   | HCPCS 1        |                    | HCPCS 2        |                    | HCPCS 3        |                    | Maximal PPS POS |
|-------|----------------|--------------------|----------------|--------------------|----------------|--------------------|-----------------|
|       | Testing Volume | HCPCS Specific POS | Testing Volume | HCPCS Specific POS | Testing Volume | HCPCS Specific POS |                 |
| 1     | 838            | 0.3136             | 24             | 0.1004             | 5,983          | 0.1575             | 0.3136          |
| 2     | 287            | 0.1074             | 58             | 0.2427             | 5,793          | 0.1525             | 0.2427          |
| 3     | 2,226          | 0.8329             | 79             | 0.3305             | 0              | 0.0000             | 0.8329          |
| 4     | 36             | 0.0135             | 84             | 0.3515             | 45,920         | 1.2090             | 1.0000          |
| 5     | 48             | 0.0180             | 145            | 0.6067             | 9,547          | 0.2513             | 0.6067          |
| 6     | 124            | 0.0464             | 17             | 0.0711             | 2,944          | 0.0775             | 0.0775          |
| 7     | 1,024          | 0.3832             | 9              | 0.0377             | 1,212          | 0.0319             | 0.3832          |
| 8     | 762            | 0.2851             | 62             | 0.2594             | 4,567          | 0.1202             | 0.2851          |
| Total | 5,345          | 2.0000             | 478            | 2.0000             | 75,966         | 2.0000             | 3.9506          |

4

CONFIDENTIAL

Exclusion Criteria from the Original Data File
| Exclusion                                                                                                  | Rationale                                                                                                                                                                                                                                                                                                                                                                 |
|------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Claims with non-CLFS HCPCS codes                                                                           | Payment rates for codes not paid through the CLFS are not set using private payer rate data.                                                                                                                                                                                                                                                                              |
| Claims with allowed charges or amount paid of $0                                                           | Eliminates non-paid claims.                                                                                                                                                                                                                                                                                                                                               |
| Outpatient claims for CLFS HCPCS codes not paid through the CLFS                                           | Payment rates for claims not paid through the CLFS are not set using private payer rate data. For example, we excluded claims with type of bill 14X without a revenue center status indicator code of “A”; such claims are paid through other payment systems, such as the Outpatient Prospective Payment System, and not the CLFS.Please see Appendix C for details.     |
| Claims with Type of Bill 12X and 13X                                                                       | We excluded claims for hospital inpatient and outpatient services. Please see the discussion above for our rationale.                                                                                                                                                                                                                                                     |
| HCPCS codes for specimen collection, travel allowances, and unlisted lab codes                             | These codes are excluded because they are not lab tests or are not priced based on private payer rates. Please see Appendix B for a list of these codes.                                                                                                                                                                                                                  |
| Claims billed by placeholder NPIs 9999999992 from the carrier file and 9999999996 from the outpatient file | We cannot identify the labs corresponding to these placeholder NPIs and therefore would not be able to sample them.                                                                                                                                                                                                                                                       |
| Exclude the professional component of claims for CLFS codes                                                | Claims for some CLFS codes include both a professional component and a lab component. Claims for the professional component is billed through the Medicare physician fee schedule and not the CLFS. As a result, we exclude the claims for the professional component and include the claims for the lab component.                                                       |

5

CONFIDENTIAL

Monte Carlo Simulation: Factors with Levels
| Lab type              |
|-----------------------|
| Hospital labs         |
| Independent labs      |
| Physician Office labs |

6
| Sampling Methodology |
|----------------------|
| MPPS                 |
| SSPPS                |
| Target Sample Size |
|--------------------|
| 10                 |
| 20                 |
| 30                 |

3x3x2 = 18 Treatments

For each treatment, we did 1,000 replicates

CONFIDENTIAL

Example Table Comparing Relative Bias and Sample Size for SPPS and MPPS
| HCPCS Code | Physician Office Labs                  |        |             |      |               |        |             |       |               |        |             |       |
|------------|----------------------------------------|--------|-------------|------|---------------|--------|-------------|-------|---------------|--------|-------------|-------|
|            | Target Sample Size for Each HCPCS Code |        |             |      |               |        |             |       |               |        |             |       |
|            | 10                                     |        |             |      | 20            |        |             |       | 30            |        |             |       |
|            | Relative Bias                          |        | Sample Size |      | Relative Bias |        | Sample Size |       | Relative Bias |        | Sample Size |       |
|            | SPPS                                   | MPPS   | SPPS        | MPPS | SPPS          | MPPS   | SPPS        | MPPS  | SPPS          | MPPS   | SPPS        | MPPS  |
| 80053      | 0.952                                  | 0.000  | 469         | 957  | 0.471         | 0.000  | 743         | 1,303 | 0.300         | 0.002  | 951         | 1,523 |
| 80061      | -0.071                                 | -0.004 | 469         | 947  | -0.065        | -0.001 | 745         | 1,291 | -0.088        | -0.001 | 953         | 1,508 |
| 82378      | -0.126                                 | 0.001  | 122         | 206  | -0.067        | -0.001 | 165         | 254   | -0.030        | -0.002 | 196         | 278   |
| 83036      | -0.131                                 | -0.002 | 460         | 934  | 0.112         | -0.002 | 738         | 1,289 | 0.067         | 0.003  | 949         | 1,520 |
| 84445      | 0.130                                  | 0.000  | 33          | 44   | 0.026         | 0.000  | 42          | 47    | 0.007         | 0.000  | 47          | 51    |
| 86003      | 0.563                                  | -0.018 | 69          | 107  | 0.334         | -0.003 | 85          | 124   | -0.214        | 0.001  | 99          | 138   |
| 86148      | 0.094                                  | 0.000  | 16          | 18   | 0.000         | 0.000  | 19          | 19    | 0.000         | 0.000  | 19          | 19    |
| 87150      | 0.000                                  | 0.000  | 10          | 11   | 0.000         | 0.000  | 12          | 12    | 0.000         | 0.000  | 12          | 12    |
| 87902      | -0.008                                 | 0.000  | 28          | 30   | -0.011        | 0.000  | 31          | 32    | -0.006        | 0.000  | 32          | 33    |
| 88262      | 0.000                                  | 0.000  | 10          | 10   | 0.000         | 0.000  | 10          | 10    | 0.000         | 0.000  | 10          | 10    |

7

CONFIDENTIAL

Recommendation: Maximal Probability Proportional-to-Size Sampling

Relatively easy to implement

Approximately unbiased estimates, even at the smallest target sample size of 10

For selected labs, can use data for all HCPCS code for which they do testing

Captures information about rare HCPCS tests

8

CONFIDENTIAL

Further Research

MPPS sample are larger than they should be.

A way to include labs sampled in the SPPS probability part to contribute to other HCPCS codes.

MPPS using systematic PPS sampling, instead of Poisson.

Nonresponse

Variance estimation

9

CONFIDENTIAL

Thank you

Phillip S. Kott, pkott@rti.org

Darryl V. Creel, dcreel@rti.org

10

# image1.png

# image2.png

# image6.png