# Presentation: 333444

## Research Triangle Park,  North Carolina

- RTI International is a trade name of Research Triangle Institute.

## Protecting Quality and Confidentiality of Micro Data by MASSCSM :A Survey Sampling-Based Method 

- Data-Quality Data-Confidentiality Team
- Statistics Research Division
- RTI International
- For Presentation at the CDC Assessment Initiative TAG meeting,
- Atlanta, GA, April 16, 2004

- MASSC is a service mark of RTI International.

## Preamble

- Micro data
- PUF
- HIPAA
- CIPSEA

**Notes:**
Any database containing personal confidential information, requires protection from disclosure before being released as a public use file (PUF) for researchers at large.  This is, in fact, obligatory in view of the confidentiality pledge given to the respondent before conducting the interview. Also, in view of regulations such as HIPAA (for protected health information) and CIPSEA (for various kinds of government data), data producers are increasingly more concerned about protecting confidentiality.

## Inside vs. Outside Intrusion Scenarios

- “Disclosure by response knowledge” – an important inside intrusion scenario from respondent’s perspective.
**IVs:e.g., **for _BRFSS_ data, age group, race, gender, education, income, height, weight, freq of eating fruits, flu shots, etc.
**SVs:** e.g., for _BRFSS_ data, asthma condition, diabetes condition, # permanent teeth removed, drinking alcohol and driving car, reason for HIV test, method of birth control, etc.

**Notes:**
It is known that an individual’s confidential information collected in surveys such as BRFSS could be at risk of disclosure. The direct identifiers such as date of birth and telephone number do not pose any problems because the details can be suppressed, e.g., only broad age categories are generally of analytic interest. An important disclosure scenario known as ‘disclosure by response knowledge’ (see Bethlehem et al., 1990) is of great concern to the subjects in the database. Here, the subject finds his own record in the released but untreated database. (This is fairly easy for the subject as the combination of values of various demographic and outcome variables corresponding to the subject can invariably make him unique in the database.) Then the subject is concerned about protecting his confidential information because someone close to him such as a family member, friend or coworker who knows about his presence in the database could possibly find values of sensitive/secret variables (SVs) such as asthma condition, and reason for HIV test for his record by using a combination of values of several indirect identifying/intrusion variables (IVs) such as age, gender, race, and marital status. In the following, such an intruder will be termed as an inside intruder as opposed to an outside intruder who doesn’t know the presence of his target in the database. 

## Disclosure Risk from an Inside Intruder

- consider a hypothetical data with 10 observations. (IVs= age, gender; SV=positive Hypertension diagnosis)

**Notes:**
There are at least two ways the inside intruder might find values of SVs: first, the combination of IVs may make the target unique in the database, and second, the combination of IVs may make the target nonunique but all records in the nonunique group may have common values for at least one SV. The above scenario of inside intrusion provides a practical way of finding out which records could possibly be at risk and then find ways of introducing uncertainty about their presence and identity in the database. The fact that some records could be at risk is of great concern to the data producer because it is the respondent's trust that the data producer wants to maintain. Therefore, it is not possible for the producer to release the original untreated  data to users for analysis purposes unless some form of a licensing agreement is in place for the user to have access to the original database. This option of a signed licensing agreement may be viable for some users. However, there is a need of PUF for general users.

## Need for Disclosure Treatment and Control on Information Loss ( a conundrum)

- Some treatment of perturbation and suppression needed to protect from disclosure.
- Any disclosure treatment leads to information loss.
- How to balance the tension between disclosure risk      due to limited amount of perturbation and suppression, and information loss      due to introduction of bias and variance?
- Useful to have          measures for any process of disclosure treatment.

- Trade-off

- Disclosure Risk

- Information Loss

**Notes:**
For disclosure treatment of any micro-data such as the BRFSS, most of the available methods consist of either some form of perturbation of IV values (such as swapping, recoding or adding noise,) or suppression of IV or SV values (such as deleting a field or the whole record) or both. Now any form of disclosure treatment introduces some loss of information. Under the inside intrusion test mentioned earlier, which provides a reasonable way of finding records at risk, typically there could be a large number of records at risk depending on the intruder's knowledge of IVs. Treatment of records found to be at risk via perturbation or suppression may introduce high information loss in that unacceptably large bias could be present in the analysis of resulting data. Therefore, a method that simultaneously protects both confidentiality and analytical quality of data is desirable such that a suitable balance between disclosure risk and information loss can be achieved. 

## Deterministic vs. Stochastic Framework 

**Deterministic Selection for Treatment**
**Stochastic Selection for Treatment**
** ****Note: **Need a probabilistic/stochastic framework to measure and control disclosure risk and information loss.

**Notes:**
RTI's MASSC method offers such an option and a version of it was used for NSDUH data for the years 1999-2002.  Alternative methods in the class of nonsynthetic disclosure treatment methods use a deterministic selection of records for treatment, i.e., all the records identified to be at risk with respect to a set of IVs are treated.  However, MASSC uses a stochastic selection of records for treatment in which all records are subject to treatment but only a small random subset is actually treated; this leads to low information loss and provides protection against new IVs that an intruder might know. Thus, under a probabilistic framework, MASSC introduces sufficient uncertainty about the presence and identity of a record, and can provide measures of disclosure risk and information loss without any modeling assumptions.

## Process of MASSC(A nonsynthetic approach)

- Steps:
**I:****	**_**M**_**icro **_**A**_**gglomeration **
**II:****	****Optimal Random **_**S**_**ubstitution **
**III:****	****Optimal Random **_**S**_**ubsampling **
**IV:****	****Optimal **_**C**_**alibration **

**Notes:**
The MASSC acronym signifies the four steps of Micro Agglomeration, Substitution, Subsampling, and Calibration. Its foundation rests on the theory of survey sampling as it uses a subtle analogy between releasing an untreated database and conducting a census. It is designed to minimize disclosure cost ehile controlling loss of information. Micro Agglomeration creates risk strata and checks for records at risk. This step controls the number of records initially at risk by determining the level of details of the IVs to be released. Substitution uses optimal sampling rates for selecting records at random for perturbation subject to substitution bias constraints. This step introduces uncertainty about the identity of a target.  Sub-sampling uses optimal sampling rates to select records from the substituted database at random for non-suppression, subject to precision constraints. This step introduces uncertainty about the presence of a target. Calibration uses optimal weight calibration to adjust subsampling weights subject to preserving key estimates from the original database. This step reduces bias due to substitution and variance due to subsampling.  The MASSC treatment adds a second phase to the NSDUH data. By making the selection of records independent from PSU to PSU for substitution and subsampling, the commonly used single phase survey data analysis methods such as the ones in the SUDAAN software can be used under general conditions to analyze the MASSC treated data. For more details on MASSC, see Singh, Yu, and Dunteman (2003).

## MASSC Illustration (Micro Agglomeration)

**Notes:**
In applying MASSC, we need to introduce distortion via substitution and subsampling, and then fix it via calibration.  First, the risk strata are created in the Micro Agglomeration step to calculate the proportion of records that are uniques. If there are too many uniques, then some further recoding of IVs into broader categories is performed to reduce the number of uniques. 

## MASSC Illustration (Substitution) 

- MASSC Illustration (Substitution)

**Notes:**
Next for each record, a substitution partner is assigned such that a suitable distance (Rao’s Quadratic Entropy, in particular) based on IVs between the recipient and the donor record is minimized. Then risk strata are subdivided into substrata such that each substratum has relatively low contribution to bias due to substitution for a given set of study variables defined by drug use for various demographic domains. Next using nonlinear optimization techniques, selection probabilities for substitution are obtained such that a disclosure cost function is minimized subject to a set of constraints on bias relative to the original estimate. When a record is selected for substitution, all variables related to the IVs are also substituted from the donor in order to maintain internal consistency.

## MASSC Illustration (Subsampling)

**Notes:**
Similarly, for the subsampling step of MASSC, risk strata are subdivided into substrata such that each stratum has relatively low contribution to variance for a given set of study variables, and then using nonlinear optimization, selection probabilities for subsampling are determined such that a disclosure cost function is minimized subject to a set of constraints on sampling variance relative to the squared original estimate.

## Confidentiality Diagnostics

**Notes:**
Based on unique and nonunique occurrence rates, substitution and subsampling rates obtained from the above steps of MASSC, it is possible to compute measures of risk as part of confidentiality diagnostics and check for their adequacy. For example, for a record appearing unique,      denotes  the chance that it survived the treatment of substitution and subsampling and is at the risk of disclosure from an inside intruder.  If it is around 20%, it may be deemed reasonably safe in analogy with tabular data where a minimum cell size of 3 to 5 is often considered safe. It may be noted that this measure is quite conservative as it is based on the assumption that the inside intruder knows quite a few IVs. In addition to checking the adequacy of these quantitative measures, it is ensured that no variable is included PUF that provides more detailed information than the IVs considered in the treatment. Also the PUF is checked for records with possible rare values of SVs, if any, which are then treated by top/bottom coding. 

## MASSC Illustration (Calibration)

**Notes:**
Finally, the last step of calibration in MASSC is applied to the substituted and subsampled data by adjusting the subsampling weights so that demographic subpopulation counts for domains such as those defined by age, race, gender, and ethnicity are preserved in the treated data. 

## Quality Diagnostics

- Average absolute bias
- Average decrease in precision

**Notes:**
Before PUF is released, several diagnostics for analytical quality are performed in order to determine the impact of disclosure treatment on bias and precision of estimates compared to the original estimates. In particular, for a number of study variables, point estimates and standard errors are computed before and after the treatment. Ultimately what really matters is the extent to which inferential results of interest to the user continue to hold in the treated database. The bias and variance constraints in MASSC in terms of point estimates for various domains provide basic quality protection for key study variables, but it is not possible to guarantee that inference for certain parameters won’t change from the original database. Since MASSC treatment depends on the choice of design parameters related to disclosure risk and information loss, it may be possible to repeat MASSC treatment on the same dataset under a different set of parameters so that the treated dataset has more desirable analytical properties 

## Summary

- (IV = Identifying/Intrusion Variable)

- MASSC Value vs. Alternatives

| Confidentiality/Quality | Alternative Methods
(Deterministic Framework) | MASSC
(Stochastic Framework) |
| --- | --- | --- |
| Data Confidentiality
   Selected IV’s
   Other IV’s
   Measure of    Disclosure Risk | 
High protection(low risk)
No protection(maximum risk)
Not generally available | 
High protection (low risk)
Moderate to high protection(low to moderate risk)
Available |
| Data Quality
   Bias
   Variance
   Information on IV’s
   Standard Analysis Tools
   Measure of Info Loss | 
High
Zero (typically)
Coarse Categories
Not Applicable
Not Available | 
Low
Low to Moderate
Fine Categories
Applicable
Available |

## Concluding Remarks 

- Currently available web-based data dissemination system such as MICA for example, tables for diagnosis by demographics are not available for higher dimensional cross-classifications. However, it can be made possible with MASSC.
- Successful experience in using MASSC for data from SAMHSA, NCHS, and DoD. CMS is also interested in using MASSC.

##  Additional Information

- Dr. Michael Samuhel, Director
-  Research and DevelopmentRTI International919-541-5803samuhel@rti.org