Methods of Selecting Variables in Data Disclosure 

Charles C Lin1 and Tom Garin1

1Data Governance and Analytics, Department of Veterans Affairs 

810 Vermont Avenue NW, Washington, DC 20420

Key words: Re-identification, Variable selection, Data disclosure, Data sparseness 

Abstract 

Data released for public use needs to be protected for personal identified information such as name and Social 

Security Number and any unusual pattern in data which could have a high risk of re-identification. All privacy 

protected information should be stripped from released variables in data according to a set of disclosure guidelines. 

Excessive restrictions of released data elements, however, could render the data useless. The task of proper data 

disclosure with minimum risk of re-identification can be overwhelming when the number of variables becomes 

large. The brute force approach of examining all becomes infeasible as the number of possible combinations of 

variables increases exponentially. To prepare a data for public use, this study proposes systematic methods 

examining each variable with respect to both risks and benefits. These methods compare a subset of variables to 

another and traverse back and forth from subsets to subsets in a step-by-step manner until a stopping criterion is met. 

The final subsets of variables will provide reasonable choices of variables for public release. The methodology with 

its implementation in SAS program will be discussed in details and demonstrated with examples. 

Introduction 

Data disclosure to public has been a general practice in government and public for many years upon requests of 

Freedom of Information Act (FOIA) signed into law in 1966 and recently issued Open Data guidance to all Federal 

agencies starting May 2013, followed Executive Order 13642 of “Making Open and Machine Readable the New 

Default for Government Information” and OMB Memorandum 13-13 of “Open Data Policy-Managing Information 

as an Asset”. However, releasing data to public has not been an easy task as personally identifiable information (PII) 

is protected by privacy laws, like the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule in

1996 and Executive Order 13556 for Controlled Unclassified Information (CUI) in 2010. Privacy protection is also 

facing challenge now as vast amount of information on individuals already are abundant in public domains like 

vehicle and voter registration records. Sweeney showed more than 87% of US citizens can be identified by the three 

variables gender, date of birth, and zip code available in public databases with the 1990 Census data [5]. Data can be 

aggregated to remove PII but will make data no longer granular enough to be fully useful. Balancing privacy against 

data release requires federal agencies to develop guidelines for reliably de-identifying datasets so they can be 

released without violating individual’s privacy. 

Risk of re-identification, the process used to identify individuals connected with information, typically arises when a 

small count of unusual pattern in values of released variables can be used to identify individuals. Restricting release 

of variables will certainly reduce occurrences of unusual patterns in data in public and hence lower re-identifying 

risk. Descriptive variables such as age and gender are common in disclosed data. But many other variables are 

deemed minimum necessary for release purpose. Pre-screening variables could drastically reduce the number of 

variables needed for release. Unique patterns in data post high risks of re-identification if the uniqueness is not 

caused by lack of sufficient data. Restricting variables for release will subsequently lessen number of unique 

patterns in data. 

Identifying uniqueness with relevant variables can be a good practice exploiting sources of high risk of reidentification in data [2][3], but the search should go beyond uniqueness and extend to unusual patterns of small 

counts which possess high risks as well [1]. Various measures of risk of re-identification associated with a set of 

variables have been proposed but involved complex computations [3], simpler risk metrics given by Emam [4] are 

found more applicable. Thus, sets of variables with low values in the risk metrics will then be desired for release 

purpose. 

1 
The task of searching sets of variables with low risk measure can be overwhelming if the number of variables is 

large. Thus, an automatic process of selecting subsets of variables within a limit in risk is warranted. The proposed 

three methods select subsets of variables according to criteria derived from the risk metrics of Emam. These 

methods offer good choices of a set of variables for release without going through all possible subsets of variables 

and the subsets generated do not require to be of minimum risk. With an implementation of SAS programs, these

variables selection methods are illustrated with examples and further discussed in details. 

Data for Release 

To prepare data for release all variables need to be screened according to disclosure guidelines for privacy 

protection. Masking or generalization maybe required to hide individual identity such as using age groups or first 

three digits of zip codes. The HIPPA Safe Harbor known rules includes limiting use of cross-year longitudinal data 

and across datasets numbers of records, suppressing geographic information below state level, and aggregating ages 

over 89 into a single age category. [3] [6] 

Released Data as Sample of Population 

The released data is considered to be a sample of a known population where an attacker can link an individual in the 

population to the released data by their common variables. Therefore, the focus is to restrict the release of variables 

in the sample which could reveal unusual values, as combining more unusual values of many variables the reidentifying risk is greatly amplified. 

The key variables, such as age and gender in the sample gives a limit number of distinct values, can be used to reidentify a target individual. A set of key variables provides a limit number of patterns of values which defines the 

universe for re-identification. Each pattern of values can be visualized as a cell in the cross-tabulation of the key 

variables. The cell size is the number of repetitions of a pattern in the released data/sample and all cell sizes will 

sum up to the sample size. Thus, cells of smaller cell sizes, with unusual pattern of values, pose greater risks of reidentification. The goal of de-identification is to restrict the release of key variables so that to produce a relatively 

small number of cells with small cell sizes.

Sample Unique and Population Unique 

Sample unique and population unique are cells of cell size one in the sample and population, respectively. Sample 

unique implies a probable population unique, which poses as the highest risk of re-identification. [2][3] In general, 

small cell sizes like less than 3 in the sample are considered at a high risk of re-identification [1]. Assuming sample 

unique is population unique makes computation feasible but leads to conservative estimate. 

 

Methodology of Variable Selection 

The development of methodology of variable selection would first rely on assessing the risk of re-identification of 

released variables in data. The risk is usually measured with limitations as the population for the released data may 

not be known. By adapting simple risk metrics and with necessary assumptions, we proposed three variable 

selection methods; the Forward, Backward/Elimination, and Stepwise methods. The Stepwise method is a variation 

of the Forward method with extra protection on risk tolerance. All methods will provide reasonable choices of 

released variables within a tolerated limit on risk. 

Risk Measures of Re-identification 

The Risk Proportion (RP) is the proportion of records with sample cell size less than a predetermined number. The 

Cell Ratio (CR) is the ratio of the number of non-empty sample cells over the total number of records. Assuming a 

target individual is known included in the released data, RP and CR will be the same as the risk metrics by Emam, 

assessing the proportion of records that have a re-identification probability larger than a threshold and the average 

probability of re-identification, respectively [4]. In general, RP over-estimates (to be conservative) the proportion of 

population at high risk of re-identification and CR over-estimates the average proportion of population re-identified, 

the former over-estimation tends to be larger the latter. 

2 



The summary of Backward method shows how the variables are dropped and the values of RP are decreased to a 

value just above a pre-set value of 5% over the steps.

 Summary of Backward Selection method 

 Stopped when pp < 5% 

 Obs step varlist 

 1 B1 sex ag8r source pow ms aor homstate means nsurg nbs cp lsr ethnic race1 psrcd 

 2 B2 sex ag8r source pow ms aor homstate means nsurg cp lsr ethnic race1 psrcd 

 3 B3 sex ag8r source pow ms aor homstate means nsurg cp lsr ethnic race1 

 4 B4 sex ag8r source pow ms aor homstate nsurg cp lsr ethnic race1 

 5 B5 sex ag8r source pow ms homstate nsurg cp lsr ethnic race1 

 6 B6 sex ag8r source pow ms homstate cp lsr ethnic race1 

 7 B7 sex ag8r source pow ms cp lsr ethnic race1 

 8 B8 sex ag8r source pow ms cp ethnic race1 

 9 B9 sex ag8r source pow ms cp ethnic 

 10 B10 sex ag8r pow ms cp ethnic 

 11 B11 sex ag8r pow ms cp 

 Obs alpha rp cr ratio 

 1 0.991 0.999 0.990 1.009 

 2 0.999 0.998 0.988 1.010 

 3 0.999 0.997 0.986 1.011 

 4 0.996 0.994 0.980 1.015 

 5 0.995 0.989 0.969 1.020 

 6 0.993 0.984 0.957 1.027 

 7 1.004 0.708 0.692 1.023 

 8 1.081 0.434 0.459 0.946 

 9 1.086 0.262 0.301 0.871 

 10 1.069 0.153 0.188 0.815 

 11 1.287 0.058 0.092 0.633 

The Stepwise method is similar to the Forward method up to Step 10. Then, it detects the variable cp needs to be 

removed (RP=0.404 > 0.35) in the Backward Step 1, and the Backward process was stopped as the variable source

cannot be removed (RP=0.275 < 0.35) in the Backward Step 2. The Stepwise method would continue with the 

Forward process but stopped in Step 11 because the variable cp selected to be entered has just been removed in the 

last step, even though RP=0.523 for the variables cp is still lower than the stopping criterion of 0.55 . 

 Forward step 10: 

 Varlist: sex ag8r pow aor nsurg psrcd psx means cp nbs source 

 ms (smallest alpha=1.052) to enter next step 

 Obs newvar alpha rp cr ratio 

 1 ms 1.052 0.523 0.533 0.983 

 2 homstate 1.125 0.870 0.828 1.051 

 3 lsr 1.052 0.565 0.575 0.983 

 4 ethnic 1.057 0.491 0.498 0.987 

 5 race1 1.063 0.525 0.529 0.993 

 Backward step 1: 

 Varlist: sex ag8r pow aor nsurg psrcd psx means cp nbs source ms 

 cp (largest alpha=1.048) to be removed next step 

 Obs delvar alpha rp cr ratio 

 1 pow 1.009 0.430 0.442 0.974 

 2 aor 1.014 0.474 0.490 0.969 

 3 nsurg 1.033 0.445 0.468 0.951 

 4 psrcd 1.000 0.523 0.533 0.983 

 5 psx 0.999 0.523 0.532 0.983 

 6 means 1.017 0.462 0.478 0.966 

 7 cp 1.048 0.404 0.430 0.938 

 8 nbs 1.040 0.418 0.442 0.945 

 9 source 1.044 0.379 0.403 0.941 

 10 ms 1.052 0.346 0.370 0.934 

 Backward step 2: 

 Varlist: sex ag8r pow aor nsurg psrcd psx means nbs source ms 

 source (largest alpha=1.044) to be removed next step

 Stopped at step 2, 35% criterion met 

6 

estimated odds ratios of high risk cells (with small cell size) will also prepare data better suitable for the variables 

selection methods. 

Conclusion 

The paper is to address the problem in disclosing large data for public use and avoiding risk of re-identification. To 

solve the problem, we gauged the usefulness of released data and its risks of being re-identified, designed the 

criterion of selecting variables for disclosure, and proposed systematic methods comparing subsets of variables in a 

step-by-step manner considering both risks and benefits. The methods will stop when the step has reached within a 

pre-set limit of tolerance level and then the subsets of variables in the final steps would be reasonable choices of 

variables for release. 

Variables selection is essential for proper disclosure of data with a large number of variables. The success of 

methodology depends on the design of selection criterion considering both risks and benefits of released data. 

Acknowledgements 

Hwai-Tai Lam1 provided test data and feedback of using the programs of the methods on real data. George Fitzelle

1

described government regulations on privacy protection. Lisa Mavrogianis1 provided information on the Open Data 

policy. 

References 

[1] Susan Hickey, Managing Re-identification Risk for Patient-Level Datasets, June 12, 2013 

[2] Susan Hickey, Protecting VHA Patient Privacy When Releasing Information, July 25, 2013 

[3] XH Andrew Zhou et al, Risk of Linking HIPAA De-identified Rheumatoid Arthritis Research Dataset with CMS 

Data, Dec 21, 2012 

[4] Khaled El Emam, Methods for the De-identification of Electronic Health Records for Genomic Research, 

Appendix: Measuring the Probability of Re-identification, Genome Medicine, 2011 

[5] Sweeney, L., Simple Demographics Often Identify People Uniquely. Carnegie Mellon University, Data Privacy 

Working Paper 3, Pittsburgh 2000. 

[6] Informative Graphics Corp., Redaction in Health Information Management, an IGC White Paper, Scottsdale AZ, 

2013

8