# Presentation: 674576

## Measuring Perceptions of Applying Alternative Disclosure Limitation MethodsJacob BournazianStatistics and Methods GroupEnergy Information Administration Presentation to the:American Statistical Association Committee on Energy StatisticsApril 6, 2006Washington, DC

**Notes:**
Today’s presentation is on a new research proposal for EIA.   The proposal is to study the perceptions of our survey respondents and data users on a specific topic relating to data confidentiality.   There have been prior studies on respondents perceptions on data confidentiality and privacy issues.  However, there has not been any study on the perceptions of applying alternative disclosure limitation methods.

## Need to Measure Perceptions of Different Groups

- How will data users perceive the utility of the data by applying different disclosure limitation methodologies?
- Will they find tables fully populated with cell values containing adjusted data more or less useful to what is currently released?
- Will applying alternative disclosure limitation methods  affect the quality or accuracy of survey responses?
- Will respondents be less diligent in accurately completing a survey if they know their responses will be adjusted or swapped with another respondent’s data by the agency?

**Notes:**
There are two parts or components to this research proposal.  Two surveys are probably needed because we are measuring the perceptions of two distinct groups. 

## Alternative Disclosure Limitation Methods

- Applied to data prior to tabulation
- Data Swapping
- Noise Addition
- Applied to data after tabulation
- Controlled Tabular Adjustment

**Notes:**
let me share some background about these new methodologies so that we have some understanding of the context of this research proposal.
For discussion purposes, I am grouping the methodologies into 2 categories
They were developed as a result of the growing demand from the user community, and the first two, data swapping and noise addition have been implemented by other federal statistical agencies.
These alternative methods are explained in greater detail in the paper, but to briefly summarize what is in the paper I am going to use a simple example 

## Example: Original Table (Actual Values)  One Cell is Sensitive 

**	****	****	****	****	****	****Total**
**	****	****	****50****	****110****	****500****	****660**
**	****	****	****90****	****30****	****70****	****190**
**	****	****	****600****	****80****	****20****	****700**
**Total****	****740****	****220****	****590****	****1550**

**Notes:**
Typical table containing magnitude data

## Table Protected by Cell Suppression

**	****	****	****	****	****	****Total**
**	****	****	****50****	****110****	****500****	****660**
**	****	****	****90****	****C****	****C****	****190**
**	****	****	****600****	****C****	****W****	****700**
**Total****	****740****	****220****	****590****	****1550**

**Notes:**
When applying cell suppression, there is a total loss of information from the cell that gets suppressed.   Every table has a point of no return will there is so much suppression that the utility of the table is destroyed.  Likewise, there is that grey zone where a table contains some suppression and only part of the data utility of a table is lost as shown above.

## Table Protected By Controlled Tabular Adjustment 

**	****	****	****	****	****	****	****	****Total**
**	****	****	****50****	****110****	****	****500****	****	****660**
**	****	****	****90 ****	****33 (+3)****	****67 (-3)****	****190**
**	****	****	****600****	****75 (-5)****	****25  (+5)****	****700**
**Total****	****740****	****218**** ****(-2)****	****592 (+2)****	****1550**

## Sensitive Cell Protected By Noise Addition

- Actual company values	Noise-added company values
- Co. 1	14,000		14,000 * 1.11 = 15,540
- Co. 2	 4,000		 	4,000 * 0.89   =   3,560
- Co. 3	 2,000		 	2,000 * 1.12   =   2,240
- Total      **20,000****	**				      **21,340**

**Notes:**
Noise is added randomly so that the adjusted aggregate estimates are unbiased.
Each company receives the same noise factor so wherever in the table that company’s data appears, their reported value is adjusted in a consistent manner.
Each facility or store for a company may not received the same exact noise factor, as long as the average of the facilities or stores equals the assigned noise factor.

## Non-Sensitive Cell With Noise Addition Applied

- Actual company values	Noise-added company values
- Co. 1	10,000		10,000 * 1.11 = 11,100
- Co. 2	 8,000		 	 8,000 * 0.89   =  7,120
- Co. 3	 5,000		 	 5,000 * 1.12   =  5,600
- Total      **23,000**					      **23,820**

## Sensitive Cell Protected By Data Swapping

- Actual company values	Swapping company values
- Co. 1	14,000		Co. 1	14,000
- Co. 2	 4,000		 	Co. 7	  5,000
- Co. 3	 2,000		 	Co. 3	  2,000
- Total      **20,000****	**			**21,000**
- Use a data key structure to find a swapping candidate.
- Sales type * product type * seller type (similar size)
- Swap data values based on a swapping variable such as geographic region.  Swap rate kept low at 3 – 5% of reported values.

**Notes:**
Swapping is used to protect household and health data, Census Bureau used it for Census 2000

## New Disclosure Methodologies Provide Alternatives to Cell Suppression

- Cell suppression still the most common method.
- New methodologies modify the data in some manner.
- Applying Noise addition and Controlled Tabular Adjustment may not cause a statistically significant change from the actual estimates.
- Benefits – All table cells show values, more data available.
- Costs – Adjusted data may distort/change relationships between variables and affect the precision of the estimates generated from the actual reported values.

**Notes:**
There are more disclosure limitation methodologies available for protecting data than compared with what was available 15 years ago.
These methodologies were developed in response to a growing demand from the data user community.  Some of these methodologies have been applied by statistical agencies, however, cell suppression is still the most common method used by federal agencies and is clearly the main method used here at EIA.
these methodologies can be categorized along two branches:
Modify prior to tabulating
Modify after tabulating

## Measuring Business Perceptions

- Survey of Business Perceptions of Confidentiality (2000)
- 	Sample of 5,000 businesses selected from a frame of 11.3 million businesses, 4 strata based on # of employees
- 	Mail survey, low response rate (10.6%)
- 	Split on whether the government could protect confidential data.
- 	Greater the trust and confidence in the agency, more willingness to provide data.
- Other studies – data sharing activity
- 	Overriding concern on whether this will reduce reporting burden

**Notes:**
The closest study I found which parallels what this research proposal is a study performed by the Census Bureau 6 years ago.

## Measuring Data User Perceptions

- Measuring household perceptions regarding data confidentiality
- 	20 years of research by Census Bureau
- 	Mode: focus groups, cognitive research (used mainly for designing questionnaire), mail surveys
- Results: Similar split over agency confidentiality pledge to protect data and concerns over data sharing

## Additional Considerations For Measuring Perceptions

- Different disclosure limitation methods impact the data differently.
- EIA has a diverse data user community.
- Perceptions may differ (gov’t, assoc., analysts, media etc.)
- Level of statistical knowledge among data users and respondents is limited.
- Asking people about their perceptions on a technical issue. Misconceptions and negative attitudes exist.

## Two Surveys: Discussion For Designing A Study

- Who should be the target respondent?
- 	Authority figure with company or contact person?
- Should all survey respondents be included or a sample?
- How should data users be selected?
- What mode(s) of data collection?
- 	mail, internet, telephone, etc.
- Should examples be used in explaining different methods? (generic versus customized)
- Other Issues that need to be considered?