Federal Committee on 

Statistical Methodology 

Research Conference 

Final Program and 

Abstract Booklet

November 2–4, 2009

Walter E. Washington Convention Center

801 Mount Vernon Place, NW

Washington, DC 

Sponsored by:

Agency for Healthcare Research and Quality

Bureau of Economic Analysis

Bureau of Justice Statistics

Bureau of Labor Statistics

Economic Research Service

Energy Information Administration

National Agricultural Statistics Service

National Center for Education Statistics

National Center for Health Statistics

Office of Research, Evaluation, and Statistics, Social Security Administration

Statistics of Income Division, Internal Revenue Service

U.S. Census Bureau

U.S. Environmental Protection Agency

Hosted by:

Council of Professional Associations on Federal Statistics
FCSM Research Conference Planning Committee

Dawn Haines, Co-Chair, U.S. Census Bureau

Kevin Cecco, Co-Chair, Statistics of Income 

 Division, Internal Revenue Service

Dale Atkinson, National Agricultural Statistics Service

Jeff Beaubier, U.S. Environmental Protection Agency

Jock Black, National Science Foundation

Jonaki Bose, Substance Abuse and Mental Health 

 Services Administration

Benjamin Bridgman, Bureau of Economic Analysis

Chris Chapman, National Center for 

 Education Statistics

John Finamore, U.S. Census Bureau

Thesia Garner, Bureau of Labor Statistics

Paul Guerino, Bureau of Justice Statistics

Charlie Hallahan, Economic Research Service, 

 U.S. Department of Agriculture

Kim Henry, Statistics of Income Division, 

 Internal Revenue Service

Anna Holaus, U.S. Census Bureau

David Kashihara, Agency for Healthcare and 

 Research Quality

Ruey-Pyng Lu, Energy Information Administration

Pamela McGovern, U.S. Census Bureau

Bill Mockovak, Bureau of Labor Statistics

Jennifer Parker, National Center for Health Statistics

Michael Planty, National Center for Education 

 Statistics

Wanda Rivers, U.S. Census Bureau

Elaine Russell, U.S. Census Bureau

Jim Sears, Social Security Administration 

Joy Sharp, Department of Transportation

Edward Spar, Council of Professional Associations on 

 Federal Statistics

Brian Harris-Kojetin, Chair, Office of Management 

 and Budget

Nancy Bates (Secretary), U.S. Census Bureau

Lynda Carlson, National Science Foundation

Steven B. Cohen, Agency for Healthcare 

 Research and Quality

Steven H. Cohen, National Science Foundation

Lawrence H. Cox, National Center for 

 Health Statistics

John Eltinge, Bureau of Labor Statistics

Patrick Flanagan, U.S. Census Bureau

Dennis Fixler, Bureau of Economic Analysis

Barry Graubard, National Cancer Institute

Deborah Griffin, U.S. Census Bureau

William Iwig, National Agricultural Statistics Service

Arthur Kennickell, Federal Reserve Board

Jennifer Madans, National Center for Health Statistics

Rochelle Martinez, Office of Management and Budget

Renee Miller, Energy Information Administration

J. Neil Russell, National Center for Education Statistics

William Sabol, Bureau of Justice Statistics

Susan Schechter, U.S. Census Bureau 

Nathaniel Schenker, National Center for 

 Health Statistics

Rolf Schmitt, Federal Highway Administration

Marilyn McMillen Seastrom, National Center for 

 Education Statistics

Monroe Sirken, National Center for Health Statistics

Nancy Spruill, Department of Defense

Clyde Tucker, Bureau of Labor Statistics

Katherine K. Wallman, (Champion) Office of 

 Management and Budget

G. David Williamson, Agency for Toxic Substances 

 and Disease Registry 

Federal Committee on Statistical Methodology Members

(July 2009) 

Additional Conference Support

 Lee Ann Sklar, Council of Professional Associations on Federal Statistics

Note: Papers and discussant comments will be available in early 2010 on <www.fcsm.gov>. 

2
2009 FCSM Research Conference

The 2009 Federal Committee on Statistical Methodology (FCSM) Research Conference was initiated 

by the FCSM. The FCSM is an interagency committee dedicated to improving the quality of federal 

statistics. The committee’s major goals are to:

฀ •฀ Communicate฀and฀disseminate฀information฀on฀statistical฀practices฀among฀all฀federal฀

 statistical agencies,

฀ •฀ Recommend฀the฀introduction฀of฀new฀methodologies฀in฀federal฀statistical฀programs฀to

 improve data quality, and

฀ •฀ Provide฀a฀mechanism฀for฀statisticians฀in฀diferent฀federal฀agencies฀to฀meet฀and฀

 exchange ideas.

The 2009 FCSM Research Conference provides a forum for experts from around the world to discuss 

and exchange information on current research and methodological topics relevant to federal government statistical programs. Each day of the conference papers will be presented on a wide range of 

topics, including the use of advanced technologies for survey design and data collection, processing 

and dissemination, data mining, data warehousing and metadata, treatment of missing data, improving coverage and response rates, confidentiality and disclosure issues, record linkage, sample design 

and estimation, cognitive research and usability testing, and data quality. 

Technical demonstrations will run concurrently on the second day of the conference during the first 

morning session. Applications will include demonstrations on Q-Notes, Field Interview (FI) Tracker 

Tool, Statipedia, and Census Coverage Measurement Clerical Matching Software.

Sessions feature papers and demonstrations by government, private sector, and academic researchers from eight countries. All paper sessions will include an open discussion, and some sessions 

will include a formal discussion. Papers will be made available at the conference on a CD-ROM and 

posted to the FCSM Web site (www.fcsm.gov) following the conference.

In the opening plenary session, Katharine Abraham from the University of Maryland will discuss 

“What We Don’t Know Could Hurt Us: Some Reflections on the Measurement of Economic Activity.” 

1
2
3

Final 

Program
4
5

Federal Committee on Statistical Methodology Research Conference

Washington, D.C.–November 2–4, 2009

Monday (11/2) Tuesday (11/3) Wednesday (11/4)

7:30 a.m.

Registration

(Foyer near Room 151A-B)

Coffee 

(Foyer near Room 151A-B)

9–10 a.m.

Welcoming Remarks and

PLENARY SESSION I

(Room 151A-B)

10–10:30 a.m.

Break 

(Foyer near Room 151A-B)

10:30 a.m.–12 noon

CONCURRENT SESSION

II-A II-B II-C

(Room 151A) (Room 151B) (Room 152A)

7:30 a.m.

Registration

(Foyer near Room 151A-B)

Coffee 

(Foyer near Room 151A-B)

9–10:30 a.m.

Technical Demonstrations

(Room 154A-B)

9–10:30 a.m.

CONCURRENT SESSION

V-A V-B V-C

(Room 151A) (Room 151B) (Room 152A)

10:30–11 a.m.

Break

(Foyer near Room 151A-B)

7:30 a.m. 

Registration 

(Foyer near Room 151A-B)

Coffee 

(Foyer near Room 151A-B)

9–10:30 a.m.

CONCURRENT SESSION

IX-A IX-B IX-C 

(Room 151A) (Room 151B) (Room 152A)

10:30–11 a.m.

Break 

(Foyer near Room 151A-B)

12 noon–1:15 p.m.

Open

1:30–3 p.m.

CONCURRENT SESSION

III-A III-B III-C

(Room 151A) (Room 151B) (Room 152A)

3–3:30 p.m.

Break

(Foyer near Room 151A-B)

3:30–5 p.m. 

CONCURRENT SESSION 

IV-A IV-B IV-C 

(Room 151A) (Room 151B) (Room 152A)

11 a.m.–12:30 p.m.

CONCURRENT SESSION

VI-A VI-B VI-C

(Room 151A) (Room 151B) (Room 152A)

12:30–1:45 p.m.

Open

2–3:30 p.m.

CONCURRENT SESSION

VII-A VII-B VII-C 

(Room 151A) (Room 151B) (Room 152A)

3:30–4 p.m.

Break

(Foyer near Room 151A-B)

4–5:30 p.m.

CONCURRENT SESSION

VIII-A VIII-B 

(Room 151A) (Room 151B) 

11 a.m.–12:30 p.m.

CONCURRENT SESSION

X-A X-B X-C

(Room 151A) (Room 151B) (Room 152A)

12:30–1:45 p.m.

Open

2–3:30 p.m.

CONCURRENT SESSION

XI-A XI-B XI-C

(Room 151A) (Room 151B) (Room 152A)

Meeting Rooms: 

Room 151A

Room 151B

Room 152A

Room 154A-B

All rooms located at street level 

in the middle building. Entrance 

at 7th and M Street, NW, near the 

Mt. Vernon Square/Convention 

Center Metro stop is closest to 

the meeting rooms.
6

Final Program1

Monday, November 2

7:30 a.m.–5 p.m. Foyer near Room 151A-B

Registration

7:30–9 a.m. Foyer near Room 151A-B

Coffee

9–9:10 a.m. Room 151A-B

Introduction and Welcoming Remarks

9:10–10 a.m.

PLENARY SESSION I

What We Don’t Know Could Hurt Us: 

Some Reflections on the Measurement 

of Economic Activity 

Katharine Abraham (University of Maryland, USA)

10–10:30 a.m Foyer near Room 151A-B

Break

10:30 a.m.–12 noon Room 151A

CONCURRENT SESSION II-A:

REDESIGNING THE NATIONAL CRIME 

VICTIMIZATION SURVEY

Chair: Michael Rand (Bureau of Justice Statistics, USA)

Assessing Sample Design Options for the 

National Crime Victimization Survey

Robert Fay (Westat, USA)

Jianzhu Li (Westat, USA)

Analysis of Possible Nonresponse Bias in the 

National Crime Victimization Survey

Fritz Scheuren (NORC at the University of 

Chicago, USA)

Zhiwei Zhang (NORC at the University of 

Chicago, USA)

Examination of Data Collection Methods for the 

National Crime Victimization Survey

Lisa Carley-Baxter (RTI International, USA)

Emilia Peytcheva (RTI International, USA)

Vince Iannacchione (RTI International, USA)

Rachel Caspar (RTI International, USA)

Experimenting with Speech Interactive Voice 

Response, Touchtone Data Entry and the Web 

for the National Crime Victimization Survey 

David Cantor (Westat, USA) 

Pat Dean Brick (Westat, USA) 

Roger Tourangeau (University of Maryland, USA) 

Frederick Conrad (University of Michigan, USA)

Evaluation of a 12-Month Reference Period in 

the National Crime Victimization Survey (NCVS)

Lisa Lee (NORC at the University of Chicago, USA) 

James Carr (NORC at the University of Chicago, USA)

Session Organizer: Paul Guerino (Bureau of Justice 

Statistics, USA)

10:30 a.m.–12 noon Room 151B

CONCURRENT SESSION II-B:

ECONOMIC INDICATORS

Chair: Dennis Fixler (Bureau of Economic Analysis, 

USA)

Modeling Corporate Failure: A PLS Logistic 

Regression

Ben Jabeur Sami (University of Toulon-Var, France)

Aroua Rachid (University of Nice Sophia Antipolis, 

France)

Music as a Capital Asset

Rachel Soloveichik (Bureau of Economic Analysis, USA)

The Transportation Services Index: Its Methodology and Relationship to the Business Cycle

Kenneth Notis (U.S. Department of Transportation)

Peg Young (U.S. Department of Transportation)

Reliability of Price Indices to Age Expenditure 

Data Used in Government Measures

Sharon O’Donnell (U.S. Census Bureau)

Stuart McDonald (University of Queensland, Australia)

Session Organizer: Benjamin Bridgman (Bureau of 

Economic Analysis, USA)

10:30 a.m.–12 noon Room 152A

CONCURRENT SESSION II-C:

BAYESIAN METHODS

Chair: Charlie Hallahan (U.S. Department 

of Agriculture)

Record Linkage Modeling in Federal Statistical 

Databases

Michael Larsen (George Washington University, USA) 

1In the case of coauthors, the presenter is underlined.
7

Generalized Maximum Likelihood Method in 

Linear Mixed Models with an Application in the 

Small-Area Estimation 

Partha Lahiri (University of Maryland, USA)

Huilin Li (National Institutes of Health, USA)

Spatial-Temporal Assimilation of Fine Fraction 

Particulate Matter

Ana Rappold (U.S. Environmental Protection Agency)

Marco Ferreira (University of Missouri, USA)

The Spatio-Temporal Determinants of Charter 

School Site Selection in New York City

Charisse Gulosino (Brown University, USA)

Session Organizer: Charlie Hallahan (U.S. Department 

of Agriculture)

12 noon–1:15 p.m.

Open Lunch

1:30–3 p.m. Room 151A

CONCURRENT SESSION III-A: 

ISSUES IN DATA CONFIDENTIALITY AND 

DISCLOSURE RISK 

Chair: Marilyn Seastrom (National Center for 

Education Statistics, USA)

Incorporating Quality Measures in Tabular 

Data Protected by Perturbation Methods

Ramesh Dandekar (Energy Information 

Administration, USA)

Protecting Numerical Confidential Data Using 

Data Shuffling: A Demonstration of Effectiveness 

of Approach and Flexibility of Delivery

Rathindra Sarathy (Oklahoma State University, USA)

Krish Muralidhar (University of Kentucky, USA)

Vulnerability of Complementary Cell 

Suppression to Intruder Attack

Lawrence Cox (National Center for Health 

Statistics, USA)

Evaluating the Impact of Data Swapping Using 

Global Utility Measures

Sylvia Dohrmann (Westat, USA)

Tom Krenzke (Westat, USA)

Shep Roey (Westat, USA)

J. Neil Russell (National Center for Education 

Statistics, USA)

Session Organizer: Michael Planty (National Center for 

Education Statistics, USA)

1:30–3 p.m. Room 151B

CONCURRENT SESSION III-B:

STATISTICAL USES OF ADMINISTRATIVE DATA 

Chair: Carol Joyce Blumberg (Energy Information 

Administration, USA)

Using the NHIS Linked to Mortality to Examine 

Social Disparities in Life Expectancy

Kimberly Lochner (National Center for Health 

Statistics, USA)

Van Parsons (National Center for Health 

Statistics, USA)

Nathaniel Schenker (National Center for Health 

Statistics, USA)

Elsie Pamuk (National Center for Health Statistics, USA)

An Evaluation of Data from the NCES Teacher 

Compensation Survey: The Relationship Between 

Teacher Compensation and Students in Poverty 

Stephen Cornman (U.S. Department of Education)

Frank Johnson (U.S. Department of Education)

Steven Honegger (American Institutes for Research, 

USA) 

Amber Noel (American Institutes for Research, USA) 

Lei Zhou (Macrosys Research, USA)

Using Administrative Data in the 2007 Census 

of Agriculture Edit

Virginia Harris (National Agricultural Statistics 

Service, USA) 

Dan Beckler (National Agricultural Statistics 

Service, USA)

Jeffrey Beranek (National Agricultural Statistics 

Service, USA)

Discussant: Sandra Decker (National Center for Health 

Statistics, USA)

Session Organizer: Jennifer Parker (National Center 

for Health Statistics, USA)

1:30–3 p.m. Room 152A

CONCURRENT SESSION III-C: 

EVALUATING AND IMPROVING SAMPLE COVERAGE AND DATA COLLECTION PROCEDURES

Chair: Chris Chapman (National Center for Education 

Statistics, USA)

Census Population Coverage Error: Results 

and Methods

Colleen Clark (Statistics Canada)
8

Assessment of Potential Bias in Telephone 

Survey Estimates Due to Noncontact and 

Noncoverage Where Respondents Primarily 

Use Wireless Telephones or Do Not Have 

Landline Telephones

Meena Khare (National Center for Health 

Statistics, USA)

Abera Wouhib (National Center for Health 

Statistics, USA) 

Developing a Common Mode–Independent 

Approach for Evaluating Interview Quality and 

Interviewer Performance

Howard Speizer (RTI International, USA)

Susan Kinsey (RTI International, USA)

Richard Heman-Ackah (RTI International, USA)

M. Rita Thissen (RTI International, USA)

The Influence of State-Specific, Programmatic 

Characteristics in the Modeling of Medicaid 

Undercount: A Record–Check Study of the 2001 

Current Population Survey Annual Social and 

Economic Supplement (CPS ASEC)

Matthew Stevens (U.S. Census Bureau)

Session Organizer: Chris Chapman (National Center 

for Education Statistics, USA)

3–3:30 p.m Foyer near Room 151A-B

Break

3:30–5 p.m. Room 151A

CONCURRENT SESSION IV-A: 

MODELING OF DATA

Chair: Chris Chapman (National Center for Education 

Statistics, USA) 

Project for the Evaluation of the Economic 

and Safety Situation of the Street Transportation 

Using a Cost-Effective Methodology: The Case 

of Cameroon

Ramses Zang (Centre for Studies and Research on 

International Development [CERDI], France)

Sebastien Djienouassi (Centre for Studies and 

Research on International Development [CERDI], 

France)

Annie Laurence Mapondjou (Centre for Studies and 

Research on International Development [CERDI], 

France)

More Efficient Local Polynomial Regression on 

Random-Effects Panel Data Models

Ke Yang (University of Hartford, USA)

Assessing Immigrant Naturalization: 

Longitudinal Research Findings and Challenges

Karen Woodrow-Lafield (Unaffiliated Scholar)

First–Line Supervisors: An Analysis of Job Satisfaction and Job Proficiency

Kelly Maguire (U.S. Environmental Protection Agency)

Charles Griffiths (U.S. Environmental Protection 

Agency)

Session Organizer: Chris Chapman (National Center 

for Education Statistics, USA)

3:30–5 p.m. Room 151B

CONCURRENT SESSION IV-B:

DEFINING THE GREEN ECONOMY; AND ISSUES IN U.S. 

OCCUPATIONAL CLASSIFICATION SYSTEMS 

Chair: Jeff Beaubier (U.S. Environmental Protection 

Agency)

Occupational Classification Systems: Analyzing 

the 2010 SOC Revision

Alissa Emmel (Bureau of Labor Statistics, USA)

Theresa Cosca (Bureau of Labor Statistics, USA)

Anne Louise Marshall (Bureau of Labor Statistics, USA) 

Defining the Green Economy

Richard Clayton (Bureau of Labor Statistics, USA)

George Stamas (Bureau of Labor Statistics, USA)

Updated Unified Category System for 1960-2000 

Census Occupations

Peter Meyer (Bureau of Labor Statistics, USA) 

Discussant: Barry Nussbaum (U.S. Environmental 

Protection Agency)

Session Organizer: Jeff Beaubier (U.S. Environmental 

Protection Agency)

3:30–5 p.m. Room 152A

CONCURRENT SESSION IV-C: 

IMPROVING COVERAGE AND RESPONSE RATES

Chair: John Finamore (U.S. Census Bureau)

SIPP 2008 Incentive Analysis

Stephen Clark (U.S. Census Bureau) 

Stephen Mack (U.S. Census Bureau)

Adaptive Contact Strategies in a Telephone 

Survey

James Wagner (University of Michigan, USA)

Characteristics of Parent Respondents in the 

National Household Education Surveys

Cameron McPhee (American Institutes for Research, USA) 

Stacey Bielick (American Institutes for Research, USA)

Measuring the Effects of Reminder Postcards in 

the Occupational Employment Statistics Survey

Carrie Jones (Bureau of Labor Statistics, USA)

Session Organizer: John Finamore (U.S. Census Bureau)
9

Tuesday, November 3

7:30 a.m.–5:30 p.m. Foyer near Room151A-B

Registration

7:30–9 a.m. Foyer near Room151A-B

Coffee

9–10:30 a.m. Room 154A-B

TECHNICAL DEMONSTRATIONS

DEMONSTRATION OF SOFTWARE TOOLS FOR 

COLLABORATION, MATCHING, AND ANALYZING 

QUESTION QUALITY AND PARADATA

Chair: William Mockovak (Bureau of Labor Statistics, USA) 

Q-Notes: Development and Use of Analysis 

Software for Cognitive Interviews to Examine 

Survey Question Comparability

Kristen Miller (National Center for Health Statistics, USA) 

Aaron Maitland (National Center for Health 

Statistics, USA) 

Heather Ridolfo (National Center for Health 

Statistics, USA) 

Stephanie Willson (National Center for Health 

Statistics, USA) 

Justin Mezetin (National Center for Health Statistics, USA) 

A Demonstration of FI Tracker: A Tool for 

Displaying and Monitoring Survey 

Interview Paradata

Joe Murphy (RTI International, USA)

Susan Myers (RTI International, USA)

Barbara Bibb (RTI International, USA) 

Statipedia—A Platform for 

Collaboration Across Statistical Agencies

Peter Meyer (Bureau of Labor Statistics, USA)

James Buszuwski (Bureau of Labor Statistics, USA)

Demonstration of Census Coverage 

Measurement Clerical Matching Software

Tamara Adams (U.S. Census Bureau)

Susanne Johnson (U.S. Census Bureau)

Lee Goldberg (U.S. Census Bureau)

Paul Clemm (U.S. Census Bureau)

9–10:30 a.m. Room 151A

CONCURRENT SESSION V-A: 

SOCIAL SECURITY ADMINISTRATION USES 

OF ADMINISTRATIVE RECORDS LINKED WITH 

SURVEY DATA 

Chair: Jeri Mulrow (Social Security Administration, USA)

Pension Coverage and Pension Types: Using 

W–2 Tax Records to Correct SIPP Suvey Reports

Howard Iams (Social Security Administration, USA)

Irena Dushi (Social Security Administration, USA) 

Measurement Error in Estimates of the Participation Rate in Means–Tested Programs: The Case 

of the U.S. Supplemental Security Income Program for the Elderly

Alexander Strand (Social Security Administration, USA) 

Kalman Rupp (Social Security Administration, USA) 

Paul Davies (Social Security Administration, USA)

Using Matched Survey and Administrative Data 

to Estimate Eligibility for the Medicare Part D 

Program Low Income Subsidy

Lynn Karoly (Rand Corporation, USA)

Erik Meijer (Rand Corporation, USA)

Pierre-Carl Michaud (Rand Corporation, USA)

Discussant: John Czajka (Mathematica Policy 

Research, Inc., USA) 

Session Organizer: Jim Sears (Social Security 

Administration, USA) 

9–10:30 a.m. Room 151B 

CONCURRENT SESSION V-B: 

STATISTICAL WEIGHTING AND EDITING 

Chair: Eugene Burns (Energy Information 

Administration, USA)

Census Editing and the Art of Motorcycle Maintenance

Michael Levin (Harvard University, USA)

Re-Weighting of CPS Data

Maryan Cammarata (U.S. Census Bureau)

Comparison of Retail Drug Prices Obtained 

in the MEPS and Commercial Claims Data: 

Implications for Price Editing Rules and 

Data Quality

Marc Zodet (Agency for Healthcare Research and 

Quality, USA)

Steven Hill (Agency for Healthcare Research and 

Quality, USA)

G. Edward Miller (Agency for Healthcare Research 

and Quality, USA)

Different Approaches for Non-Response 

Adjustments to Statistical Weights in the 

Continuous NHANES (2003–2004)

Lisa Mirel (National Center for Health Statistics, USA)

Vicki Burt (National Center for Health Statistics, USA)

Lester Curtin (National Center for Health Statistics, USA)

Cindy Zhang (National Center for Health Statistics, USA)

Session Organizer: Ruey-Pyng Lu (Energy Information 

Administration, USA)
10

9–10:30 a.m. Room 152A

CONCURRENT SESSION V-C: 

IMPROVEMENTS IN SAMPLE DESIGN AND 

SURVEY ESTIMATION

Chair: John Finamore (U.S. Census Bureau)

An Evaluation of the Use of the American 

Community Survey as the Sampling Frame for 

the National Immunization Survey (NIS)

James Singleton (Centers for Disease Control 

and Prevention, USA)

Philip Smith (Centers for Disease Control and 

Prevention, USA)

Andrea Piani (U.S. Census Bureau)

Christopher Stringer (U.S. Census Bureau)

NIS Evaluation Study

Kimball Jonas (U.S. Census Bureau)

Methodological Changes in the Job 

Openings and Labor Turnover Survey

John Wohlford (Bureau of Labor Statistics, USA)

Issues Related to Adding Sub-Annual 

Estimates to the Data Products Available 

from the American Community Survey

Karen King (U.S. Census Bureau)

Alfredo Navarro (U.S. Census Bureau)

Mark Asiala (U.S. Census Bureau)

B. Dale Garrett (U.S. Census Bureau)

Michael Starsinic (U.S. Census Bureau)

Sirius Fuller (U.S. Census Bureau)

J. Gregory Robinson (U.S. Census Bureau)

Michael Ikeda (U.S. Census Bureau)

Charles Nelson (U.S. Census Bureau)

Scott Boggess (U.S. Census Bureau)

Session Organizer: John Finamore (U.S. Census Bureau)

10:30–11 a.m. Foyer near Room 151A-B

Break

11 a.m.–12:30 p.m. Room 151A

CONCURRENT SESSION VI-A: 

UNIQUE METHODS FOR CONSIDERATION 

REGARDING DATA CONFIDENTIALITY

Chair: Tom Petska (Retired, Statistics of Income 

Division, Internal Revenue Service, USA)

The Unique Method for Obtaining Data: Entering Agreements to Share Administrative Records

Stephen Cornman (U.S. Department of Education, USA)

Statistical Disclosure Control of Tabular Format 

Magnitude Data: Why It Is Not A Good Idea to 

Use Home Grown (Manual or Automated) Cell 

Suppression Procedures

Ramesh Dandekar (Energy Information Administration, 

USA)

Privacy Violations in Accountability Data 

Released to the Public by State Educational 

Agencies

Krish Muralidhar (University of Kentucky, USA)

Rathindra Sarathy (Oklahoma State University, USA) 

Expert Task Force Consideration of Confidentiality, Effect Sizes, and Computerized Adaptive 

Testing at NCES

Andrew White (National Center for Education 

Statistics, USA)

Marilyn Seastrom (National Center for Education 

Statistics, USA)

Session Organizer: Kevin Cecco (Statistics of Income 

Division, Internal Revenue Service, USA) 

11 a.m.–12:30 p.m. Room 151B

CONCURRENT SESSION VI-B: 

IMPROVING DATA QUALITY THROUGH THE USE 

OF ECONOMETRIC MODELING

Chair: Mina Kim (Bureau of Labor Statistics, USA)

A Model to Improve Timeliness of Business 

Register Data

Filipponi Danila (National Institute of Statistics 

[Istat], Italy)

Patrizia Cella (National Institute of Statistics 

[Istat], Italy)

Giuseppe Garofalo (National Institute of Statistics 

[Istat], Italy)

Fitting A Linear Model to Survey Data When the 

Long-Term Average Daily Intake of a Dietary 

Component Is An Explanatory Variable

Phillip Kott (RTI International, USA)

Patricia Guenther (U.S. Department of Agriculture)

David Wagstaff (Pennsylvania State University, USA)

WenYen Juan (U.S. Food and Drug Administration)

Sibylle Kranz (East Carolina University, USA)

Collinearity Diagnostics for Complex 

Survey Data

Dan Liao (University of Maryland, USA)

Multilevel Models and Small Area Estimation 

in the Context of Vietnam Living Standards 

Surveys 

Phong Nguyen (General Statistics Office of Vietnam, 

Vietnam)

Dominique Haughton (Bentley University and Toulouse 

School of Economics, USA) 

Irene Hudson (University of South Australia, Australia)

John Boland (University of South Australia, Australia)

Session Organizer: Thesia Garner (Bureau of Labor 

Statistics, USA) 
11

11 a.m.–12:30 p.m. Room 152A

CONCURRENT SESSION VI-C: 

VARIANCE ESTIMATION 

Chair: Tamara Rib (Internal Revenue Service, USA)

The RAO-WU Rescaling Bootstrap: From Theory to 

Practice

Claude Girard (Statistics Canada)

Linearization Variance Estimation and Allocation 

for Two-Phase Sampling Under Mass Imputation

Abdellatif Demnati (Statistics Canada)

J.N.K. Rao (Carleton University, Canada)

Variance Approximations for Price Index 

Estimation

John Eltinge (Bureau of Labor Statistics, USA)

Moon Jung Cho (Bureau of Labor Statistics, USA)

On Point Estimation and Variance Estimation for 

Mean Expenditures in the Consumer Expenditure 

Survey

Michail Sverchkov (Bureau of Labor Statistics, USA)

John Eltinge (Bureau of Labor Statistics, USA)

Lawrence Ernst (Bureau of Labor Statistics, USA)

Session Organizer: Kim Henry (Internal Revenue 

Service, USA)

12:30–1:45 p.m.

Open Lunch

2–3:30 p.m. Room 151A

CONCURRENT SESSION VII-A: 

INFORMED CONSENT AND PARTICIPATION

Chair: Paul Guerino (Bureau of Justice Statistics, USA)

Securing Informed Consent from Less Educated 

Respondents: Results from the National Inmate 

Survey

Rachel Caspar (RTI International, USA)

Paul Guerino (Bureau of Justice Statistics, USA)

Motivating Non-English-Speaking Populations 

for Census and Survey Participation

Yuling Pan (U.S. Census Bureau)

Nancy Bates (U.S. Census Bureau)

Informed Consent and the Statistical Use of 

Administrative Records

Michael Hawes (U.S. Census Bureau)

Carol Comisarow (U.S. Census Bureau)

Discussant: Al Zarate (Consultant, National Center for 

Health Statistics, USA) 

Session Organizer: Paul Guerino (Bureau of Justice 

Statistics, USA)

2–3:30 p.m. Room 151B

CONCURRENT SESSION VII-B: 

CHALLENGES OF FEDERAL HOUSEHOLD SURVEYS 

IN EVALUATING FINANCIAL WELL-BEING

Chair: Thesia Garner (Bureau of Labor Statistics, USA)

A Tale of Two Surveys: Mortgage Wealth Data 

in the AHS and the SIPP

George Carter III (U.S. Census Bureau)

Alfred Gottschalck (U.S. Census Bureau)

Income Measurement in Federal Surveys

John Czajka (Mathematica Policy Research, Inc., USA)

Gabrielle Denmead (Denmead Services and 

Consulting, USA)

Evaluation of Health Insurance Coverage Estimates from the American Community Survey

Joanna Turner (U.S. Census Bureau)

Michel Boudreaux (University of Minnesota, USA)

Victoria Lynch (Urban Institute)

Discussant: Brian Bucks (Federal Reserve Board, USA)

Session Organizer: Thesia Garner (Bureau of Labor 

Statistics, USA)

2–3:30 p.m. Room 152A

CONCURRENT SESSION VII-C: 

EFFECTS OF IMPUTATION ON SURVEY ESTIMATES 

Chair: John Bushery (U.S. Census Bureau)

Effects of Imputation on CPS Poverty Series, 

1987–2007 

Joan Turek (U.S. Department of Health and Human 

Services)

Brian Sinclair-James (U.S. Department of Health 

and Human Services)

Bula Ghose (U.S. Department of Health and 

Human Services)

Fritz Scheuren (NORC at the University of Chicago, USA)

Charles Nelson (U.S. Census Bureau)

Edward Welniak (U.S. Census Bureau)

Testing New Imputation Methods for Earnings in 

the Survey of Income and Program Participation

Martha Stinson (U.S. Census Bureau)

Gary Benedetto (U.S. Census Bureau)

Sensitivity of Inference Under Imputation: 

An Empirical Study

Jeffrey Gonzalez (Bureau of Labor Statistics, USA)

John Eltinge (Bureau of Labor Statistics, USA)

Imputation Methods for the Current Employment 

Statistics Survey

Kennon Copeland (NORC at the University of Chicago, 

USA)

Lan Pham (Bureau of Labor Statistics, USA)

Session Organizer: Pamela McGovern (U.S. 

Census Bureau) 
12

3:30–4 p.m. Foyer near Room 151A-B

Break

4–5:30 p.m. Room 151A

CONCURRENT SESSION VIII-A: 

IDENTIFYING AND REDUCING SOURCES OF

SURVEY ERROR

Chair: William Mockovak (Bureau of Labor 

Statistics, USA)

Measuring the Complexity and Importance of 

Businesses in Order to Better Manage our Data 

Collection Efforts

Serge Godbout (Statistics Canada)

Sungjin Youn (Statistics Canada)

Assessing and Correcting the Effects of 

Measurement Error on the 2008 Economic 

Directorate Editing and Imputation Inventory

Laura Ozcoskun (U.S. Census Bureau)

La Toya Thomas (U.S. Census Bureau)

Michael Hayes (U.S. Census Bureau)

Models of Nonresponse and Their Relationship 

to Survey Estimates

John Dixon (Bureau of Labor Statistics, USA) 

Findings from a Pretest of a New Approach to 

Measuring Health Insurance in the Current 

Population Survey

Joanne Pascale (U.S. Census Bureau) 

Session Organizer: William Mockovak (Bureau of 

Labor Statistics, USA)

4–5:30 p.m. Room 151B

CONCURRENT SESSION VIII-B: 

FORECASTING

Chair: Charlie Hallahan (U.S. Department 

of Agriculture)

Using Spectral Peaks to Detect Seasonality

Tucker McElroy (U.S. Census Bureau)

Scott Holan (University of Missouri-Columbia, USA)

The Term Structure with Macro Factors, 

Expectations Hypothesis and Regime Shift

Xiaoneng Zhu (Nanyang Technological University, 

Singapore)

A Review of Air Travel Passengers Forecasting 

Models and Their Robustness

Ali Dadpay (Macrosys Research, USA)

Charlie Han (Macrosys Research, USA)

David Chesser (Macrosys Research, USA)

Antony Homan (U.S. Department of Transportation)

Seasonal Adjustment of Short Time Series with 

Calendar Effect Treatments – Applied Methods in 

the Bureau of Labor Statistics

Brian Dahlin (Bureau of Labor Statistics, USA)

Session Organizer: Charlie Hallahan (U.S. Department 

of Agriculture)
13

Wednesday, November 4

7:30 a.m.–5:30 p.m. Foyer near Room151A-B

Registration 

7:30–9 a.m. Foyer near Room 151A-B

Coffee

9–10:30 a.m. Room 151A-B

CONCURRENT SESSION IX-A: 

ISSUES CONCERNING THE STATISTICAL USE OF 

ADMINISTRATIVE RECORDS

Chair: Jock Black (National Science Foundation, USA)

Overview of Open Research Questions in 

Methodological and Empirical Work with 

Statistical Uses of Administrative Records

Shelley Wilkie Martinez (Office of Management and 

Budget, USA)

John Eltinge (Bureau of Labor Statistics, USA)

Jenna Fulton (University of Maryland, USA)

Expanding Statistical Use of Administrative 

Data: A Research Proposal Focused on Privacy 

and Confidentiality

Gerald Gates (Gerald Gates Privacy Consulting, USA) 

Investing in a Data Quality Assurance Program 

for Administrative Data Linked to Survey Data 

for Policy Research Purposes Is Essential

Michael Davern (University of Minnesota, USA)

Discussant: Robert Fay (Westat, USA)

Session Organizer: Jock Black (National Science 

Foundation, USA)

9–10:30 a.m. Room 151B

CONCURRENT SESSION IX-B: 

WEB SURVEYS 

Chair: Joy Sharp (Bureau of Transportation 

Statistics, USA)

Internet Surveys and the Demise of the Matrix

Grace O’Neill (Energy Information Administration, 

USA)

Stan Freedman (Energy Information Administration, 

USA)

The Effects of Grouping Response Options in 

Factual Questions With Many Options

Cleo Redline (University of Maryland, USA)

Roger Tourangeau (University of Maryland, USA)

Mick Couper (University of Michigan, USA)

Frederick Conrad (University of Michigan, USA)

Cong Ye (University of Maryland, USA)

Response Format Effects on Measurement of 

Employment

Randall Thomas (ICF International, USA)

Jolene Smyth (University of Nebraska-Lincoln, USA)

Don Dillman (Washington State University, USA)

Mode Effects in Mixed-Mode Surveys of Veteran 

and Military Populations

Boris Rachev (ICF International, USA)

Session Organizer: Joy Sharp (Bureau of Transportation Statistics, USA)

9–10:30 a.m. Room 152A

CONCURRENT SESSION IX-C: 

ISSUES IN SAMPLE DESIGN

Chair: David Kinyon (U.S. Census Bureau) 

Revisiting Nested Stratification of Primary 

Sampling Units

Tom Krenzke (Westat, USA)

Wen-Chau Haung (Westat, USA)

Statistical Sample Design for Coalbed Methane 

Industry Survey: Projects Versus Wells

Marla Smith (U.S. Environmental Protection Agency)

Carey Johnston (U.S. Environmental Protection Agency)

Maximizing Overlap of Large Primary Sampling 

Units in Repeated Sampling: A Comparison of 

Ernst’s Method with Ohlsson’s Method

Padraic Murphy (U.S. Census Bureau)

Reid Rottach (U.S. Census Bureau)

Minimizing Duplication of Samples Drawn from 

Overlapping Frames of Multiple Surveys

Peter Kwok (NORC at the University of Chicago, USA)

Hee-Choon Shin (NORC at the University of 

Chicago, USA)

Colm O’Muircheartaigh (NORC at the University of 

Chicago, USA)

Whitney Murphy (NORC at the University of 

Chicago, USA)

Angela Debello (NORC at the University of 

Chicago, USA)

Kari Carris (NORC at the University of Chicago, USA)

Youlian Liao (Centers for Disease Control and 

Prevention, USA)

Session Organizer: Jock Black (National Science 

Foundation, USA)
14

10:30–11 a.m. Foyer near Room 151A-B

Break

11 a.m.–12:30 p.m. Room 151A

CONCURRENT SESSION X-A: 

PRODUCING AND COMMUNICATING DATA 

AND STATISTICS

Chair: Jennifer Parker (National Center for Health 

Statistics, USA)

Metadata and Data Harmonization

Daniel Gillman (Bureau of Labor Statistics, USA)

Frank Farance (Farance Inc., USA)

Innovative Uses of Data Mining Techniques in 

the Production of Official Statistics 

Jaki McCarthy (National Agricultural Statistics 

Service, USA) 

Thomas Jacob (National Agricultural Statistics 

Service, USA) 

Dale Atkinson (National Agricultural Statistics 

Service, USA)

Benchmarking the Performance of Statistical 

Agencies

Marilyn Seastrom (National Center for Education 

Statistics, USA)

Valena Plisko (National Center for Education 

Statistics, USA)

Session Organizer: Jennifer Parker (National Center 

for Health Statistics, USA)

11 a.m.–12:30 p.m. Room 151B

CONCURRENT SESSION X-B:

REDESIGNING LARGE-SCALE FEDERAL SURVEYS: 

CONNECTING TO THE PAST, ANTICIPATING 

THE FUTURE

Chair: Jonaki Bose (Substance Abuse and Mental 

Health Services Administration, USA)

Searching for Alternatives to a Random Digit 

Dial Telephone Interview–Redesigning the 

National Household Education Surveys

Chris Chapman (National Center for Education 

Statistics, USA)

Mary Hagedorn (Westat, USA)

Redesigning the National Surveys on Drug Use 

and Health

Joseph Gfroerer (Substance Abuse and Mental Health 

Services Administration, USA)

Jonaki Bose (Substance Abuse and Mental Health 

Services Administration, USA)

Dicy Painter (Substance Abuse and Mental Health 

Services Administration, USA)

Michael Jones (Substance Abuse and Mental Health 

Services Administration, USA)

Joel Kennet (Substance Abuse and Mental Health 

Services Administration, USA) 

Redesigning the National Crime Victimization 

Survey

Michael Rand (Bureau of Justice Statistics, USA)

Discussant: Jennifer Madans (National Center for 

Health Statistics, USA)

Session Organizer: Jonaki Bose (Substance Abuse and 

Mental Health Services Administration, USA)

11 a.m.–12:30 p.m. Room 152A

CONCURRENT SESSION X-C: 

ADVANCES IN AGRICULTURAL FRAME 

DEVELOPMENT AND USE

Chair: William Iwig (National Agricultural Statistics Service, 

USA)

Utilizing an Alternative Sampling Frame to Produce Agricultural Survey Indications

Wendy Barboza (National Agricultural Statistics Service, 

USA)

Mark Harris (National Agricultural Statistics Service, USA)

Four Digits or No-Digit Social Security Numbers–

Impact on the National Agricultural Statistics 

Service Record Linkage Maintenance Processes

Denise Abreu (National Agricultural Statistics Service, 

USA)

Kara Daniel (National Agricultural Statistics Service, 

USA)

William Iwig (National Agricultural Statistics Service, 

USA)

Stan Hoge (National Agricultural Statistics Service, 

USA)

Comparison of Methods for Updating Census 

Based Estimates of Number of Farms to 

Non-Census Years

Michael Bellow (National Agricultural Statistics Service, 

USA)

Phillip Kott (RTI International, USA)

Discussant: Ron Fesco (U.S. Government Accountability 

Office)

Session Organizer: Dale Atkinson (National 

Agricultural Statistics Service, USA)
15

12:30–1:45 p.m. 

Open Lunch

2–3:30 p.m. Room 151A

CONCURRENT SESSION XI-A: 

APPLICATIONS OF SAMPLING

Chair: Dale Atkinson (National Agricultural Statistics 

Service, USA)

Evolutionary Algorithms for Optimal Sample 

Design

Charles Day (Statistics of Income Division, Internal 

Revenue Service, USA)

Deviation of Sample Size Formula for Cluster 

Randomized Trials with Binary Responses Using a 

General Continuity Correction Factor and Identification of Optimal Settings for Small Event Rates

Majnu John (Weill Cornell Medical College, USA)

Madhu Mazumdar (Weill Cornell Medical College, USA)

Sampling from Discrete Distributions: 

Application to An Editing Problem

Lawrence Cox (National Center for Health Statistics, USA)

Marco Better (OptTek Systems, Inc., USA

Discussant: Michael P. Cohen (Statistical Consultant, 

NORC at the University of Chicago, and George Mason 

University, USA)

Session Organizer: Dale Atkinson (National 

Agricultural Statistics Service, USA)

2–3:30 p.m. Room 151B

CONCURRENT SESSION XI-B: 

INCOME AND THE BUSINESS CYCLE

Chair: Benjamin Bridgman (Bureau of Economic 

Analysis, USA)

Sources of Earnings Volatility

Beethika Khan (U.S. Department of Commerce)

David Beede (U.S. Department of Commerce)

The Impact of a Rapid Recession on the 

Statistical Process

Pieter Vlag (Statistics Netherlands)

Government Policy Strategies over the Current 

Business Cycle: Should New Business Start-Ups 

or Existing Businesses Expansion Be Favored

Ting Zhang (University of Baltimore, USA)

David Stevens (University of Baltimore, USA)

Discussant: John Ruser (Bureau of Labor 

Statistics, USA)

Session Organizer: Benjamin Bridgman (Bureau of 

Economic Analysis, USA)

2–3:30 p.m. Room 152A

CONCURRENT SESSION XI-C: 

EFFECTS OF QUESTIONNAIRE AND SURVEY 

DESIGN ON RECALL AND RESPONSE

Chair: David Kashihara (Agency for Healthcare Research 

and Quality, USA)

Establishment Respondents as Survey Managers: 

Using Survey Design Features to Empower Respondents to Find and Recruit Knowledgeable Company 

Personnel for Assistance

Alfred Tuttle (U.S. Census Bureau)

Using Negatively Framed Questions to 

Evaluate Nursing Home Care

Elizabeth Frentzel (American Institutes for 

Research, USA)

Judith Sangl (Agency for Healthcare Research and 

Quality, USA)

Julie Brown (Rand, USA)

Carol Cosenza (University of Massachusetts, USA)

Chris Evensen (American Institutes for Research, USA)

San Keller (American Institutes for Research, USA)

The Effects of Recall Length and Reporting Aids 

on Household Reporting of Health Care Events 

in the Medical Expenditure Panel Survey

Samuel Zuvekas (Agency for Healthcare Research and 

Quality, USA)

Response Order Effect and Social Desirability 

Bias in the New Dutch Security Monitor

Thomas Kraan (Statistics Netherlands)

Jan van den Brakel (Statistics Netherlands)

Bart Buelens (Statistics Netherlands)

Harry Huys (Statistics Netherlands)

Session Organizer: David Kashihara (Agency for 

Healthcare Research and Quality, USA)
16
17

Abstract 

Booklet

This section represents abstracts received as of August 2009.

The following abstracts have not been edited for content.
18

CONCURRENT SESSION II-A:

REDESIGNING THE NATIONAL CRIME VICTIMIZATION SURVEY

Assessing Sample Design Options for the National Crime Victimization Survey

Robert Fay and Jianzhu Li (Westat, USA)

The National Crime Victimization Survey (NCVS) began full-scale data collection in 1972. Over time, increasing 

costs of data collection and a diminishing budget has forced a gradual decline of the sample size from 72,000 

households in 1972 to 38,600 in 2005, leaving the NCVS far less able to meet its initial goals. The sponsor of 

the NCVS, the Bureau of Justice Statistics (BJS), asked a panel of the National Academy of Sciences to review the 

survey. The panel’s interim report, released in 2008, recommended to BJS a series of actions, including a 

systematic review of a range of sample design options for the survey.

Since the beginning of the NCVS, the Census Bureau has been the primary architect and data collector. The 

report addressed the question of whether BJS should consider an alternative collector, but it detailed reasons 

for and against continuing with the Census Bureau. The report recommended, however, that the Census Bureau 

provide BJS with detailed information on costs to allow external assessment of survey design options. The 

report also recommended that BJS consult external sources to evaluate design options. On that basis, BJS has 

funded the research to be reported here.

This paper summarizes our research design for the study. The panel report provided two forms of guidance 

for this research. For one, it listed the primary components of the design of a national household survey—the 

design, stratification, and selection of primary sampling units, methods for selecting households, the choice 

between a rotating panel or cross-sectional design, and possible subsampling of individuals within households. The panel report also recommended consideration of specific alternative designs, including the design 

of the British Crime Survey. Our paper responds to both forms of guidance, attempting to build on the panel’s 

thoughtful review of the current status of the NCVS.

Analysis of Possible Nonresponse Bias in the National Crime Victimization Survey

Fritz Scheuren and Zhiwei Zhang (NORC at the University of Chicago, USA)

NORC is conducting a nonresponse bais study on the National Crime Victimization Survey (NCVS) for the Bureau 

of Justice Statistics (BJS). Although the NCVS still achieves a household response rate over 90 percent, response 

rates for most household surveys in the U.S. are declining – a cause of concern for the NCVS. Major consequences of increasing nonresponse rates include higher survey costs and potential biases in survey estimates.

In this presentation, we will discuss the four analytical methods that are being used for the study of potential 

nonresponse bias in the NCVS. These four methods include:

1. A new modeling approach labeled Capture/Recapture which uses two or more waves of the NCVS to estimate 

the average propensity to respond.

2. Examining differences between easy and hard-to-get NCVS respondents - both households and selected 

individual within selected households.

3. Comparing response rates and dispositions among key subgroups using call record paradata and log 

linear models. 

4. Comparing respondents and nonrespondents directly by using sample frame variables or external data that 

can be matched to the survey.

All of these methods are consistent with OMB guidelines. 

Examination of Data Collection Methods for the National Crime Victimization Survey

Lisa Carley-Baxter, Emilia Peytcheva, Vince Iannacchione, and Rachel Caspar (RTI International, USA)

The National Crime Victimization Survey (NCVS) has become the Nation’s primary source of information on the 

frequency, characteristics, and consequences of criminal victimization in the United States. Each year data are 

obtained from a nationally representative sample of approximately 38,600 housing units comprising nearly 

67,000 persons. As the cost of collecting survey data through a face-to-face mode continues to increase, the 

NCVS has faced the challenge of maintaining adequate sample sizes to support analysis needs within the funding currently available. The primary purpose of the current research is to identify new survey methods that will 

lower the cost per case for completed NCVS interviews while minimizing the impact on response rates and standard errors of key estimates. We will investigate three major avenues, as well as associated issues, for potentially reducing the costs for completing the NCVS: mixed mode data collection, reducing the number of persons 

in the household to interview, and using address-based samples to collect data in different modes. First, we 
19

will design and implement an experiment to evaluate the impact of mixed-mode (in person, telephone, and 

web) data collection coupled with incentives on survey costs, response rates, and survey estimates, including 

an examination of potential mode bias. Second, we will examine the optimal number of persons to interview 

in each household. We will use existing NCVS data to examine how survey estimates would change if only one 

person per household was interviewed. Third, we will examine the feasibility of using address-based samples to 

collect data in different modes, in part by attempting to locate names and phone numbers using commercially 

available databases. Since NCVS targets the general population in the U.S. 12 and older, we will also investigate 

possible approaches to obtaining consent and pilot test informed consent-assent procedures in the proposed 

modes of data collection.

Experimenting with Speech Interactive Voice Response, Touchtone Data Entry and the Web for the 

National Crime Victimization Survey

David Cantor, and Pat Dean Brick (Westat, USA), Roger Tourangeau (University of Maryland, USA), and 

Frederick Conrad (University of Michigan, USA)

The National Crime Victimization Survey (NCVS) is a rotating panel with housing units staying in sample for a 

total of 7 interviews. Interviewers conduct an in-person visit at the first and fourth time in sample. The Bureau 

of Justice Statistics (BJS), the sponsor of the NCVS, is initiating a redesign of the survey to improve data quality, 

increase efficiency and increase utility. One design feature being considered is the use of self-administration 

as a mode of interview. The goal is to improve the measurement of victimization, which may be underreported 

because of the sensitive nature of the events. Another goal is to reduce the cost of conducting intervieweradministered surveys. This paper will describe a project that will experiment with several types of self-administration, including the web, Touchtone Data Entry (TDE) and Speech Interactive Voice Response (SIVR).

The first part of the paper will discuss the issues involved with converting the NCVS to each of the above 

self-administered modes. The NCVS uses a set of screening questions to determine if an individual has been a 

victim. A Detailed Incident Form is filled out for each event reported on the screener. We will discuss the cognitive and human factor issues that are involved with converting this structure to a self-administered questionnaire that is audio-based (TDE, SIVR) and visually based (Web).

The second part of the paper will describe a field experiment that will examine the effects of these different 

modes of interviewing. The experiment will have four experimental conditions: 1) SIVR, 2) TDE, 3) Web and 

4) Computer Assisted Telephone Interview (CATI). Analysis of the results will compare the rates of victimization 

reported for each mode, as well as other measures of data quality (e.g., missing data). 

Evaluation of a 12-Month Reference Period in the National Crime Victimization Survey (NCVS)

Lisa Lee and James Carr (NORC at the University of Chicago, USA)

BJS has recently set forth four priority areas for methodological research on potential improvements to the 

National Crime Victimization Survey (NCVS). One of these priority areas is a recommendation to explore the 

possibility of changing the 6-month reference period to a 12-month period to increase the cost efficiency 

of the survey. 

A challenge in determining the reference period for the NCVS is to find the best balance between costs, data 

quality and respondent burden. Increasing the length of the reference period from 6 to 12 months has an 

obvious favorable impact on costs, since the data can be collected in fewer interviews. However, an increased 

reference period also risks data quality. The further crime incidents are from the data of the interview, the more 

likely they will be forgotten, leading to underreporting. A competing effect on data quality with a longer reference period is increased telescoping, in which errors in the recall of date of crime incidents may result in overreporting. Although the bounding procedures used by the NCVS may reduce the error due to telescoping, it 

may not eliminate it. Further, the longer reference period may increase interview length, increasing respondent 

burden and possibly resulting in higher unit and item non-response and break-offs during the interview. 

NORC is conducting an investigation of the impact of a 12-month reference period on data quality and respondent burden in the NCVS. NORC is developing and testing two methods of improving the accuracy of recall and 

minimizing respondent burden through the use of enhanced contextual priming and event history calendars. 

These memory aids will be tested in both web and telephone modes. In our paper we will describe the issues 

posed by a 12-month reference period for the NCVS and give a progress report on our research. 
20

CONCURRENT SESSION II-B:

ECONOMIC INDICATORS

Modeling Corporate Failure: A PLS Logistic Regression

Ben Jabeur Sami (University of Toulon-Var, France) and Aroua Rachid (University of Nice Sophia Antipolis, 

France) 

The objective of this article is to apply the technique of the PLS (Partial Least Squares) logistic regression to 

predict the financial distress of the French firms. This research is justified by the insufficiencies of the traditional forecasting models. The sample consists of 120 French companies for which the accounting and financial 

data were collected and a battery of 35 financial ratios was calculated over the period 1999—2003. 

The forecasts stemming from the PLS logistic regression are compared with that of the logistic regression. The 

results of the study show that the technique of the PLS logistic regression is better in terms of predictability.

Music as a Capital Asset

Rachel Soloveichik (Bureau of Economic Analysis, USA)

In 2002, I estimate that musicians and recording studios spent $7.9 billion producing original music. These 

songs were sold in CD in 2003, and will be played on the radio, on television and at live concerts for decades to 

come. Because of their long working life, the international guidelines for national accounts, System of National 

Accounts 1993, recommends that countries classify production of music and other entertainment, literary and 

artistic originals as an investment activity and then depreciate those songs over time. However, BEA does not 

capitalize this category of intangible assets at the present time. As a first step in considering the treatment of 

this category of intangibles as a fixed asset, I collected data on music production and calculate what GDP statistics would be if songs were classified as a capital asset.

To preview, my empirical results are: 1) In 2002, musicians and record studios spent $7.9 billion dollars producing recorded music; 2) The recorded music industry has been shrinking dramatically since 2000. Between 

2000 and 2007, Real GDP growth falls by 0.005% per year if recorded music is treated as a capital asset; 3) 

Original music remains valuable for decades after it is first produced. In 2002, I calculated that the aggregate 

capital value of all original music was $38 billion.

The Transportation Services Index: Its Methodology and Relationship to the Business Cycle

Kenneth Notis and Peg Young, and Long Nguyen (U.S. Department of Transportation)

BTS has been publishing TSI, a seasonally adjusted, cross modal index of the volume of transportation services 

on a monthly basic since March, 10, 2005. Since that initial entry, BTS has conducted research into how the 

TSI relates to the economy. This report provides a summary of the research conducted and results obtained 

regarding the relationship of the TSI and the economy. The history and development of the index are reviewed, 

and the methodologies for seasonal adjustment, for aggregation across modes, and for chaining are described. 

Peaks and troughs in the Index, smoothed using Spencer curve methodology, are compared to measures of the 

general business cycle, including NBER declared recessions and growth cycles defined by Zarnowitz and Ozyildirim in order to determine TSI’s potential as an indicator. 

Reliability of Price Indices to Age Expenditure Data Used in Government Measures

Sharon O’Donnell (U.S. Census Bureau) and Stuart McDonald (University of Queensland, Australia)

1. Description of research question: A number of government measures are based on expenditure data from 

national population surveys and administrative sources. Given the frequency and timing of data releases, current data from any source may not be present at the time the measure is calculated. In the absence of current 

data, data from the most recent period are aged using a price index. This paper will determine the duration of 

aging that maintains a consistent measure of the expected current expenditures. Is there a point when aging 

fails to generate a reliable measure?

2. Data sources used: Survey of Income and Program Participation (SIPP)

3. Statistical method used: Survival analysis

4. Preliminary conclusions reached: This paper characterizes conditions when aging fails to provide consistent 

results. Empirical tests are applied to two types of expenditure data, non-zero expenditure data (e.g., annual 

shelter cost) and data with zero expenditures (e.g., annual medical out of pocket expenditures and child 

care costs).
21

CONCURRENT SESSION II-C:

BAYESIAN METHODS

Record Linkage Modeling in Federal Statistical Databases

Michael Larsen (George Washington University, USA)

Record linkage, or exact file matching, consists of bringing together records in two or more files on the same 

population. Files are linked for the purposes of creating a larger database, enabling analyses that would otherwise not be possible, and counting the population. When unique, error-free identification codes are not available on both files, then record linkage can be accomplished through probabilistic methods. The U.S. Census 

Bureau uses record linkage in population undercount estimation. The National Center for Health Statistics 

uses record linkage to match surveys to the National Death Index (NDI) for studies of mortality and morbidity. 

This talk discusses advances in record linkage theory related to these efforts. The models allow estimation of 

error rates and decision making about match/nonmatch status of pairs of records. Methods of record linkage 

that enforce one-to-one matching between individuals have been implemented. Generally the files being linked 

at census and NCHS have been unduplicated, so that one-to-one matching is required. Bayesian methods that 

allow variability across blocks and incorporate one-to-one matching into statistical models have been studied. 

Advances have been made in analysis of files created through record linkage, including some accounting of 

potential matching errors. The work on record linkage has direct relevance for methods of preserving confidentiality in publicly released databases.

Generalized Maximum Likelihood Method in Linear Mixed Models with an Application in the SmallArea Estimation 

Partha Lahiri (University of Maryland, USA) and Huilin Li (National Institute of Health, USA)

Standard variance component estimation methods frequently produce zero estimates of variance components 

in the underlying linear mixed model. As a consequence, the EBLUP estimate of a small area mean reduces to 

a simple regression estimate. In this paper, we will consider an adjusted maximum residual likelihood method 

that yields strictly positive consistent estimators of the variance components. In addition, the mean squared 

error of the corresponding EBLUP estimator is asymptotically equivalent to those of the maximum likelihood 

and residual maximum likelihood estimators in the higher order asymptotic sense. However, the adjusted 

maximum residual likelihood method has an advantage over the standard methods in estimating the shrinkage 

parameters and in constructing the parametric bootstrap prediction intervals of the small area means. We shall 

illustrate our methodology using a few examples from the US federal statistical system.

Spatial-Temporal Assimilation of Fine Fraction Particulate Matter

Ana Rappold (U.S. Environmental Protection Agency) and Marco A. R. Ferreira (University of Missouri, USA)

There is a strong interest in developing methodology for assimilation of the observed data and mathematically 

modeled predictions within many scientific disciplines. In the area of air quality research we are particularly 

interested in assimilating models such as EPA’s Community Multi-scale Air Quality model (CMAQ) with the 

monitoring data to obtain maps of daily concentrations. Such maps can be used in epidemiological research 

of health effects, in the formulation of regulatory policies, and in the evaluation of regional attainment of 

regulatory polices. The goal of this paper is to present a novel model for data assimilation with emphasis on 

CMAQ model validation, computational feasibility and interpretability.

We present a Bayesian hierarchical model for data assimilation in both time and space. Joint distribution of the 

two data types is modeled through a common temporally evolving latent process. The model accommodates 

the change of support problem and handles spatially and temporally distributed additive bias associated with 

CMAQ data through a multiscale spatio-temporal model. The multiscale approach to estimating bias gives 

insight into the similarities and discrepancies of two data types at different scales of resolution through time. 

We illustrate the value of data assimilation in the imputation of daily time series across all locations, and in 

prediction over areas with sparse data collection.

The Spatio-Temporal Determinants of Charter School Site Selection in New York City

Charisse Gulosino (Brown University, USA)

This paper assesses the determining factors of charter school location in New York City using spatial econometric models. It examines observable predictors of charter school locations in neighborhoods, such as socioeconomic characteristics, nonprofit proximities, property values, and standardized test scores. The results suggest 

that an OLS model without consideration of spatial autocorrelation is likely to inflate the association between 

outcome and explanatory variables. Using spatial econometric models the paper examines the role of spatial 

autocorrelation in charter school location, in particular the effect of data varying in time as well as space for 
22

contiguous census tracts. The goal of optimizing the location of charters in the presence of both efficiency and 

equity constraints is shown using a spatio-temporal model. 

No other research has attempted to explore the determinants of charter school site selection and their implications for planning. This study hopes to fill this research gap. Major advances in statistical visualization through 

the development of new tools make it possible to enhance the quality of research that measure neighborhood 

characteristics, determine what assets are available to help improve local education and quality of life, and 

help match needs and resources. Extending the research findings to policy-makers is crucial, as they ultimately 

shape the implementation of charter school programs through legislation that can restrict, mandate, or facilitate the distribution of public resources across schools and localities.

CONCURRENT SESSION III-A:

ISSUES IN DATA CONFIDENTIALITY AND DISCLOSURE RISK

Incorporating Quality Measures in Tabular Data Protected by Perturbation Methods

Ramesh Dandekar (Energy Information Administration, USA)

Statistical agencies routinely publish analysis reports by using survey data summarized in tabular format. The 

analysis reports often provide industry specific regional and sub-regional fluctuations in the reported tabular 

values by comparing them with the data collected in previous survey cycles. Based on the information presented in the tabular data, data users often make their own inferences about the changes in the market conditions for their own immediate geographical areas. To continue to meet these requirements, newly proposed cell 

perturbation-based tabular data protection methods such as controlled tabular adjustment (Dandekar 2001), 

and a micro data level noise addition method need to have a strategy in place to ensure that the data users do 

not confuse cell perturbation error with fluctuations in published cell values attributed to other sources. To 

achieve this objective, we propose a simple mechanism to continue to safeguard data quality and to provide 

the most accurate information to data users in the tabular format protected by cell perturbation methods. Our 

proposed method with minor modifications could be used to convey the relative standard errors (RSE) associated with tabular format estimates derived from sampled survey data.

Protecting Numerical Confidential Data Using Data Shuffling: A Demonstration of Effectiveness of 

Approach and Flexibility of Delivery

Rathindra Sarathy (Oklahoma State University, USA) and Krish Muralidhar (University of Kentucky, USA)

Protecting numerical confidential data in data released by government agencies and other organizations poses 

a considerable challenge. There have been many different procedures that have been developed for this purpose. Among these procedures, Data shuffling offers the best compromise between preventing disclosure of 

information while preserving the usefulness of the data. Data shuffling is a non-parametric procedure that 

offers the advantages of both perturbation and data swapping while simultaneously minimizing disclosure risk. 

Specifically Data shuffling offers the following advantages: (1) Individual data values are not modified but are 

shuffled between records thereby maintaining the distribution of individual variables exactly, (2) All linear and 

monotonic non-linear relationships between all variables are maintained providing a high level of analytical 

value, and (3) Providing the highest possible level of security (lowest level of both identity and value disclosure 

risk). In this demonstration, we provide an extensive analysis of the analytical value and disclosure risk characteristics of Data shuffling. We demonstrate multiple modes of delivery, including spreadsheet-based, web-based 

and downloadable modes, to suit the needs of different types of users.

Vulnerability of Complementary Cell Suppression to Intruder Attack

Lawrence Cox (National Center for Health Statistics, USA)

Complementary cell suppression was the first and remains a popular method for disclosure limitation of magnitude data such as economic censuses data. We show that, when not solved in a rigorous mathematical way, 

suppression can fail to protect data, sometimes fatally. When solved properly as a mathematical programming 

problem, suppression is guaranteed to meet certain conditions related to protecting individual data, but we 

demonstrate that other vulnerabilities exist. Suppression sacrifices both confidential and nonconfidential data, 

forcing potentially significant degradation in data quality and usability. These effects are often compounded 

because mathematical relationships induced by suppression tend to produce “over-protected” solutions. To mitigate these effects, it has been suggested that the data releaser provide exact interval estimates of suppressed 

cell values. We demonstrate for two standard data sensitivity measures that, even when safe, exact intervals 

further threaten data security, in some situations completely.
23

Evaluating the Impact of Data Swapping Using Global Utility Measures

Sylvia Dohrmann, Tom Krenzke, and Shep Roey (Westat, USA) and J. Neil Russell (National Center for Education 

Statistics, USA)

The issue of whether the release of statistical data for public use may lead to identity disclosure of individual 

units is a long-standing concern. Federal and state agencies are struggling with the need to release survey data 

while protecting the confidentiality of the individuals or institutions included in these surveys. The confidentiality standards of the Institute of Education Sciences (IES) require (1) the identification and masking of sensitive 

variables and records and (2) introducing an additional measure of uncertainty with random swapping. The IES 

DataSwap software employs a controlled random swapping approach and was recently enhanced in an effort to 

improve the implementation of the second procedure. 

DataSwap includes three sets of global utility measures to help the user evaluate the impact of swapping on the 

weighted distribution of the swapped variables, selected pairwise associations, and coefficients in regression 

models of the key survey outcome variables on the swapped variables. The objective of having these measures 

built into the software is to help users select the best swapping result from several replicated runs generated 

by using different swapping targets (with all other parameters unchanged). These global utility measures are 

comparable when other software parameters are changed, such as those relating to the selection of targets and 

their swapping partners. 

In this paper we present each of the global utility measures, explain the characteristics of each, and their 

interpretation. We will further demonstrate how the measures may be used to evaluate various swapping 

scenarios and display the variability among replicated runs. We address how swapping scenarios may be modified to reduce swapping impact, and present strategies for determining the best swapped dataset for a study. 

CONCURRENT SESSION III-B:

STATISTICAL USES OF ADMINISTRATIVE DATA

Using the NHIS Linked to Mortality to Examine Social Disparities in Life Expectancy

Kimberly Lochner, Van Parsons, Nathaniel Schenker, and Elsie Pamuk (National Center for Health Statistics, USA)

Nationally representative data with good quality health, mortality and socio-demographic information are 

important for federal initiatives aimed at reducing health disparities. For example, data on race/ethnicity collected in our national vital statistics system, has shown the life expectancy of blacks to be lower than that of 

whites. Yet, there are limitations to such data for examinations of socioeconomic (SES) disparities, including 

lack of correspondence between SES indicators in the numerator and denominator, the quality of the SES information on death certificates and finally, the absence of information on family income. The linkage of population-based health surveys to death records fills a data gap in the U.S., with respect to the continued interest in 

SES disparities. Large sample sizes and survey reported SES information coupled with mortality information permits analyses not usually available for the U.S. adult population. Using the linkage of National Health Interview 

Survey (NHIS) respondents to their death records, we examined SES differences in life expectancy at age 25, 45, 

and 65. Life expectancy was found to decrease with decreasing levels of education and income. Further examination of the effect of smoking status on observed differences in life expectancy by SES as well as time trends 

will be explored. The SES and health information in the NHIS linked mortality files provide a unique data source 

for examinations of social disparities in life expectancy and their contributing causes.

An Evaluation of Data from the NCES Teacher Compensation Survey: The Relationship Between 

Teacher Compensation and Students in Poverty

Stephen Cornman and Frank Johnson (U.S. Department of Education), Steven Honegger and Amber Noel (American Institutes for Research, USA), and Lei Zhou (Macrosys Research, USA)

In response to the need for individual teacher level data, the National Center for Education Statistics (NCES) 

developed the Teacher Compensation Survey (TCS), an administrative records survey that for the first time 

collects total compensation, teacher status, and demographic data about individual teachers in multiple states. 

The long-term goal is to expand the TCS to become a national survey. 

In 2007, NCES launched the pilot TCS data collection, and seven (7) states volunteered to participate in the 

survey. In 2008, (17) volunteer states participated. The TCS has 1.29 million records representing 1.12 million 

teachers (approximately one third of the teachers in the country). One of the paramount purposes of making an 

FCSM presentation is to actively solicit additional states to participate in the survey.

The presentation discusses the experiences with the new data collection—the methodologies employed and 

problems encountered in collecting and reviewing the data. The presentation discusses reasons why only five 

out of seventeen states can provide data on health and retirement expenditures. 
24

Data on the median salaries of teachers and counts for different groupings by experience, education level, age, 

race, and gender will be presented. 

Recently school officials in urban districts such as Denver, New York, and Washington are contemplating 

“front loading” teacher compensation by increasing the salaries for new teachers. The median salaries for new 

teachers are discussed at length in light of recent calls for increasing their salaries.

The presentation demonstrates how to merge the TCS with other NCES data files, such as the School 

Universe Survey. An OLS regression could be run to explore the association between teachers’ base salaries 

and students in poverty and geographic location, experience and education, while controlling for teachers’ 

personal characteristics.

Using Administrative Data in the 2007 Census of Agriculture Edit

Virginia Harris, Dan Beckler, and Jeffrey Beranek (National Agricultural Statistics Service, USA)

The National Agricultural Statistics Service (NASS) collects information about U. S. Department of Agriculture 

farm program payments received by agricultural establishments on its quinquennial Census of Agriculture, 

in part to help determine whether or not the establishments qualify as farms. However, past experience has 

shown that farm program payments tend to be underreported by recipients on the census. In an effort to 

enhance the edit and improve the quality of the published totals for the 2007 Census of Agriculture, NASS 

attempted to use administrative data for farm program payments from another USDA agency, the Farm Service 

Agency (FSA). Difficulties in matching categories of payments recorded by FSA to those collected by NASS and 

matching FSA recipients of payments to NASS farms resulted in NASS using the data only as an indicator of 

whether a payment was received. Even with this limited use, published totals for farm program payments came 

closer to published amounts paid out by FSA. Future work will be focused on achieving a more robust mapping 

of FSA payments to NASS farms in order to further improve the edit and published numbers, and in the hope 

that, ultimately, response burden on the census could be reduced by using this data in place of collecting farm 

program payment data on the Census of Agriculture form.

CONCURRENT SESSION III-C:

EVALUATING AND IMPROVING SAMPLE COVERAGE AND DATA COLLECTION PROCEDURES

Census Population Coverage Error: Results and Methods

Colleen Clark (Statistics Canada)

Throughout the census-taking process at Statistics Canada, every effort is made to ensure high-quality results. 

The resulting data, however, are subject to a certain degree of inaccuracy. Population coverage error is the 

extent to which census data both excludes persons who should have been enumerated and includes persons 

who were enumerated more than once. These two errors, census population undercoverage and census population overcoverage, offset each other. Census net population undercoverage quantifies the net number of 

persons missed by the census. Net undercoverage affects both census counts and census distributions. Both 

undercoverage and overcoverage increased in the 2006 Census. Many of the demographic trends continue but 

there are some new patterns, especially for overcoverage. Estimates of 2006 Census population coverage error 

will be presented including geographic and demographic distributions. A description of the study that measures 

population undercoverage with a focus on the important improvements realized for the 2006 version will be 

included. The new study that measures population overcoverage will also be discussed.

Assessment of Potential Bias in Telephone Survey Estimates Due to Noncontact and 

Noncoverage Where Respondents Primarily Use Wireless Telephones or Do Not 

Have Landline Telephones

Meena Khare and Abera Wouhib (National Center for Health Statistics, USA)

Quality of survey estimates depends on coverage of the target population, response rates, and measurement 

errors. Landline telephone surveys exclude households that do not have landline telephones including wirelessonly households. Among children under 18 years of age, the prevalence of living in wireless-only households 

increased from 3.7% in 2003 (January–June) to 16% in 2008 (July–December) according to the National Health 

Interview Survey (NHIS) and accounted for more than 83% of the noncoverage due to substitution. In addition, 

households with mixed telephone services where members primarily use wireless telephones for most calls 

(wireless-mostly) may not answer their landline telephones and result in noncontact. In 2007, wireless-mostly 

and wireless-only households accounted for 15% and 13% of children under 18 years of age, respectively. In the 

U.S., the prevalence of wireless-only and wireless-mostly population has been consistently increasing and may 

affect validity of estimates from telephone surveys such as the National Immunization Survey (NIS). Household 
25

telephone status and interview data for children under 18 years of age from the NHIS-Sample-Child file are used 

to compare alternative methods to adjust sampling weights and evaluate potential bias in weighted estimates. 

We use interview data to compare characteristics of children and estimates of selected outcome measures that 

are associated with health conditions and vaccinations. The purpose of this paper is to evaluate potential bias 

in RDD survey estimates due to noncoverage of wireless-only and noncontact with wireless-mostly households.

Developing a Common Mode-Independent Approach for Evaluating Interview Quality and 

Interviewer Performance

Howard Speizer, Susan Kinsey, Richard Heman-Ackah, and M. Rita Thissen (RTI International, USA)

RTI is developing an interview quality monitoring evaluation system based on audio recordings of the survey 

interview to be used across all interviewer administered survey modes. In-person and telephone interviewers 

are evaluated by similar quality metrics and their performance is tracked across projects. The interviewer evaluation, feedback and coaching processes have been standardized, helping make the quality monitoring process 

more efficient. The system provides an easy to navigate front-end to all recorded survey interactions for use 

by quality supervisors, project staff and clients. This paper examines the design of this new system including 

the strategies used to standardize evaluation metrics across survey modes, store and analyze digital files and 

develop interviewer-, and survey item-, level quality evaluation databases. We discuss the system’s fit across a 

wide spectrum of interview quality monitoring processes. The paper concludes by contrasting the benefits and 

drawbacks of using survey interaction recordings to more traditional efforts for monitoring interview quality.

The Influence of State-Specific, Programmatic Characteristics in the Modeling of Medicaid 

Undercount: A Record–Check Study of the 2001 Current Population Survey Annual Social and 

Economic Supplement (CPS ASEC)

Matthew Stevens (U.S. Census Bureau)

Much research has been conducted regarding underreporting of benefit receipt on population surveys. Specifically, much work has been devoted to measuring Medicaid undercount on the Current Population Survey Annual 

Social and Economic Supplement (CPS ASEC). Previous studies have revealed estimates of false-negative reporting rates varying from 10–36%. As one of the primary indicators of Medicaid benefit receipt, measurement error 

within this estimate will not only bias survey results, but subsequently bias the program evaluation, policy 

development, and funding decisions informed by the CPS ASEC. 

Upon gaining estimates of the size of underreporting, the next round of research focused on seeking to explain 

why Medicaid recipients provide erroneous reports. Record check studies conducted by the Census Bureau 

revealed significant associations between respondent-level characteristics (such as recency and intensity of 

coverage, as well as household income and relationship to person being reported for) and underreporting.

Using the same administrative records database utilized in previous studies, this study searches for the root 

of state-level differences in underreporting that still exist after controlling for respondent-level characteristics. 

Specifically, how effectively can the characteristics of state-run Medicaid programs (such as eligibility rules, 

policy on premiums, and portrayal to the public) explain the likelihood of recipients within said state to failto-report benefits? Examination of Medicaid programmatic literature enabled creation of a series of indicator 

variables to represent these state programmatic differences within logistic regression models. Results suggest 

that while many characteristics of the respondent effect Medicaid reporting, characteristics of the state Medicaid program itself may equally impact reporting propensity, and that such sources of measurement error may 

be easier to remedy on the CPS ASEC than previously identified, respondent specific sources.

CONCURRENT SESSION IV-A:

MODELING OF DATA

Project for the Evaluation of the Economic and Safety Situation of the Street Transportation Using a 

Cost-Effective Methodology: The Case of Cameroon

Ramses Zang, Sebastien Djienouassi, and Annie Laurence Mapondjou (Centre for Studies and Research on 

International Development [CERDI], France)

This paper proposes a cost-effective project for the monitoring and evaluation of the street transportation and 

safety situation. It provides a sampling method and three data sources for this purpose. Partly due to a shortage of resources and to the amount of informal activities, Cameroon is facing a tremendous shortage in road 

safety and transportation information. There is an increasing need for quality information, although some 

non-rigorous methodologies have been previously processed. A cost-effective methodology that allows a better 

handling of the sector has been proposed and successfully tried. This methodology considers axes as a net-
26

work where crossroads are data collection areas and one-way road sections, the first level of statistical units. 

These specifications make it possible to reduce some costs by halving the number of respondents for a given 

sample size. For Cameroon, this methodology will be using the set of cities of at least 20,000 inhabitants from 

which 16 road sections corresponding to 5 crossroads or data collection areas are specified. Companies would 

be selected proportionately to the number of companies exploiting a specific road section at the last level of 

statistical units. Three populations for the interurban formal traffic are targeted here: representatives of units of 

production, public transportation drivers and a defined sample of one-way journey progresses. Enterprises will 

be mainly answering questions regarding the tendencies of the sector, the costs of production and the accident 

risk management. Public transporters will be providing for each way of their usual traffic line, data on their personal, professional and load characteristics and the situation of infrastructure and official controls. The one-way 

journey progresses will be providing data on the state of vehicles and drivers at the departure point and the 

mood and professional behavior of drivers during the ride.

More Efficient Local Polynomial Regression on Random-Effects Panel Data Models

Ke Yang (University of Hartford, USA) 

There has been substantial interest recently on improving efficiency of kernel regression methods for panel 

data. Previous research show that for local polynomial regression, correctly specifying the variance-covariance 

structure actually hurts estimation efficiency and the best strategy is to ignore the correlation structure entirely. 

In this paper a novel local polynomial estimator is proposed for a random effects panel data model. The proposed estimator achieves higher estimation efficiency by accounting for the dependence structure existing in 

the data. Asymptotically, the proposed estimator outperforms the working independence estimator uniformly 

up to the first order. Finite-sample performance of the proposed procedure is assessed through Monte Carlo 

simulation studies. It shows that the gains in efficiency in the presence of within-subjection correlation can be 

significant in small samples.

Assessing Immigrant Naturalization: Longitudinal Research Findings and Challenges

Karen Woodrow-Lafield (Unaffiliated Scholar, USA)

Linked administrative records on immigrants and naturalizations constitute a dataset for modeling naturalization over duration in lawful permanent residence with covariates related to admission characteristics, reflecting 

social capital and human capital. This presentation reviews several recommendations and research programs 

in establishing such data for researchers, beginning with expert recommendations in the early 1980s that led 

to release of two public use datasets in the early 1990s. Through the Immigration-to-Naturalization Project, the 

linked dataset was created for 1978–1991 immigrants followed until 1996. Several empirical studies show the 

importance of origin country in explaining differential timing of naturalization and varying influences of human 

and social capital. In particular, key studies show that understanding naturalization outcomes involves not only 

considering observed heterogeneity on demographic, origin, and admission characteristics but also accounting for unobserved heterogeneity on reception contexts, socioeconomic assimilation, and orientation to origin 

communities. The discussion reviews improvements in federal statistics on immigrant naturalization, addresses 

the challenge of researcher access, and sets forth a paradigm for statistical policy on immigrant naturalization 

as an aspect of immigrant incorporation. Beyond analyses involving either administrative records or surveys, 

more research is needed about immigrant decision-making to reside in the United States, to become naturalized 

citizens, and in navigating the application process as well as taking English classes and learning U.S. civics 

and history.

First-Line Supervisors: An Analysis of Job Satisfaction and Job Proficiency

Kelly Maguire and Charles Griffiths (U.S. Environmental Protection Agency)

Employer surveys provide valuable information to organizations that are seeking to improve the effectiveness 

of their workforce. Typically, however, these surveys are analyzed using basic tools, such as descriptive statistics or simple correlation measures. Such methodology can inform basic relationships, but can mask the causal 

relationship between variables.

In 2006 the U.S. EPA conducted a survey of 444 first-line supervisors (76% response rate) to better understand 

their strengths and challenges. Participants provided responses to questions regarding satisfaction, proficiency, 

and supervisory responsibilities using a four-point likert scale, and followed up with factors that may have 

negatively or positively impacted the score.

We analyze responses to questions about job satisfaction, job proficiency, and the utility of challenging assignments using an ordered logit model. Independent variables include the demographic profile and responses to 

possible policy interventions. We assess the importance of the regressors by examining their sign and significance, but the magnitude of the coefficients is more difficult to interpret. We estimate the predicted prob-
27

abilities using the mean value for all independent variables, and then assess marginal effects by changing the 

independent variables by one, according to the likert scale. We also use contingency table, with policy interventions based on statistical significance and the gamma term.

This technique allows us to examine the impact of specific policy options, such as decreasing the number of 

direct report employees by 1 or adding team leaders to the management structure. Results show that senior 

level support is critical; training, development and experiential learning opportunities are important; and mentoring is valuable. These results are not apparent by viewing descriptive statistics alone. 

This innovative application of these well-known tools can serve to enhance the way in which employer surveys 

or program evaluation efforts are conducted such that key causal relationships are identified in a rigorous 

manner.

CONCURRENT SESSION IV-B:

DEFINING THE GREEN ECONOMY; AND ISSUES IN U.S. OCCUPATIONAL CLASSIFICATION SYSTEMS

Occupational Classification Systems: Analyzing the 2010 SOC Revision

Alissa Emmel, Theresa Cosca, and Anne Louise Marshall (Bureau of Labor Statistics, USA)

The Standard Occupational Classification (SOC) system, used by all Federal statistical agencies to collect and 

report occupational data, has been revised in a 2010 edition. The 2010 SOC replaces the previous 2000 edition, 

and will serve data users by accommodating changes in the workforce and facilitating accurate data collection. 

Revisions are necessitated by changing technology and business processes or adoption of new tasks and training for existing occupations that decrease the relevancy of any classification scheme over time.

This paper discusses how classifying each worker to a unique set of tasks promotes consistency throughout 

the 840 occupations and consistency in coding over the period of use of the 2010 SOC. It details the process 

used and the issues the SOCPC considered as well as the application of the classification theory and values to 

changes in the principles, coding guidelines, classification structure, or occupational definitions.

The SOC Policy Committee (SOCPC), convened by the Office of Management and Budget and chaired by the 

Bureau of Labor Statistics, elected to use a set of classification principles and a task-based system that focuses 

on coding workers according to shared primary tasks rather than according to job titles or skill and education level. The SOCPC considered, but ultimately rejected, alternatives to the task-based system, such as 

licensing and occupational and educational credentials. With the goal of encouraging participation, the SOCPC 

approached the revision by soliciting recommendations based on specific criteria, determining primary tasks 

and specialized functions, and updating the classification structure in a way that allows opportunity for future 

revision. The paper remarks on the International Labor Organization’s update of the International Standard 

Classification of Occupations (ISCO) for 2008, which the SOCPC considered in making its final decisions.

Defining the Green Economy

Richard Clayton and George Stamas (Bureau of Labor Statistics, USA)

The Green Economy is one of the hotly debated topics in current economic policy. Measuring and monitoring 

green industries and green collar occupations requires a definition that is clear, objective and widely accepted. 

However, there is no such definition at this time. A review of several existing studies indicates that there is no 

strong agreement on the boundaries of the green economy and that some of the studies vary considerably in 

the scope of industries and occupations covered. 

A suitable definition must be clearly defined and also be collectible and thus readily understood by business 

respondents. For example, business respondents will have to report on the characteristics (inputs, processes 

and outputs) that include or exclude them as green and the number of employees that fit the definition. Further, not all construction businesses are green, but some are when directly involved in building and installing 

wind towers, for example. Similarly, some electricians are green while connecting solar panels but not while 

wiring a new home. Many methodological problems face this new classification and collection challenge. 

The Bureau of Labor Statistics (BLS) is leading an effort to synthesize a range of existing research reports and 

studies and to consult with other federal and state agencies and other parties to derive a suitable definition, or 

definitions, for ongoing measurement. This paper describes the approach, process and progress to date toward 

this goal; and provides the basis for consistent measurement across a range of economic statistics programs. 

We will present prototype definitions, and analysis of selected microdata illustrating issues and approaches to 

addressing them.
28

Updated Unified Category System for 1960–2000 Census Occupations

Peter Meyer (Bureau of Labor Statistics, USA)

An earlier paper proposed a consistent category system for occupations in U.S. Census of Population data 

and CPS data from 1960 to 2000. The harmonized categories are based mainly on the 1990 Census occupation definitions. The IPUMS.org project adopted this category system for its “occ1990” variable which can be 

downloaded with population census data. This paper corrects and updates the assignment of respondents to 

occupations based on feedback from users and further study. In the previous assignment, persons in a census 

occupation category in each year were assigned all together to one category in the standardized category 

system. 

But it is possible to do a little better, based on “dual-coded” data sets in which a subset of respondents were 

categorized into two consecutive systems such as the 1990 and 2000 systems. This paper describes a revised 

assignment scheme, in which respondents in a single Census occupation category are sometimes assigned into 

two different categories in the harmonized system, based on other information such as the industry of work. 

This involves much more detail but is more accurate. The new method slightly reduces the number of empty 

category-years in the unified system, and the jumps in the aggregate statistics of income and occupation-size 

for the harmonized occupation categories.

CONCURRENT SESSION IV-C:

IMPROVING COVERAGE AND RESPONSE RATES

SIPP 2008 Incentive Analysis

Stephen Clark and Stephen Mack (U.S. Census Bureau)

In an effort to determine an effective incentive for increasing response rates in the Survey of Income and Program Participation (SIPP), we implemented a randomized experiment in the first wave of the 2008 SIPP Panel. To 

contain the costs for the experiment, 50% of the sample received no incentive. Twenty five percent were sent 

a $20 debit card with their advance letter introducing them to the survey. The remaining 25% were eligible for 

a $40 debit card conditional on their participation in the survey. The field staffs, in the regional offices, were 

given enough $40 debit cards to cover 15% of their workload. We interviewed the first wave of SIPP from September through December 2008. For each month and the complete first wave, we computed and compared the 

response rates for overall effectiveness for each of the individual incentive groups, their associated errors, the 

household refusal rates, and their associated errors.

Adaptive Contact Strategies in a Telephone Survey

James Wagner (University of Michigan, USA)

In attempting to contact households in a telephone survey, it is necessary to determine the timing of each call. 

Often, average “best” times to call are used in order to place the first call(s). The timing of subsequent calls is 

then governed by very general rules, for example, that attempts should be made at different times of day and 

days of the week. In fact, we are not interested in the average best time to call. When faced with the decision 

about when to time the next call for a case, the average is simply an estimate that does not use any information 

about the specific household. What we really want to know is the household-specific probability that someone 

will be at home and answer the telephone at different times of the day and days of the week. In this paper, 

multi-level models are used to estimate the best time to attempt contact. These models are updated daily in 

order to guide the decision about timing the next call. The predictions of these models become more specific to 

the household as more attempts are made. The method is evaluated experimentally.

Characteristics of Parent Respondents in the National Household Education Surveys

Stacey Bielick and Cameron McPhee (American Institutes for Research, USA)

This methodological paper will examine demographic characteristics of parent respondents in the National 

Household Education Surveys (NHES), an RDD telephone survey sponsored by the National Center for Education 

Statistics (NCES). The report will examine changes in respondent characteristics in each administration of the 

NHES since its inception in 1991 to the most recent survey in 2007, covering nine administrations with continually declining response rates. The report will also examine differences in key estimates by respondent characteristics and for cases where the respondent was different in the initial household screener and the extended 

topical survey.

Respondents to the NHES are the parent or adult in the household who is most knowledgeable about the care 

and education of the sampled children. The analysis will examine parents of different child populations, including children not yet enrolled in school, children enrolled in school, and home schooled children. 
29

The respondent characteristics include sex, age, relationship to the child, labor force participation, educational 

attainment, and marital status. The key survey items include childcare participation, receipt of public assistance, school readiness, after-school activities, school choice, and parent involvement in educational activities 

at home and at school.

The results from the analysis will inform future design work for the NHES program as NCES looks to switch the 

design from a phone to a mail survey. The analysis will also provide general information about parent respondents for NCES and other statistical agencies and organizations that conduct survey research with parents 

about their children.

Measuring the Effects of Reminder Postcards in the Occupational Employment Statistics Survey

Carrie Jones (Bureau of Labor Statistics, USA)

The Occupational Employment Statistics (OES) Survey is a voluntary establishment survey that produces occupational employment and wage estimates for the U.S., each State, and select U.S. Territories and metropolitan 

areas within States. The survey is conducted primarily via mail in cooperation with State Employment Security 

Agencies by the Bureau of Labor Statistics. The OES sample is 1.2 million establishments collected over three 

years in semi-annual panels. Each panel has an initial mailing followed by three follow-up mailings at fourweek intervals. In May of 2007, the United States Postal Service (USPS) initiated a substantial postage increase 

and was also granted authority to automatically increase postage rates annually. OES absorbed a 30 percent 

postage increase. Concurrently, examination of data files indicated that many respondents received follow-up 

solicitation packages after they had sent in their completed survey. Faced with increasing postage costs, many 

unnecessary “overlap” mailings, and a shrinking budget OES decided to test using reminder postcards during 

the first follow-up mailings in November 2007 and again in May 2008. The November test featured substitution 

of a postcard for a survey packet. The May test featured a postcard in addition to a survey packet with the idea 

that immediate increases in response would reduce the number of follow-up mailings. Six States were randomly selected for a two-panel test and the samples for each State were split into two groups: a test group that 

received the postcard and a control group that received a regular mailing. Logistic regression is used to evaluate the effect of the postcard experiment on response rates, and a cost analysis is presented, as well as results 

from a qualitative assessment questionnaire completed by each test State.

TECHNICAL DEMONSTRATIONS

DEMONSTRATION OF SOFTWARE TOOLS FOR COLLABORATION, MATCHING, AND ANALYZING QUESTION 

QUALITY AND PARADATA

Q-Notes: Development and Use of Analysis Software for Cognitive Interviews to Examine Survey 

Question Comparability

Kristen Miller, Aaron Maitland, Heather Ridolfo, Stephanie Willson, and Justin Mezetin (National Center for 

Health Statistics, USA)

Recent work has shown that cognitive interviewing studies can provide essential information regarding the 

comparability of survey questions, specifically, how respondents interpret and process questions and whether 

particular sub-populations or groups may process questions differently from others. To achieve this goal, 

however, studies must be based on empirical evidence and systematically analyzed across interviews and subpopulations a process which can yield a massive amount of qualitative data across numerous countries and in 

multiple languages. To be sure, one of the biggest challenges for comparative, multinational cognitive testing 

is data management, that is, the organization and reduction of cognitive interview data such that it can be analyzed systematically. This paper will describe software that was specifically developed by the National Center 

for Health Statistics to analyze cognitive interviews in this capacity. To illustrate the software’s use, the paper 

will draw from NCHS evaluation projects. 

A Demonstration of FI Tracker: A Tool for Displaying and Monitoring Survey Interview Paradata

Joe Murphy, Susan Myers, and Barbara Bibb (RTI International, USA)

Survey organizations typically have more data on the survey process than they can use without appropriate tools for review. Paradata at the contact attempt, household, and telephone or field interviewer (FI) levels 

are very useful for identifying data collection concerns that should be addressed to maximize efficiency and 

data quality. But the source of these data, the record-of-calls, is usually difficult to analyze in its raw format. 

This demonstration presents a tool called the FI Tracker that compiles raw record-of-calls data using SAS and 

Excel to create an interactive chart for graphic display. The FI Tracker displays call history data including the 

date, time, case ID, interviewer ID, disposition code, and free-text comments, for every call made to a sample 
30

member. With the report, a reviewer can trace the pattern of calls made by an interviewer to a single household on a single day, get an overall picture of the progress of the field activity or make comparisons across 

interviewers, regions, and types of sample members. For instance, a reviewer can easily and quickly identify 

which interviewers may be making less than optimal call patterns by visiting the same household multiple 

times during a period of hours or at the same time each day resulting in chronic noncontact. Having identified 

such instances, supervisors can work with interviewers to improve their call strategies and ultimately improve 

survey efficiency and the likelihood of successful completion. For this demonstration, we will use de-identified 

data from the 2008 wave of the Community Advantage Panel Survey, a longitudinal study of homeowners 

participating in the Community Advantage Program plus a sample of renters. During this demonstration, we will 

walk through the process of importing the raw paradata into the FI Tracker tool, provide examples of its utility, 

and discuss the value of its implementation.

Statipedia—A Platform for Collaboration Across Statistical Agencies

Peter Meyer and James Buszuwski (Bureau of Labor Statistics, USA)

This presentation outlines a design for a Web 2.0 platform for official statistical agencies to share common 

software tools, methodology, and discussion of issues related to their work.

In the decentralized U.S. statistical system, many separate agencies produce official statistics. Their separate 

computer networks are secured against outside intrusion and which have made the users across agencies inaccessible to one another. This hinders some natural collaboration and discovery between specialists with similar 

work assignments or research agendas in different agencies. Statipedia would facilitate their work by creating 

secure workspaces with wikis and source code control systems.

We propose a user interface similar to the Intellipedia, an existing set of web tools that streamlines communications among various agencies in the U.S. intelligence community. It has a spectrum of joint wikis and blogs 

across the agencies, instant messaging, web page tagging, and the storage of videos. The tools in Statipedia 

must be usable for statistical work, so they should include features like LaTeX and footnotes available in the 

wiki software at scholarpedia.org, openwetware.org, and wikipedia, which are appropriate for statistical 

definitions. Statipedia should include a source code control system, and a specialized search engine. We show 

how these capabilities would be useful for normal statistical agency work designing seasonal adjustment, price 

index, and productivity calculations.

When possible Statipedia should incorporate tools for statistical programming languages; organization charts; 

geographical information systems; access to high speed computing services; shared bibliographies; discussion 

boards; software product evaluations; and tools to monitor blog references to the agencies or the data they 

produce. We discuss how costs and risks can be lowered by offering a unified tools platform across the agencies and allowing them to choose whether to use it or not.

Demonstration of Census Coverage Measurement Clerical Matching Software

Tamara Adams, Susanne Johnson, Lee Goldberg, and Paul Clemm (U.S. Census Bureau)

As part of the 2010 Census, the U.S. Census Bureau and Gunnison Consulting Group, Inc. have developed a 

series of clerical matching software packages to conduct computer-assisted clerical matching of both addresses 

and persons between the census and the independent enumeration as part of the Census Coverage Measurement. These systems include functionality to allow clerical matchers to review records with the results of 

computer matching, filter records, conduct duplicate searches throughout the country, link and unlink records, 

and view other details about each record. In addition, each system includes a quality assurance component for 

a dependent review of each unit of work by a higher-level clerical matcher to ensure a prespecified outgoing 

quality level. These systems, based on the 2000 coverage measurement matching systems, have been further 

developed and expanded to accommodate our program’s current goals.

CONCURRENT SESSION V-A:

SOCIAL SECURITY ADMINISTRATION USES OF ADMINISTRATIVE RECORDS LINKED WITH SURVEY DATA

Pension Coverage and Pension Types: Using W-2 Tax Records to Correct SIPP Survey Reports

Howard Iams and Irena Dushi (Social Security Administration, USA)

Pension income traditionally has been one pillar of retirement income in the United States. The type of pension has been shifting over the past 30 years from defined benefit pension plans (DB) to defined contribution 

pension plans (DC). In traditional DB pension plans the employer bears the risk of providing retirement income 

which is determined based on years of service and final salary levels. In DC plans, often known as 401(k) 

type plans, the employee bears all the risks and retirement income is determined by the amount that both the 
31

employee and the employer contributed over time and their investment return in the financial market. The 

Census Bureau’s Survey of Income and Program Participation (SIPP) is a major source of information on participation and the type of pension plan held by the labor force. Using the 1996 and 2004 panels of SIPP, this 

analysis will assess the extent of pension participation by pension type over the period. It also will examine 

whether using W-2 tax record data on tax-deferred earnings contributions to DC accounts would improve the 

SIPP estimates on DC pension participation. Preliminary analysis indicates that pension participation is about 10 

percentage points higher when the survey data is supplemented by W-2 tax record data. It also finds that, when 

survey data is supplemented by W-2 tax record data, about 6 percentage points more workers have both types 

of plans (DB and DC). 

Measurement Error in Estimates of the Participation Rate in Means–Tested Programs: The Case of 

the U.S. Supplemental Security Income Program for the Elderly

Alexander Strand, Kalman Rupp, and Paul Davies (Social Security Administration, USA)

The Survey of Income and Program Participation (SIPP) has the data elements necessary to estimate financial 

eligibility for the Supplemental Security Income (SSI) program, namely income, resources and family relationships. These data elements, however, may contain measurement error that leads to a bias in the estimated 

SSI participation rate for the elderly. Replacing some of the data elements with corresponding administrative 

data increases the estimated participation rate by more than ten percentage points. The increase is partly due 

to replacing self-reported SSI payment receipt with administrative measures, but also due to replacing selfreported Social Security income with administrative payment amounts. The two sources of administrative data 

each have an impact on the participation rate among the elderly. Moreover, the impact is increasing over time. 

This suggests that the SSI-eligible population is changing over time or that the SIPP has diminishing ability to 

measure SSI eligibility. In addition to the biases uncovered by use of administrative data, there may be biases 

resulting from measurement error in variables for which there is no administrative counterpart, most notably 

the measures of asset holdings. We conduct sensitivity analyses gauging the extent of these biases.

Using Matched Survey and Administrative Data to Estimate Eligibility for the Medicare Part D 

Program Low Income Subsidy

Lynn Karoly, Erik Meijer, and Pierre-Carl Michaud (Rand Corporation, USA)

The 2003 Medicare Prescription Drug Improvement and Modernization added a new prescription drug benefit 

to the Medicare program known as Part D (prescription drug coverage), as well as the Low-Income Subsidy (LIS) 

program to provide “extra help” with premiums, deductibles, and copayments for Medicare Part D beneficiaries 

with low income and limited assets. The goal of this paper is to estimate the size of the LIS-eligible population as of 2006 using matched survey and administrative data. In particular, we employ individual-level data 

from the Survey of Income and Program Participation (SIPP) and the Health and Retirement Study (HRS) to cover 

the potentially LIS-eligible noninstitutionalized and institutionalized populations of all ages. The survey data 

are matched to Social Security Administration (SSA) administrative data to improve on potentially error-ridden 

survey measures of income components (e.g., earnings and beneficiary payments from Supplemental Security 

Income and Old Age, Survivors, and Disability Insurance) and program participation (e.g., participation in Medicare or a Medicaid/Medicare Savings program). The administrative data include the Master Beneficiary Record/

Payment History Update System, the Master Earnings File, and the Supplemental Security Record. The survey 

data are the source of information on asset components, as well as the income components (e.g., private pensions) and individual characteristics (e.g., health status) not covered in the administrative data. Our baseline 

estimate, based on the matched data, is that about 12 million individuals were potentially eligible for the LIS as 

of 2006. A sensitivity analysis indicates that the use of administrative data has a relatively small effect on the 

estimates but does suggest that measurement error is important to account for. The estimate of the size of the 

LIS-eligible population is more sensitive to the relative weight we place on the two survey data sources, rather 

than the choice of methods we apply to either data source.

CONCURRENT SESSION V-B:

STATISTICAL WEIGHTING AND EDITING

Census Editing and the Art of Motorcycle Maintenance

Michael Levin (Harvard University, USA)

When a country takes a population and housing census, it makes an implicit contract with its residents to 

provide the highest quality data for planning and policy formation. The census enumeration provides compatible information about 98 percent of the time. Those data could be used directly for public and private sector 

purposes. The other two percent consists of either invalid single entries or inconsistent responses for two or 
32

more items: females 80 years old with 3-year old children, thatch walls with concrete roof, etc. We edit the data 

to make them more aesthetically pleasing and the resulting tables completely consistent. No edit can really 

improve the quality of the collected data, so all our efforts go towards doing as little damage as possible. In 

this paper I develop rules to assign the order of edits and order to the edits. We also “correct” or change items 

out of range or inconsistent.

The UN Principles and Recommendations focus on about 30 population items (demographic, social, education, 

and economic) and about 30 housing items. I recently completed the 2010 Census update to the previously 

developed United Nations Editing Handbook for the 2000 Census Principles and Recommendations. My paper 

looks at state-of-the-art general editing “rules and resolutions. The recently released Census and Survey Processing System (CSPro) 4.0 demonstrates both the specifications and programs needed for prototype edits. The 

present paper uses general and specific examples from recent censuses of Fiji, Ethiopia, Lesotho, and others to 

illustrate the suggested rules.

Re-Weighting of CPS Data

Maryan Cammarata (U.S. Census Bureau)

When conducting record linkage research projects using survey data, the first step is creating a unique identifier on each file. The Census Bureau assigns these unique linking ids to records using the Person Identification 

Validation System (PVS). The unique identifiers are called Protected Identification Keys (PIK). Not all records validate and receive PIK through this process; records without PIK cannot be used for linkage and research. Using 

the Current Population Survey Annual Social and Economic Supplement (CPS ASEC), this paper discusses the 

prevalence of records without PIK and explores methods to re-weight the CPS ASEC records with validated PIK. 

The CPS ASEC data can be re-weighted using ratio adjustments to distribute the weights of cases without PIK 

to those with PIK, but some bias undoubtedly remains. Modeling the probability of a record having a PIK using 

a logistic regression will potentially compensate for the remaining bias. The resulting logit coefficients can be 

used to re-weight the CPS data. Model comparisons will be performed to compare the regression-adjusted and 

ratio-adjusted weights.

Comparison of Retail Drug Prices Obtained in the MEPS and Commercial Claims Data: Implications 

for Price Editing Rules and Data Quality

Marc Zodet, Steven Hill, and G. Edward Miller (Agency for Healthcare Research and Quality, USA)

The Medical Expenditure Panel Survey (MEPS) is a national probability sample survey designed to provide 

nationally representative estimates of health care use, expenditures, and insurance coverage for the U.S. civilian noninstitutionalized population. An important component of MEPS is the prescribed medicines (PMED) use 

and expenditures data. One quality control check in the PMED editing process involves an examination of the 

ratio of the retail unit price (RUP) paid for each drug to its average wholesale unit price (AWUP). We refer to this 

ratio as the PRATIO. Drug price outliers are identified as those whose PRATIO is outside of a specified range 

(i.e., the RUP is too far from the AWUP). To correct potential errors in reporting by pharmacies, drug purchases 

with outlier RUPs are edited by adjusting either the reported prices or quantities so that the edited RUP=AWUP. 

This research proceeds in three major steps: 1) determine whether the editing rules, which were originally 

implemented for the 1996 MEPS data, are still valid for 2006 and future years of MEPS data; 2) if not, provide 

guidance for modifying the editing; and 3) examine the impact of any proposed rule modifications on the distribution of retail prices. Specifically, the 2006 MEPS PMED data are compared with prescription medicine claims 

for those with private insurance coverage through large employers from the same year. Data for this research 

come from the 2006 MEPS PMED production file and from the 2006 MarketScan Outpatient Drug Claims file. 

PRATIOS were derived for drug purchases from the MEPS among persons with private insurance and for a 10% 

sample of events from the MarketScan file. Distributions of PRATIO for MEPS and MarketScan were compared for 

all drugs, and by drug patent status. These distributions reveal a clear need to revise the rules for editing the 

MEPS PMED prices.

Different Approaches for Non-Response Adjustments to Statistical Weights in the Continuous 

NHANES (2003-04)

Lisa Mirel, Vicki Burt, Lester Curtin, and Cindy Zhang (National Center for Health Statistics, USA)

It is possible to use different weighting methods to analyze National Health and Nutrition Examination Survey 

(NHANES) data. Typically when we re-weight NHANES data we use a weighting class approach of predefined 

post stratification cells. However, there are model-based alternatives, which allow for more flexibility in calculating the statistical weights. In this report, using NHANES data from 2003–2006, we illustrate alternative 

methods to calculate statistical weights using both a weighting class approach and a model based approach 

to account for unit non-response. The purpose of non-response adjustments to statistical weights is to reduce 

the bias arising from the fact that non-respondents may be different from those who responded to the survey. 
33

We used the WTADJUST procedure in SUDAAN version 10.0 to calculate weights using different approaches. We 

assessed the different weighting approaches using examples with the environmental subsample data and the 

dietary recall data from NHANES. We compared summary statistics derived from the different methods. We compared the distributions of the weights and assessed the estimated design effects due to the differential weighting. We assessed the differences between the procedures and evaluated the implications for analysis results. 

The different weighting approaches varied given the type of analysis and the degree of unit non response. 

Therefore, it is important to assess different weighting approaches when analyzing data with non-response.

CONCURRENT SESSION V-C:

IMPROVEMENTS IN SAMPLE DESIGN AND SURVEY ESTIMATION

An Evaluation of the Use of the American Community Survey as the Sampling Frame for the 

National Immunization Survey (NIS)

James Singleton and Philip Smith (Centers for Disease Control and Prevention, USA) and Andrea Piani and Christopher Stringer (U.S. Census Bureau)

Substantial telephone screening is required to identify the 3–4% of households with children aged 19–35 

months for the National Immunization Survey (NIS). One intended use of the American Community Survey (ACS) 

as a sampling frame is the identification of more difficult to reach populations. Thus, the Centers for Disease 

Control and Prevention and the U.S. Census Bureau partnered to conduct an evaluation study of the NIS using 

the ACS as the sampling frame. The evaluation study uses households that participated in the ACS and reported 

having children in the NIS target population. The ACS provides a more complete sampling frame than the current NIS list-assisted landline sampling frame, by including non-landline and cell-only households. To achieve 

the sample size requirements for the evaluation study, an alternative sampling frame was used to supplement 

the ACS sample in some geographic strata. Data collection in the current NIS design consists of two phases. 

First, households are contacted by telephone to identify those with age-eligible children and parents/guardians 

are asked to give permission to contact their vaccination providers. Second, vaccination providers are mailed 

a survey to determine vaccination status. Provider-reported vaccination histories are used to determine vaccination status. In this evaluation study, the household component has been expanded to include a multi-mode 

approach by adding an in-person follow-up for households without telephone numbers and those who refused 

to participate via telephone (landline or cell phone). This presentation explains the need to explore new design 

options for the NIS, considers the potential this new methodology has to improve coverage of age-eligible children, and discusses the special circumstances surrounding privacy and protection of the confidentiality of children’s identities in terms of Title 13 and the Privacy Act. Preliminary results of the household operation will be 

presented, as well as available findings from the provider component.

NIS Evaluation Study

Kimball Jonas (U.S. Census Bureau)

The National Center for Immunization and Respiratory Diseases of the Centers for Disease Control and Prevention and the U.S. Census Bureau have partnered to conduct an evaluation study of the National Immunization 

Survey (NIS). The NIS target population, households with children age 19-35 months, can be hard to identify. 

The NIS Evaluation Study approaches this challenge by using the American Community Survey (ACS) as the 

primary sampling frame, while drawing additional sample from an information reseller (IR) file in one locality 

where insufficient sample is available from the ACS. 

This paper/presentation discusses the procedures and challenges involved in creation of the sample universe, 

selection of the initial sample, subsampling for the field follow-up operation, and weighting of the results. It 

also discusses prospective sample designs to maximize use of the somewhat ephemeral available sample of 

ACS-identified households with 19–35 month old children if the NIS becomes a continuing survey using the ACS 

as its primary sampling frame.

Methodological Changes in the Job Openings and Labor Turnover Survey

John Wohlford (Bureau of Labor Statistics, USA)

The Bureau of Labor Statistics (BLS) Job Openings and Labor Turnover Survey (JOLTS) provides valuable information on labor demand, hiring, and turnover for the U.S. labor market. Monthly job openings, hires and separations data are collected from private sector employers as well as local, state, and federal government units. 

Although not originally designed for this purpose, the JOLTS hires and separations data can be combined to 

produce an implied measure of monthly employment change. However, the change implied by the JOLTS data 

did not track well with the larger and better-known Current Employment Statistics (CES) Survey. BLS embarked 
34

on a multi-year research effort to understand the causes of the divergence between CES monthly employment 

change and JOLTS hires minus separations data and to identify ways to mitigate them. The resulting program 

improvements included a revised sample design, a birth/death model, modified microdata review procedures, 

and an alignment procedure that uses CES employment change as an input. After implementing these improvements, the entire historical JOLTS series was recalculated and re-released. The revised data series now helps 

illustrate the factors underlying monthly net employment change, and also shows some promising leading 

indicator properties. 

Issues Related to Adding Sub-Annual Estimates to the Data Products Available from the American 

Community Survey

Karen King, Alfredo Navarro, Mark Asiala, B. Dale Garrett, Michael Starsinic, Sirius Fuller, J. Gregory Robinson,

Michael Ikeda, Charles Nelson, and Scott Boggess (U.S. Census Bureau)

The American Community Survey (ACS) is a rolling monthly survey, which collects population and housing unit 

data historically collected by the Decennial Census Long Form. The ACS went into full implementation in 2005 

with roughly 250,000 housing unit addresses are contacted each month. Annual data products are released 

yearly for areas greater than or equal to 65,000 persons. Three-year estimates products are released for areas 

greater than or equal to 20,000 persons and five-year products for all areas. 

These three sets of period estimates approximate the average characteristic over twelve, thirty-six or sixty 

months, respectively. The shortest time period the ACS was designed to produce estimates is the calendar year. 

However, data users that study characteristics with potentially large seasonal fluctuations have identified a 

need for and have requested sub-annul estimates, i.e., monthly or quarterly estimates. For example, requests 

have come from service providers who need to estimate utility usage on and off peak season. The ACS has a 

policy of not publishing estimates for periods shorter than a calendar year due to concerns with interpretation 

and data quality.

Research has been developed and implemented to determine if there are estimates that are a reasonable 

approximation of what is happening in the population for shorter time periods using the current ACS design 

and estimation methodology. This talk will describe some results of this research, point out factors that may 

contribute to what we see in the results, and our plans for future research. 

CONCURRENT SESSION VI-A: 

UNIQUE METHODS FOR CONSIDERATION REGARDING DATA CONFIDENTIALITY

The Unique Method for Obtaining Data: Entering Agreements to Share Administrative Records

Stephen Cornman (U.S. Department of Education)

Data sharing agreements are the unique method for governmental entities, universities, and non-profit institutions to exchange administrative records. Data sharing agreements can be used to formally agree to share 

records, provide needed services, or coordinate programs to optimize the benefits from each party’s efforts. 

Agreements should be used whenever there is an exchange of data, funds, personnel, or property, services, or 

any type of commitment or obligation. 

The model data use agreement contains provisions covering common issues in data sharing agreements. The 

concepts are broad enough that they should be considered prior to entering into any data sharing agreement. 

The model agreement offers recommendations for utilizing various provisions in the agreements. For example, 

legal provisions such as legal authority, mutual interests of parties, consideration, costs and reimbursement, 

confidentiality and privacy, data security procedures, and records retention are discussed. 

Some boilerplate provisions can be the same across data use agreements and could be applied to any memorandum of understanding. In the model agreement, there are samples of actual provisions from ten data sharing agreements. There are also some factual scenarios where similar agreement provisions could be applied. 

This model agreement will promote uniform implementation of inter-agency agreements, while considering 

specific factual circumstances and different individual program requirements and procedures.

Statistical Disclosure Control of Tabular Format Magnitude Data: Why It Is Not a Good Idea to Use 

Home Grown (Manual or Automated) Cell Suppression Procedures 

Ramesh Dandekar (Energy Information Administration, USA)

Multiple variations of complementary cell suppression procedures are used by statistical agencies to protect 

sensitive tabular format magnitude data from a statistical disclosure. In this presentation, we address potential 
35

weaknesses associated with some of the most commonly used home-grown cell suppression procedures to 

protect tabular format magnitude data. We also provide some guidance to the survey operations staff on the 

standard procedures that need to be followed/used to ensure adequate protection from statistical disclosure of 

sensitive tabular cells in statistical publications.

Privacy Violations in Accountability Data Released to the Public by State Educational Agencies

Krish Muralidhar (University of Kentucky, USA) and Rathindra Sarathy (Oklahoma State University, USA)

As a part of the No Child Left Behind Act of 2001, every state is required to release information to the public 

regarding the performance of schools in their state. In accordance with this requirement, education agencies 

at practically every state now release information either at the district or school level. When releasing data, the 

education agencies must also ensure that the released data does not violate the requirements of the Family 

Educational Rights and Privacy Act (FERPA) of 1974. Except in special circumstances, FERPA specifically requires 

that individual performance information regarding students cannot be released without written permission. 

In this study, we show that the current methodologies employed by many state educational agencies across 

the country do not satisfy the FERPA privacy requirements. Using accountability data from several states, we 

illustrate the violations of privacy that occur when data is released at the school or district level and show that 

individual performance information for individual students and/or small (primarily ethnic, gender, or other 

disadvantaged) subgroups are easily computed using the data that is released to the public. We describe the 

efforts of the office of Assessment and Accountability at the Kentucky Department of Education to identify these 

privacy violations. It is likely that education agencies continue to release detailed performance information even 

if they are not required by law. The analysis provided in this paper will enable education agencies to provide 

useful data to the public without violating the privacy of individuals or small subgroups.

Expert Task Force Consideration of Confidentiality, Effect Sizes, and Computerized Adaptive 

Testing at NCES

Andrew White and Marilyn Seastrom (National Center for Education Statistics, USA)

The National Center for Education Statistics (NCES) has established a relationship with the National Institute of 

Statistical Sciences (NISS) to establish joint task forces of recognized experts in methodology and data collection to review specific activities of NCES in a given topic areas and make suggestions regarding best practices 

for NCES to consider in planning its future activities. Task forces on the topics of confidentiality, effect sizes, 

and computerized adaptive testing has completed their work and made reports to NCES. This presentation summarizes the considerations of the task forces on their respective topics, the best practices suggested by each 

task force on its appointed topic, and activities undertaken by NCES on the topics.

CONCURRENT SESSION VI-B:

IMPROVING DATA QUALITY THROUGH THE USE OF ECONOMETRIC MODELING

A Model to Improve Timeliness of Business Register Data

Filipponi Danila, Patrizia Cella, and Giuseppe Garofalo (National Institute of Statistics [Istat],Italy) 

The Italian Business Register (BR), also named Statistical Register of Active Enterprises, records all the active 

enterprises of industry and services and their structural characters. This register is the result of the integration 

of administrative/statistical sources. With reference to the year t, the set-up process starts in the last quarter of 

the year t+1, when the yearly data supplies from the main sources are available. Then, after a process of standardization of the administrative units the data are integrated and the main structural variables are estimated 

for each unit. The output is the BR for the reference year t and it is read at year t+16 months.

The availability of Business Registers is fundamental to provide indicators of both short-term and structural 

economic developments. Therefore, the quality of BR is mainly evaluated through its relevance that corresponds to the extent to which it meets user needs. Of course, relevance is strictly connected to timeliness that 

is the lag between the data reference time and the moment in which the information is available. A lack of 

timeliness causes a lack of relevance.

Here, we are trying to improve BR data timeliness exploiting only the administrative/statistical sources available 

in the first quarter of the year t+1 and the time series of BR. Of course, an improvement of the timeliness will 

reduce its accuracy. Indeed, we are not able to produce the BR as individual statistical units, but only an estimate of the business population and their employees regarding to the main structural variables like economic 

activity and class of employees. Generalized linear mixed models are extensions of generalized linear model 

with additional sources of variability due to unobservable random effects. We consider a GLMM to estimate the 

number of enterprises and employees, where the random effects incorporate time correlations among response.
36

Fitting a Linear Model to Survey Data When the Long-Term Average Daily Intake of a Dietary 

Component Is an Explanatory Variable

Phillip Kott (RTI International, USA), Patricia Guenther (U.S. Department of Agriculture), David Wagstaff (Pennsylvania State University, USA), WenYen Juan (U.S. Food and Drug Administration), and Sibylle Kranz (East Carolina 

University, USA)

The National Health and Nutrition Examination Survey (NHANES) collects information on both dietary intake 

and health conditions from a complex sample of individuals in the US. Instrumental-variable regression can be 

used to model an individual’s health-related attribute as a linear function of explanatory variables including the 

average daily intake of dietary components. This overcomes the apparent limitation of the NHANES collecting 

dietary intake data on only two days per sampled individual because the averages of two days per individual 

exhibit considerable intra-individual variability. Readily available software routines can perform survey-sensitive 

instrumental-variable regression with data like that collected by the NHANES, but the relevant quantitative 

literature is not clear about what parameters these routines are actually estimating. We fit the long-term (usual) 

serum beta-carotene level of a population of women ages 20–64 to a linear function of each woman’s long-term 

average (usual) daily beta-carotene intake from food and other explanatory variables using survey-sensitive 

instrumental regression and provide two interpretations of the results.

Collinearity Diagnostics for Complex Survey Data

Dan Liao (University of Maryland, USA)

The Variance Inflation Factor (VIF) is widely employed to assess the degree of variance inflation of the parameter estimate for the i_th independent variable by its collinearity with the other independent variables in a 

regression model. However, little research has been done to extend it for the analysis of complex survey data 

by incorporating complex survey design features, which may relate to the independent variables and affect the 

collinearity patterns in the model. In this paper, we will compare the variances of parameter estimates between 

the regression model with only the single independent variable and the one with multiple independent variables, when the error variance matrix V and the survey weight matrix W are involved in the regression analysis. 

Two adjustment coefficients, varrho_i and zeta_i are defined to adjust conventional VIF when computing collinearity diagnostics in the analysis of complex survey data. We present both the model-based and design-based 

theory to justify the methods. Extended statistics will be applied to the data from the 1998 Survey of Mental 

Health Organizations (SMHO) in a case study. A discussion of rule-of-thumbs and remedies for collinearity 

issues in analysis of complex survey data will be offered in the last section.

Multilevel Models and Small Area Estimation in the Context of Vietnam Living Standards Surveys

Phong Nguyen (General Statistics Office of Vietnam, Vietnam), Dominique Haughton (Bentley University and 

Toulouse School of Economics, USA), and Irene Hudson and John Boland (University of South Australia, 

Australia)

This talk will discuss a methodology to obtain small area estimates in the context of the Vietnam Living Standards Surveys. The presentation will proceed in three parts. First we will introduce the Viet Nam Living Standards Surveys, their historical development, topics covered, sample size issues and challenges. Second, we 

will briefly review main concepts in small area estimation, including the use of auxiliary data, and will contrast 

simple small area models with regression small area models. This will then lead to the notion of random effects 

in small area regression models, and to our proposed multilevel model for small area estimation at the commune level in Vietnam, to our knowledge the first such model built with Vietnam living standards data. The 

third part of the talk will discuss this model. 

Our proposed multilevel model for estimating the commune-level mean (log of) household expenditure per 

capita relies on independent variables available both in the 1999 Census and in the Vietnam Household Living 

Standards Survey of 2002. Following ideas given in work by Moura (1994, 1999), the small area estimation is 

performed by plugging the population means of the independent variables into the regression equation, inclusive of suitable random effects both in the intercept and in the coefficient of the dummy variable for the urban 

location of a household. 

We will discuss how the random effects in the model can also be used to examine the urban-rural gap across 

the country. We will also mention how to measure the accuracy of our small area estimators. Finally, we will 

touch upon the use of sampling weights in models such as presented in the talk.
37

CONCURRENT SESSION VI-C:

VARIANCE ESTIMATION

The RAO-WU Rescaling Bootstrap: From Theory to Practice

Claude Girard (Statistics Canada)

The Rao-Wu bootstrap is the survey-adapted variant of Efron’s original bootstrap method most used at Statistics Canada to conduct variance estimation. Its popularity is largely due to its methodology which gives rise 

to weights (called bootstrap weights) and its ease of use. But with the increased use of the Rao-Wu bootstrap 

method, some strange variance estimates began to emerge, which pointed to flaws in the way the method has 

come to be implemented by survey statisticians. This has triggered an in-depth investigation of the implementation of the Rao-Wu bootstrap which has led to significant improvements and a deeper practical understanding 

of the method. We propose an overview of these issues and the revised methodology brought up to circumvent 

them. While obtained in the context of the Rao-Wu bootstrap, many of the lessons learned along the way are 

universal: they apply to just about any variance estimation methodology being implemented.

Linearization Variance Estimation and Allocation for Two-Phase Sampling Under Mass Imputation

Abdellatif Demnati (Statistics Canada) and J.N.K. Rao (Carleton University, Canada)

We consider two-phase sampling in which values of a variable of interest are observed only in the second-phase 

sub-sample. Values for the first-phase units not sampled in the second phase are mass imputed, using values 

from an administrative file when available and regression imputation otherwise. Such two-phase sampling 

methods are often used in annual business surveys to reduce survey costs and respondent burden, assuming 

that obtaining values from administrative sources is much cheaper than obtaining values through questionnaires from sampled units. We study both naive and design-consistent estimators for a total or mean under the 

above set-up. We also obtain associated variance estimators using a unified approach proposed by Demnati and 

Rao (Survey Methodology, 2004). Simulation results on the finite sample performance of the estimators and 

associated variance estimators are also presented, using substitution or ratio mass imputation. We also study 

the effect of imputation when the underlying model is misspecified. We also study the case of missing sub-sample values and develop estimators of the total and associated variance estimators.

Variance Approximations for Price Index Estimation

John Eltinge and Moon Jung Cho (Bureau of Labor Statistics, USA)

This paper explores several sets of methods for approximation and estimation of variances related to price 

index estimators. The main ideas are motivated by, and illustrated with, examples from the International Price 

Program (I) of the Bureau of Labor Statistics. Following a review of applicable literature, the paper places primary emphasis on six topics. 

(1) Distinctions among multiple sources of variability related to, respectively, superpopulation terms and 

several components of a total survey error (TSE) model.

(2) Large-sample and small-sigma (small-error) methods for variance approximation, with special emphasis on 

features that arise from the hierarchical tree-structured point estimators used for price indices.

(3) Effects related to the fact that the hierarchy in the multistage sample design is somewhat distinct from the 

hierarchy in the tree-structured estimation system. 

(4) Approximation of additional effects arising from the production of index estimators under somewhat 

distinct classification systems, e.g., the Harmonized System (HS), the North American Industrial Classification 

System (NAICS), and the Bureau of Economic Analysis (BEA) System. 

(5) Properties of replication-based variance estimators under the framework developed in parts (1)–(4) above. 

(6) Generalized variance functions based on cross-sectional or temporal aggregation of variance estimation. 

On Point Estimation and Variance Estimation for Mean Expenditures in the Consumer 

Expenditure Survey

Michail Sverchkov, John Eltinge, and Lawrence Ernst (Bureau of Labor Statistics, USA)

The Consumer Expenditure Survey (CE) is a survey with multistage design. The first stage CE sample includes 

a set of areas (PSUs) selected from the set of U.S. Core Based Statistical Areas, and CE additionally selects a set 

of PSUs to represent the rest of the nation. After selecting the original sample of PSUs, a number of selected 

PSUs were cut from the sample for budgetary reasons. Mean expenditures are estimated by Hajek-type ratio 
38

estimator based on the sample weights calibrated to 23 known demographical population totals. The variances 

of these estimators are estimated by Balance Repeated Replication (BRR) technique. In this paper we investigate 

how the sample cut in CE influences current estimates of mean expenditures and examine BRR estimates of 

their variances.

CONCURRENT SESSION VII-A:

INFORMED CONSENT AND PARTICIPATION

Securing Informed Consent from Less Educated Respondents: Results from the National 

Inmate Survey

Rachel Caspar (RTI International, USA) and Paul Guerino (Bureau of Justice Statistics, USA)

The Prison Rape Elimination Act (PREA) of 2003 (P.L. 108-79) requires the Bureau of Justice Statistics (BJS) to 

develop new national data collections on the incidence and prevalence of sexual assault within correctional 

facilities. The Act requires BJS to survey each year not less than 10% of all federal, state, and local correctional 

facilities and to provide facility-level estimates of sexual assault for each sampled facility. To fully implement 

PREA, BJS has developed a multiple-measure, multiple-method data collection strategy. One of these data 

collection activities, the National Inmate Survey (NIS), involves obtaining information directly from adult 

inmates on their experiences with sexual assault. Approval to conduct the NIS was sought from a number of 

different Institutional Review Boards (IRBs) including RTI’s (which served as the IRB of record), the IRBs for each 

of the 50 state Departments of Corrections, and IRBs associated with individual jails. As inmates are defined as 

a protected class by DHHS’s Office of Protection from Research Risks (OPRR) and because the NIS collects data 

of a very sensitive nature, these IRBs were especially concerned with issues related to inmate coercion, the right 

to refuse, and ensuring inmates understood exactly what participation entailed. This presentation will discuss 

the challenges of developing and administering consent materials for an inmate population where literacy and 

formal education levels are typically lower than in the general U.S. population. We will provide a review of the 

consent form used in the NIS and an analysis of the questions and comments raised by sampled inmates as 

they decided whether or not to participate in the study. Our analysis will examine whether there were particular 

aspects of the NIS consent materials that were especially difficult for the inmate population to understand and 

discuss the implications of our findings for other low literacy/less educated populations.

Motivating Non-English-Speaking Populations for Census and Survey Participation

Yuling Pan and Nancy Bates (U.S. Census Bureau)

To meet the growing demand of including non-English-speaking populations in the U.S. Census Bureau decennial censuses and demographic surveys, the Census Bureau has recently expanded its effort to produce survey 

materials in multiple languages. This effort is reflected in the recent translations of the American Community 

Survey (ACS) letters and information brochures into ten non-English languages. These survey materials provide 

important informed-consent messages about the ACS. 

Cognitive testing of the translations of these documents with respondents of the target languages has shown 

that some informed-consent messages were not well received by or not effective for the target populations. 

We intend to gain further understanding on this issue. In the current study, we will use data collected from 

the Census Barriers Attitudes and Motivators Survey (CBAMS) conducted in 2008 to inform the Census 2010 

communications campaign. The CBAMS was a multi-mode survey that oversampled hard-to-count populations, 

including Census tracts that contained a large proportion of non-English-speaking households. The CBAMS measured constructs such as Census knowledge, attitudes and awareness; self-reported propensity to participate in 

the Census; ranking of potential Census messages; barriers and motivators to participation; and consumption 

of mass and social media. 

In this paper, we focus on four groups: English-speaking Hispanics, non-English-speaking Hispanics, Englishspeaking Asians, and non-English-speaking Asians. We will compare their responses to four series of questions: 

(1) intent of Census participation, (2) Census awareness, (3) legal requirement for the Census, and (4) knowledge of Census data uses. We will examine if the four groups are similar or different on these issues. 

The purpose of this study is to advance the qualitative research previously conducted and to better understand 

what motivates non-English-speaking populations to participate in the Census. Results will also help to develop 

more effective Census Bureau documents to encourage Census and survey participation from non-Englishspeaking populations. 
39

Informed Consent and the Statistical Use of Administrative Records

Michael Hawes and Carol Anne Comisarow (U.S. Census Bureau)

Administrative records represent a great source of information, and statistical agencies are increasingly turning 

to these administrative record sources as a high quality, more efficient alternative to direct collection of data. 

Obtaining respondent information from government agencies and commercial entities, rather than collecting it 

directly from the individual, poses significant challenges to the communication of legal authority and disclosure 

potential, and to the proper acquisition of informed consent. This paper will investigate the ethical and logistical challenges relating to the principle of “informed consent” that result from the increased use of administrative records by statistical agencies.

CONCURRENT SESSION VII-B: 

CHALLENGES OF FEDERAL HOUSEHOLD SURVEYS IN EVALUATING FINANCIAL WELL-BEING

A Tale of Two Surveys: Mortgage Wealth Data in the AHS and the SIPP

George Carter III and Alfred Gottschalck (U.S. Census Bureau)

In 2004, homeownership rates peaked in the United States with home prices peaking 2 years later in 2006. 

Since these peaks, homeownership rates and home prices have fallen at the national level. In the current housing market downturn, it is crucial that researchers have accurate mortgage data to document changes over 

time. Two longitudinal surveys conducted by the U.S. Census Bureau collect mortgage data. The American 

Housing Survey (AHS) follows housing units over time and collects information on the quality of housing in the 

United States, as well as information on household characteristics. The Survey of Income and Program Participation (SIPP) is a panel study of households. The main objective of the SIPP is to provide accurate and comprehensive information about income and program participation in the United States. 

To provide a baseline for analyzing housing unit and household mortgage data after the burst of the housing 

bubble, we analyze data on owner-occupied housing units and homeowners with a least one mortgage from 

the 2005 AHS and wave 3 of the 2004 SIPP panel. The presentation will document (1) differences in mortgage 

questions asked in both surveys; (2) the magnitude of differences between mortgage items between the two 

surveys; and (3) the extent to which differences are due to sample and questionnaire differences.

We examine data on the prevalence of fixed mortgages, adjustable rate mortgages, and government insured 

mortgages, as well as data on interest rates, mortgage terms, and levels of home equity. In addition to analyzing the differences between the surveys, we examine demographic, economic, and regional differences in mortgage characteristics. Preliminary research suggests that terms for first and second mortgages are consistent 

across both surveys, that the AHS underrepresents the number of adjustable rate mortgages (ARMs), and the 

SIPP overrepresents FHA and VA program participation on second mortgages. 

Income Measurement in Federal Surveys

John Czajka (Mathematica Policy Research, Inc., USA) and Gabrielle Denmead (Denmead Services and 

Consulting, USA)

This paper presents selected findings from a study of income measurement in eight surveys, focusing on the 

six that are conducted by the federal government: the Current Population Survey Annual Social and Economic 

Supplement, the Survey of Income and Program Participation, the American Community Survey, the National 

Health Interview Survey, the Medical Expenditure Panel Survey Household Component, and the Medicare 

Current Beneficiary Survey. The study, which was funded by the Office of the Assistant Secretary for Planning and Evaluation in the Department of Health and Human Services, included both descriptive and empirical 

components. Findings from the descriptive component will include the treatment of students living away from 

home and the definition of a family. Empirical findings will include comparative estimates of aggregate income 

and its distribution by quintile; poverty status; earned versus unearned income; the proportion of income allocated due to nonresponse; the frequency of rounding; and the impact of family definition on estimated poverty. 

In addition to summarizing these findings we will discuss implications of the study findings for users of income 

data collected in federal surveys.

Evaluation of Health Insurance Coverage Estimates from the American Community Survey

Joanna Turner (U.S. Census Bureau), Michel Boudreaux (University of Minnesota, USA), and Victoria Lynch (Urban 

Institute, USA)

The Children’s Health Insurance Program Reauthorization Act of 2009 tasks the U.S. Census Bureau with 

evaluating the quality of the American Community Survey (ACS) health insurance data as a source of state-level 

children’s health insurance coverage estimates. The ACS began asking about current health insurance cover-
40

age in 2008, with the first release scheduled for the fall of 2009. This paper evaluates the quality of the health 

insurance data with an emphasis first on the collection and editing procedures. Second, the paper will present 

national estimates of coverage and non-coverage by select demographic and socioeconomic characteristics. 

CONCURRENT SESSION VII-C:

EFFECTS OF IMPUTATION ON SURVEY ESTIMATES

Effects of Imputation on CPS Poverty Series, 1987–2007

Joan Turek, Brian Sinclair-James, and Bula Ghose (U.S. Department of Health and Human Services), Fritz Scheuren (NORC at the University of Chicago, USA), and Charles Nelson and Edward Welniak (U.S. Census Bureau)

The Current Population Survey (CPS) has been the primary source of consistently reported income data for over 

50 years. Like most household surveys, the CPS utilizes imputation procedures to adjust for nonreponse to 

income questions. During this time period, many changes have occurred in society and in the willingness of 

respondents to report income. Improvements have also occurred in CPS data collection instruments and 

procedures. This paper examines the effects of imputation on CPS income data from survey year 1987–2007. 

Analysis is conducted by poverty status and by selected demographic characteristics. 

Testing New Imputation Methods for Earnings in the Survey of Income and Program Participation

Martha Stinson and Gary Benedetto (U.S. Census Bureau)

This paper explores the feasibility and effectiveness of three significant changes to standard Census Bureau 

methods of imputing earnings in the Survey of Income and Program Participation (SIPP). Currently imputation 

is performed by stratifying the data based on a set of analyst-chosen characteristics, randomly sorting within 

each sub-group, and choosing a donor based on the nearest neighbor. We investigate the possibility of using a 

model-based approach, supplementing survey-collected job and demographic characteristics with administrative earnings data, and using multiple imputations as proposed by Rubin. We will model monthly earnings from 

January 2004 to December 2005 using the SIPP 2004 panel linked to W-2 tax records extracted from the Social 

Security Master Earnings file. We will use linear regression techniques to estimate a posterior predictive distribution that is the distribution of earnings conditional on all observed characteristics (including administrative 

earnings). From this distribution, we will take four draws to create four imputed values per case with missing 

earnings. After thus “completing” the missing data, we will compare results using original versus new imputed 

values from several standard analyses in order to assess the impact of our new method. In particular, we will 

look at coefficients in a classic earnings regression, trends in earning changes over time, the moments of the 

cross-sectional earnings distribution for a particular month, and poverty levels as based on family income, of 

which earnings are an important component. The four imputed values will allow us to calculate variance estimates using Rubin’s multiple imputation variance formulae and to assess the impact of imputation on the significance of regression coefficients, the shape of the earnings distribution, and the margin of error on poverty 

estimates.

Sensitivity of Inference Under Imputation: An Empirical Study

Jeffrey Gonzalez and John Eltinge (Bureau of Labor Statistics, USA)

Item nonresponse, a common problem in many surveys, occurs when a respondent fails to provide a response 

for a survey question. Imputation models can be used to fill in the item missing information with plausible 

values. These models are built on assumptions about the nature of the missing information. Varying the 

assumptions on the imputation model would likely change the imputed value. If the primary inferential goal 

was point prediction of the missing value, then an undesirable result of the imputation procedure would be 

variation in the imputed values. Oftentimes, however, the main analytic goal is estimation of aggregate values, 

such as population means. Thus, variation in the individual imputed values is of lesser importance while variation in the final population estimate moves to the forefront. Therefore, we examine to what extent, if any, the 

imputation model assumptions affect the estimation of these aggregate values. 

To investigate the sensitivity of inferences when using imputation models built on different assumptions, we 

provide a simulation study with historical data from the U.S. Consumer Expenditure Interview Survey (CE). The 

CE allows an in-depth consideration of the impact of three features on this potential sensitivity. They are (1) 

panel survey design; (2) range of expenditure dollar amounts; and (3) prevalence of certain expenditures (i.e., 

rare vs. frequently incurred expenses). The imputation models should account for these special features of the 

CE. Thus, we develop several imputation models for imputing a variety of expenditures. These expenditures 

vary in dollar amount, proportion of item nonresponse, and proportion of respondents with true zero-dollar 

expenses. We then calculate and compare estimates of population means based on the imputed data. Finally, 

we offer a commentary on imputation model parsimoniousness and implementation feasibility.
41

Imputation Methods for the Current Employment Statistics Survey

Kennon Copeland (NORC at the University of Chicago, USA) and Lan Pham (Bureau of Labor Statistics, USA)

The Bureau of Labor Statistics’ Current Employment Statistics survey collects employment, hours, and earnings 

data monthly from a sample of over 300,000 U.S. establishments. To provide timely information, preliminary 

estimates are generated three to four weeks after the survey reference period. These estimates are revised in 

the subsequent two months, incorporating late reports received after production of the preliminary estimates. 

Benchmark estimates, incorporating administrative population data from the BLS’ ES-202 program for March of 

the prior year, are released annually with the data for May.

Nonresponse potentially introduces bias into survey estimates, if respondents differ from nonrespondents relative to the variables of interest, and also reduces the effective sample size of a survey, thereby increasing variances for survey estimates. The current estimation methods are developed so as to account for nonresponse 

and lessen its impact on bias and variance. These methods, however, assume nonresponse is ignorable within 

defined estimation cells and, hence, do not distinguish among various patterns of nonresponse. Nonresponse is 

used here to encompass both nonreporting and late reporting. Late reporting is temporal nonresponse, as the 

data become available at a subsequent point in time.

The objective of this research is to identify imputation methods that yield decreased revisions in month-tomonth change and total employment estimates. Alternative methods are proposed, and findings are presented 

relative to criteria under which the alternative methods should be applied and performance of the resulting 

employment estimates relative to the current method.

CONCURRENT SESSION VIII-A:

IDENTIFYING AND REDUCING SOURCES OF SURVEY ERROR

Measuring the Complexity and Importance of Businesses in Order to Better Manage our Data Collection Efforts

Serge Godbout and Sungjin Youn (Statistics Canada)

Statistics Canada has adopted a Holistic Response Management (HRM) approach to business surveys. The basic 

principles are founded on a business-centric approach and various types of intervention depending on the size, 

complexity and significance of the business. A methodological framework has been developed to assign each 

business in the Unified Enterprise Survey sample to one of four response management tiers; it will be carefully 

studied during the 2008 production cycle as a pilot study.

The presentation will describe the methodological scheme; explain the relationship between the sampling 

design and the proposed HRM strategy and propose a complexity measure to identify the largest and most 

complex enterprises. Impacts on response management from the pilot study and some issues related to the 

implementation will also be discussed.

Assessing and Correcting the Effects of Measurement Error on the 2008 Economic Directorate 

Editing and Imputation Inventory

Laura Ozcoskun, LaToya Thomas, and Michael Hayes (U.S. Census Bureau)

There are over 100 programs that are conducted within the Economic Directorate at the U.S. Census Bureau. 

Although there are common data correction and imputation procedures used throughout, each program has 

developed procedures that best suit their data. On the surface, this appears to conflict with the directorate-wide 

movement towards generalized processing systems. This conflict is minimized by designing flexible systems 

that offer the most commonly used procedures. Necessary updates to the processing systems occur as new 

“common” methods emerge.

In 2008, we conducted a directorate-wide inventory of editing and imputation procedures, with the primary 

goal of identifying what procedures are already generalized and what procedures need to be generalized. This 

paper describes our efforts to control measurement error throughout the inventory process. We initially sought 

to control measurement error by developing questionnaires targeted to a variety of data correction and imputation procedures and a variety of programs that include ongoing surveys and censuses. Despite an extensive 

review and testing procedure, we received numerous requests for clarification during the data collection process. Moreover, responses on a subset of completed questionnaires conflicted with verifiable or known 

practices. Consequently, we conducted respondent debriefings on a selected set of questions for all participants in the inventory. We describe how we used the results of the respondent debriefings to both produce 

quantitative measures of response error and to revise the inventory results. Lastly, we use these measures to 

make recommendations for similar inventory endeavors that might be conducted in the future. 
42

Models of Nonresponse and Their Relationship to Survey Estimates

John Dixon (Bureau of Labor Statistics, USA)

Nonresponse is thought to have many causes, including privacy, confidentiality, and time constraints. In this 

study, factor models using item nonresponse and paradata are used to separate different reasons for reluctance 

to respond and difficulty to contact respondents. The resulting latent variables are related to survey estimates 

to explore if some sources of nonresponse are more likely to produce bias in economic surveys.

Findings from a Pretest of a New Approach to Measuring Health Insurance in the Current 

Population Survey

Joanne Pascale (U.S. Census Bureau)

This paper presents results from a pretest of an experimental questionnaire on health insurance coverage (the 

“Redesign”), developed to reduce measurement error in the Current Population Survey (CPS). Research has 

shown that some respondents ignore the calendar year reference period and instead report on their current 

status or their most recent spell of coverage, and that those with recent coverage are more likely to report 

accurately than those with coverage in the more distant past. Nevertheless, providing data on calendar year 

coverage is a CPS requirement. Thus the Redesign employs a new set of integrated questions on both current 

and past calendar year status. Another problematic feature of the CPS is the way in which plan type is determined through a series of eight fairly detailed questions on source of coverage. The Redesign takes a different 

approach, first asking about any coverage at all, then identifying general source (job, government or some 

other way) and then following up with tailored questions to elicit the necessary detail. And finally, the household-level design of the CPS can induce underreporting, and yet a person-level design lengthens the survey, 

inducing respondent fatigue and underreporting. The Redesign employs a hybrid approach. It begins by asking 

questions at the person-level and once a particular plan type is identified, questions are asked to determine 

which other household members are also on the plan. Subsequent people on the roster are then asked about 

by name, one at a time, and for those who had been reported as covered on a previously-reported plan, that 

coverage is simply verified and a question is asked to determine if they had any additional plans. Pretest results 

(n=54) were generally positive, suggesting only minor changes to the questionnaire were needed, paving the 

way for a large-scale split-ballot field test in March, 2010.

CONCURRENT SESSION VIII-B:

FORECASTING

Using Spectral Peaks to Detect Seasonality

Tucker McElroy (U.S. Census Bureau) and Scott Holan (University of Missouri-Columbia, USA)

Peaks in the spectrum of a stationary process are indicative of the presence of stochastic periodic phenomena, 

such as a stochastic seasonal effect. This work proposes to measure and test for the presence of such spectral 

peaks via assessing their aggregate slope and convexity. Our method is developed nonparametrically, and thus 

may be useful during a preliminary analysis of a series. The technique is also useful for detecting the presence 

of residual seasonality in seasonally adjusted data. The diagnostic is investigated through simulation and an 

extensive case study using data from the U.S. Census Bureau and the Organization for Economic Co-operation 

and Development (OECD).

The Term Structure with Macro Factors, Expectations Hypothesis and Regime Shift

Xiaoneng Zhu (Nanyang Technological University, Singapore)

This paper shows that the expectations hypothesis (EH) of the term structure of interest rates is defensible 

despite the robust evidence on the predictability of excess bond returns. This is achieved through the introduction of regime-dependent heteroscedasticity into the discrete Vasicek model. The empirical purpose of 

this paper is to reexamine the adequacy of the EH. The restrictions implied by the EH are derived in the 

framework of regime switching macro-finance model. The recursive tests provide evidence in favor of the EH. 

The results may suggest that time-varying risk premiums are second-order effects of yield dynamics. Furthermore, two regimes are found to be intimately related to the Fed operating procedure shift in addition to usual 

business cycles.

A Review of Air Travel Passengers Forecasting Models and Their Robustness 

Ali Dadpay, Charlie Han, and David Chesser (Macrosys Research, USA) and Antony Homan (U.S. Department of 

Transportation)
43

Recent years have witnessed growing significance of air travel forecasting models in business strategic decision 

making and government policy making on one hand and the development of new information entropy (IE) modeling methodologies in econometrics and statistics. Recently commercial aviation has been exposed to several 

macroeconomic events; September 11, 2001, the financial crisis of 2008 and the increased volatility of gas 

prices. These cause us to ask how robust is the performance of forecasting models under high volatility and 

how reliable are their outcomes in this industry. We wonder how these new methodologies would improve the 

performance of air travel forecasting models and their reliability. To answer these questions this study reviews 

existing air travel forecasting modeling techniques. It also introduces new ones by applying Bayesian time 

series modeling and implementing IE methodology. Using nationwide data it estimates the suggested models 

and compares their accuracy and analyzes their robustness during recent macroeconomic events. It suggests 

that using these modeling techniques, which permit us to take into account the different levels of uncertainty 

in the market, would reduce forecasting errors during macroeconomic volatility. It contributes to the existing 

forecasting literature by introducing the very first application of IE forecasting in commercial aviation and by 

offering a benchmark for future studies in this field.

Seasonal Adjustment of Short Time Series with Calendar Effect Treatments: Applied Methods in the 

Bureau of Labor Statistics

Brian Dahlin (Bureau of Labor Statistics, USA)

The Bureau of Labor Statistics (BLS) Current Employment Statistics (CES) program uses a sample-based survey 

methodology to produce monthly estimates of employment, hours, and earnings by industry for the nation. In 

2007, CES expanded its coverage of hours and earnings data. Previously, hours and earnings estimates were 

made for production workers only. While maintaining the production worker concept, CES began to publish 

hours and earnings estimates of all employees on an experimental basis. All employee hours and earnings estimates are available from CES on a not seasonally adjusted basis from March 2006 forward.

Hours and earnings data are subject to seasonal variations. CES seasonally adjusts these series to remove 

such variations, thereby isolating series cyclical and non-seasonal movements. In 2010, CES intends to begin 

publication of seasonally adjusted all employee hours and earnings data, as the minimum three-year history 

required by X-12 ARIMA will be available to produce seasonal factors. The publication of seasonally adjusted all 

employee hours and earnings estimates will mark the conclusion of the experimental phase of estimation for 

these series, after which they will be considered production-quality.

CES routinely applies various calendar effects treatments in the seasonal adjustment process of hours and earnings data. The existing process by which these treatments are applied is incompatible with efforts to seasonally adjust series less than five years in length. CES therefore developed an alternative approach to enable the 

seasonal adjustment of the all employee hours and earnings series that enables calendar effect treatments. This 

approach, described in this paper, will be used until these series reach sufficient length that the existing methodology (developed for lengthier time series) may be applied.

CONCURRENT SESSION IX-A:

ISSUES CONCERNING THE STATISTICAL USE OF ADMINISTRATIVE RECORDS

Overview of Open Research Questions in Methodological and Empirical Work with Statistical Uses 

of Administrative Records

Shelley Wilkie Martinez (Office of Management and Budget, USA), John Eltinge (Bureau of Labor Statistics, USA), 

and Jenna Fulton (University of Maryland, USA)

This paper provides an overview of several areas of methodological and empirical research that could enhance 

the production of statistical information from administrative records. First, we present some definitions and 

review some standard statistical uses of administrative records. Second, we expand on the ideas of House 

(2008) and others to outline a “life cycle” in the abovementioned uses. Inferential goals, data-sharing agreements, and processing steps receive primary attention. Third, the paper highlights five areas of research opportunities, including evaluation of the prospective and realized value of statistical information; links between 

value and common measures of data quality; cost structures; and methodological tools to enhance value or 

reduce costs and risks. For each of these five areas, we highlight some of the important issues, provide a selective review of some applicable previous literature, and outline possible approaches to further exploration of the 

open questions. The paper closes with comments on commonalities and distinctions among research opportunities in surveys and administrative records, respectively.
44

Expanding Statistical Use of Administrative Data: A Research Proposal Focused on 

Privacy and Confidentiality

Gerald Gates (Gerald Gates Privacy Consulting, USA)

This paper outlines a research agenda necessary to help understand why barriers to the statistical use of administrative records exist and how they can be overcome. While there are legal issues that must be addressed 

by researchers in accessing these records, the most intransigent issues involve policy decisions related to 

confidentiality of personal information and privacy of individuals. Addressing these policy concerns has been 

difficult and time consuming for statistical agencies and researchers, has resulted in missed opportunities, and 

has not necessarily facilitated privacy and confidentiality. To better address these concerns, it is critical that 

negotiations between administrative agencies and researchers recognize real risks both to privacy and to data 

use. Regarding confidentiality, research should focus on identifying and limiting risks to confidentiality from 

security breaches or inadequate disclosure avoidance measures. Privacy concerns are more subjective and are 

the most difficult to overcome. Research on privacy needs to focus on public awareness of uses for these data 

and how to ensure appropriate informed consent. Additional research is proposed to determine the extent 

where valuable research is abandoned because agreements cannot be reached because guidance on addressing privacy and confidentiality is lacking. The findings will be helpful in establishing model agreements and 

forming more generalized legislation and policy support for the statistical use of administrative records. The 

paper concludes with a research plan to inform the decades-long discussion on how the public, from a privacy 

perspective, might view an expanded role for administrative records in the decennial census.

Investing in a Data Quality Assurance Program for Administrative Data Linked to Survey Data for 

Policy Research Purposes Is Essential

Michael Davern (University of Minnesota, USA)

Investing in an adequate process of quality assurance is essential for linked survey and administrative data. 

Administrative and survey data have many known limitations for policy research purposes for which they are 

used (e.g., administrative data do not have detailed demographic information and survey data have flawed 

program participation data). When the two types of data are linked together the potential for both good (e.g., 

one type of data can help correct for limitations of the other) and harm (compounding errors/limitations leading to inappropriate inferences) is amplified. Survey data are the only source for some data we need to predict, 

evaluate and learn about the impact of potential and actual changes in policy. Public program administrative 

data that are used to keep track of program beneficiaries but lack data on unenrolled (and potentially eligible 

currently or if program rules changed). A major drawback of the linked data files is that most of them will never 

be put into the public domain due to the sensitive nature of linking. Therefore, important quality assurance 

work done in concert by the data producing entities (such as the Census Bureau) and policy research scholars 

on the public use survey data is not possible. There is a need for a significant investment in not only linking 

the data sources but in producing methodological studies and developing documentation on how to use the 

data appropriately. In this paper we set out a research agenda for improving linked data files for policy research 

considering research that needs to be conducted concerning coverage error, non-response error, sampling 

error, measurement error, editing/imputation, documentation of metadata and production of timely linked data 

files. We also propose ways that agencies entrusted with the linked data can engage the assistance of the policy 

research community.

CONCURRENT SESSION IX-B:

WEB SURVEYS

Internet Surveys and the Demise of the Matrix

Grace O’Neill and Stan Freedman (Energy Information Administration, USA)

Survey designers are constantly faced with the challenge to maximize the amount of data collected while 

minimizing the overall burden placed on data providers. This tradeoff has led to the ethos that a shorter survey 

is less burdensome than a longer one. This ethos, has not led survey designers to ask for fewer data items on 

fewer pages. Rather it has led to cramming more data items into the same number of pages. This practice can 

actually increase respondent burden as it takes longer for the respondent to navigate the instrument and it has 

unknown consequence for the quality of the data collected. This is especially true in establishment surveys 

where survey designers often use complex matrixes to collect data because we think it’s easier for respondents 

to report their data and because it takes up less space on a survey form. 

This paper will discuss not only the cognitive implications of using complex matrixes but also alternative 

methods to gathering the same information with less cognitively taxing methods. Specifically, we will address 

the use of electronic data collection methods to reduce respondent burden. These implications and alternative 
45

methods are based on EIA’s cognitive testing findings across several survey forms, including one that collects 

complex data using a format other than a matrix. Finally, we will conclude with a discussion of possible ways 

to measure respondent burden and different measure of survey design success. 

The Effects of Grouping Response Options in Factual Questions with Many Options

Cleo Redline and Roger Tourangeau (University of Maryland, USA), Mick Couper and Frederick Conrad 

(University of Michigan, USA), and Cong Ye (University of Maryland, USA)

We studied two questions drawn from the American Community Survey that offer a long list of response 

options: education and transportation. We carried out two experiments to investigate how various methods of 

grouping and orienting the response options in the two questions affected respondents’ processing of them. 

One of the experiments was a large-scale experiment embedded in two Web surveys (n = 2400); the other 

experiment was a smaller eye movement analysis conducted in the laboratory (n = 155). In both experiments, 

we grouped the response options conceptually and spaced them an equal distance apart. We then varied the 

direction in which the groups were arrayed: either vertically down the screen or horizontally across the screen. 

We also varied whether headings preceded the groupings. Since these are factual questions, we predicted that 

respondents’ answers would not differ between the various conditions, but that response times would. This 

prediction was borne out in both studies. We thought that response times would be reduced when the response 

options were arrayed horizontally across the screen and when headings were introduced to guide respondents’ 

selections. In general, the results suggest that headings hindered (slowed) processing rather than helped it. 

Video clips of respondents’ eye movement further reveal respondents’ confusion with the headings. We will 

present detailed findings of these studies and discuss implications for the design of such questions.

Response Format Effects on Measurement of Employment

Randall Thomas (ICF International, USA), Jolene Smyth (University of Nebraska-Lincoln, USA), and Don Dillman 

(Washington State University, USA)

Accurate measurement of employment is essential to track employment trends in a nation and within regions 

of a nation and this information can provide information to determine the effectiveness of a variety of private 

and governmental programs designed to increase employment. Some have noted discrepancies in estimated 

employment numbers between the Census and the CPS (Census typically has a lower count of employed 

people); most often attributed to differences in interviewing mode, time frame reference, or sampling frame. 

Many survey researchers using paper-pencil or web-based questionnaires present a multiple response question 

(“Select all that apply’) to assess employment. However, in a telephone interview, employment is often asked 

through a series of yes-no questions, with the interviewer requesting a “yes” or “no” response for each item 

presented in sequence (cf. Smyth, Christian, and Dillman, 2008, POQ).

In research with self-administered questionnaires, the Yes-No Grid format has been found to yield a higher level 

of endorsement than the Multiple Response format in self-administered surveys (Smyth, Dillman, Christian, and 

Stern, 2006, POQ; Thomas and Klein, 2006, JOS). 

Method

We designed a web-based study that was conducted across 24 monthly waves with over 60,000 respondents 

(18 or older), we sought to compare the effects of response format on employment measurement.

Respondents were randomly assigned to one of the 3 employment response scale formats: Multiple Response 

Format (MRF); Yes-No Grid (YNG for employment); Single Response Format (SRF).

Results Overall, endorsement of every category was higher with the YNG and lowest with the SRF. We will also 

describe how these results are related to changes across quarters and how they are related to a number of 

other work-related variables, including hours worked per week. 

Mode Effects in Mixed-Mode Surveys of Veteran and Military Populations

Boris Rachev (ICF International, USA)

Studies on mixed-mode survey designs and mode effects so far have produced mixed findings. We hope to 

contribute to the literature by examining mode effects between mail administration of paper surveys and Web 

administration of their electronic equivalents in three recent surveys of veteran and military populations. The 

three surveys are: U.S. Army MWR Leisure Needs Survey, Survey of Veteran Satisfaction with the VA Home Loan 

Guaranty Process, and 2008 Veterans Burial Benefits Survey. Initial study results indicate that the mail survey 

mode is preferred over the Web, with some notable variation in mode preference across surveys. The analysis 

examines how selected survey elements such as response rates, respondent profiles, item response rates, and 

key statistical estimates differ between mail and web. Logistical regression and T-tests used to compare modes 

and population profiles.
46

CONCURRENT SESSION IX-C:

ISSUES IN SAMPLE DESIGN

Revisiting Nested Stratification of Primary Sampling Units

Tom Krenzke and Wen-Chau Haung (Westat, USA) 

Stratified multi-stage cluster area samples are designed when conducting in-person surveys in the U.S. In the 

first stage of selection, a usual approach is to form, stratify, and select Primary Sampling Units (PSUs), which are 

typically counties or groups of counties. These geographic areas are formed to reduce interviewer travel costs 

and to maximize the heterogeneity within PSUs. Prior to selection, PSUs are stratified into homogeneous groups 

in order to reduce the anticipated standard errors on the resulting survey estimates. Another objective is to 

form equal-sized strata to help achieve close-to-equal interviewer workloads and a self-weighted sample. 

A nested stratification design begins with identifying major strata (primary strata), and then forming substrata 

(secondary strata) independently within each major stratum. The approach essentially forms a tree structure 

with nodes representing intermediate substrata, and ending nodes representing final substrata. The result is a 

clear set of strata with explicitly defined boundaries. Kish (1965) discusses exhaustive efforts while implementing a nested stratification approach, and questions the benefits of such expensive efforts.

In the past couple of decades, computer-intensive stratification searches have been implemented using sophisticated multivariate clustering algorithms, such as described in Jewitt and Judkins (1988) and Ludington (1992). 

In the same spirit, we describe exhaustive searches for a simplified multivariate algorithm using nested stratification that maximizes homogeneity (using distance measures) under size constraints, while arriving at explicit 

boundaries. In effect, we revisit the efforts conducted by Kish and his colleagues, measuring the variation 

across hundreds of stratification schemes. The evaluation is conducted using data from the 2003 National 

Assessment of Adult Literacy and other county-level data from a variety of sources, including the Census 

Bureau’s Population Estimates and Small Area Income and Poverty Estimates programs, and the Bureau of Labor 

Statistics’ Local Area Unemployment Statistics program. 

Statistical Sample Design for Coalbed Methane Industry Survey: Projects Versus Wells

Marla Smith and Carey Johnston (U.S. Environmental Protection Agency)

This presentation will describe the statistical sample design for an establishment survey of the coalbed methane (CBM) industry. In 2006, CBM extraction activities accounted for about 9.4 percent of the total U.S. natural 

gas production. Today, CBM activities are expanding across the United States. Because CBM extraction requires 

removal of large amounts of water from underground coal seams before CBM can be released, the Office of 

Water at the U.S. Environmental Protection Agency (EPA) is studying whether it should develop regulations on 

pollutant discharges to the nation’s waters. As part of this study, EPA has conducted a survey of the CBM 

industry. The sample design for the survey was particularly challenging because the available public data 

sources did not directly identify the primary unit of analysis. From public data sources, EPA identified 484 

operators and over 45,000 wells, but was primarily interested in the organization of wells into “projects” that 

are operated as economic units for business decisions. This critical information about projects was available 

only from the operators. In the first of a two-phase design, EPA required operators to identify its projects and 

the number of wells within each project. This information was used to verify adequate coverage and to select a 

sample of projects for the second phase. This presentation will describe EPA’s experiences and the challenges 

of the sample design.

Maximizing Overlap of Large Primary Sampling Units in Repeated Sampling: A Comparison of 

Ernst’s Method with Ohlsson’s Method

Padraic Murphy and Reid Rottach (U.S. Census Bureau)

Many large repeated or continuous demographic surveys employ a multi-stage design where large geographic 

areas (such as counties or clusters of contiguous counties) are sampled in the first or primary stage. Usually, 

a new sample of these primary sample units (PSUs) is selected periodically in order to account for changes in 

population, survey objectives, or other considerations. But because hiring and training new interviewers can be 

expensive, and replacing experienced interviewers with inexperienced ones can have an adverse effect on data 

quality, there is often a strong incentive to retain as many as possible of the PSUs from the “old” sample design 

when selecting the “new” PSU sample. At the same time, one wishes to also retain the advantages of having 

a probability sample. Various methods have been proposed to coordinate repeated samples with these two 

considerations in mind. This paper discusses and compares two such methods. The first method, due to Ernst 

(1986), has been used for demographic surveys at the U.S. Census Bureau. This method does not require independent sampling between strata in the previous design, and is cast as a constrained optimization problem, so 
47

in some respect the solution is optimal. The second method, due to Ohlsson (1996), uses exponential sampling, 

and does have the requirement of independent sampling; but it may be used repeatedly because it does not 

destroy independence in the current design. We simulate first stage sampling with each method to estimate 

relative overlap, costs, and sampling variance using data from the 1980, 1990, and 2000 Decennial Censuses 

of Population and Housing and from a household survey with sample designs based on those censuses.

Minimizing Duplication of Samples Drawn from Overlapping Frames of Multiple Surveys

Peter Kwok, Hee-Choon Shin, Colm O’Muircheartaigh, Whitney Murphy, Angela Debello, and Kari Carris (NORC 

at the University of Chicago, USA) and Youlian Liao (Centers for Disease Control and Prevention, USA)

REACH US (Racial and Ethnic Approaches to Community Health Across the United States) is a CDC communitybased initiative to eliminate health disparities among various racial and ethnic groups. Five of the 28 participating communities are located within Los Angeles and Orange counties, California, with complicated overlapping 

among their geographies. Although all respondents are asked the same questions, those communities will not 

share any completed results because they have different scientific interests and eligibility requirements. Therefore, this is a multiple-frame, multiple-survey problem that requires samples to be independently drawn from 

overlapping areas. We will discuss how the address-based sampling design can meet this demand and what 

challenges it faces. In particular, we present an estimation algorithm that aims to minimize sample duplication 

to protect the independence assumption.

CONCURRENT SESSION X-A:

PRODUCING AND COMMUNICATING DATA AND STATISTICS

Metadata and Data Harmonization

Daniel Gillman (Bureau of Labor Statistics, USA) and Frank Farance (Farance Inc., USA)

The theory of terminology for special languages deals with concepts, their structure, relationships among them, 

and signifiers that designate them. An example of a special language is the set of terms and concepts used by a 

statistical survey. This includes the data they produce, as data may be defined as terminology. In this way, we 

can account for the underlying concepts the data represent. These concepts constitute the meaning, or semantics, of data. The semantics include survey specific concepts and the allowed computations for the data.

Through the technique we call factoring, the semantics are broken into a derivation chain, whereby they may 

be compared or mapped. For example, the semantics for a datum such as U.S. average hourly earnings in Feb 

2009 is $18.47 can be broken into a derivation chain. The chain includes an aggregate (U.S. employers), a characteristic of the aggregate (average hourly earnings), a unit of measure (dollars), and a datatype (non-negative 

real number), among others. This provides a means for analysis.

Harmonization of data occurs when corresponding concepts in the derivation chains for each datum are 

mapped to each other. For corresponding concepts, one defines a generic concept, for which each correspondent is a specialization. However, it is fairly difficult to analyze concepts without writing them down in some 

way. Metadata for concepts include the reification of them into definitions and lists of their relevant characteristics. These metadata convey the meaning of the concept. It is the metadata for all those concepts that conveys 

the intended meaning of the datum to a user.

This paper contains a fuller description of the above and provides details on the factoring technique.

Innovative Uses of Data Mining Techniques in the Production of Official Statistics

Jaki McCarthy, Thomas Jacob, and Dale Atkinson (National Agricultural Statistics Service, USA)

Data mining techniques are used to find patterns, classify records, and extract information from large data 

sets. These techniques, often used in the private sector for market research, fraud detection, and customer 

relationship management, can also be used by statistical agencies to analyze their large survey data sets. While 

large datasets are common in many statistical agencies, data mining techniques have not been widely used to 

improve the production of official statistics. However, innovative applications of these techniques can be very 

effectively used to improve survey data, processing and estimation.

There are many potential applications of data mining techniques with large survey data sets. This paper will 

discuss data mining applications already underway in NASS, as well as several proposed future data mining 

projects. NASS has used classification trees to model survey reporting errors, identify records most likely to be 

population members (for NASS, agricultural operators), predict survey non-respondents, and to create nonresponse weighting groups. Cluster analysis is being used to identify subgroups within larger sets of respon-
48

dents. In the future, cluster analyses may be used to suggest possible subgroups of respondents for which data 

collection may be tailored. Association analyses may have potential applications in designing data editing systems. These and other potential applications of the broad range of data mining techniques for improvements in 

data collection and processing will be discussed. 

Benchmarking the Performance of Statistical Agencies

Marilyn Seastrom and Valena Plisko (National Center for Education Statistics, USA)

Benchmarking is the process of identifying, sharing, and using knowledge and best practices to improve any 

given business process. One can think of benchmarking against past performance by your own organization, 

benchmarking against internal or external standards of performance, or benchmarking against other organizations. For a government statistical agency like the National Center for Education Statistics (NCES) in the United 

States, one can think about benchmarking at a number of different levels. At the broadest level, we could use 

established models and criteria to benchmark NCES as an organization. Looking a bit more specifically at the 

work of a statistical agency, we could also develop a set of criteria for benchmarking against other statistical 

agencies. And finally, at the most micro-level, we could benchmark the actual data produced at NCES against 

accepted statistical standards or cutting-edge statistical techniques, describing survey and non-survey errors. 

A similar analysis in 2001 revealed that agencies had taken very different approaches in measuring their performance, both in terms of what they were measuring and in how they were measuring it. In the several years 

since this initial look, one might expect more commonality across agencies in both their measures and their 

targets for performance, as agencies have responded to federal accountability provisions to document concrete 

outcomes and show continuous progress. In this paper, using information obtained from publicly available 

federal agency performance reports and supporting documents, we will explore the ways in which NCES can 

benchmark its performance against other U.S. statistical organizations.

CONCURRENT SESSION X-B:

REDESIGNING LARGE-SCALE FEDERAL SURVEYS: CONNECTING TO THE PAST, ANTICIPATING THE FUTURE

Searching for Alternatives to a Random Digit Dial Telephone Interview–Redesigning the National 

Household Education Surveys

Chris Chapman (National Center for Education Statistics, USA) and Mary Hagedorn (Westat, USA)

The National Center for Education Statistics (NCES) has fielded the National Household Education Surveys 

(NHES) every few years since its inception in early 1991. The surveys are designed to allow NCES to collect 

data directly from parents regarding their preschool through high school children’s education, and from adults 

about their education. Random Digit Dialing (RDD) and Computer Assisted Telephone Interviews (CATI) have 

been used to generate NHES samples and to collect the data. Declining response rates affected NHES collections 

and increasing cell phone use increased undercoverage problems given the landline number frames available 

for RDD sampling. These problems led NCES to redesign how NHES samples are constructed and NHES data are 

collected. The presentation will focus on the redesign process. Information will be presented regarding how 

NCES decided to proceed to study new ways of fielding NHES including details about the extent of the problems 

with the study before a redesign effort was undertaken. The presentation will also provide information about 

the project budget and the implications of this for the kinds of designs that are being evaluated. Details on 

the different designs under consideration will be presented, including the rationale behind opting to test one 

approach over another. This part of the presentation will include a discussion about how we plan to use results 

from the initial stage of redesign tests to inform a much larger field test scheduled to be fielded in early 2011. 

This second, large scale field test will be used to evaluate how shifting away from an RDD and CATI interview 

approach not only might affect potential nonresponse and coverage bias, but also to study how the shift 

might affect trends analyses that can be conducted using previous NHES data and NHES data collected using 

new methods.

Redesigning the National Surveys on Drug Use and Health

Joseph Gfroerer, Jonaki Bose, Dicy Painter, Michael Jones, and Joel Kennet (Substance Abuse and Mental Health 

Services Administration, USA)

The Office of Applied Studies (OAS) in the Substance Abuse and Mental Health Services Administration (SAMHSA) 

conducts the National Survey on Drug Use and Health (NSDUH), an annual survey of the civilian, noninstitutionalized population of the United States aged 12 years old or older. The survey is used to produce national and 

state-level estimates of the prevalence of use of illicit drugs, alcohol, and tobacco products as well as measures 

related to mental health. OAS is planning for a redesign of the survey that will balance three competing goals: 

1) maintain valid trend measurement; 2) update and improve the questionnaire and survey methodology; 3) 
49

keep costs within expected budget levels. This paper describes the redesign process and the challenges faced 

in achieving these three goals. Some of the topics covered are how to assess priorities and data needs, how to 

design a survey that remains flexible to policy needs and budgetary constraints without affecting trends, studies conducted by OAS to evaluate current and new methodologies, uncertainties about future budgets, involvement of senior managers prior to and during the development of the redesign, and ways to unveil and sell the 

redesigned survey to key constituents. 

Redesigning the National Crime Victimization Survey

Michael Rand (Bureau of Justice Statistics, USA)

The National Crime Victimization Survey (NCVS) was instituted in 1972 to produce national estimates of the 

levels and rates of crimes of violence and theft, including crime not reported to police departments. In recent 

years the viability of the survey and its ability to meet the goals it was created to meet have being threatened 

by increasing costs in a period of fiscal austerity. The Bureau of Justice Statistics has initiated a research program towards redesigning the NCVS to improve the cost efficiency of the survey design and the precision of the 

survey’s victimization estimates, as well as to enhance the survey’s ability to permit subnational estimates and 

to better address emerging issues related to crime and criminal justice. 

The methodological research and implementation of the redesign is anticipated to take about 5 years, and will 

encompass a broad program of research concerning all aspects of the survey’s design. Currently, BJS has initiated research in 5 areas: sample design, the survey’s reference period, modes of interview, non-response bias, 

and subnational estimation. Planned studies will focus on the crime screening questions and integrating the 

varying competing goals into the redesigned survey.

This paper reviews the reasons BJS has initiated a survey redesign, the agency’s strategies for developing the 

redesign process, the projected goals for the redesigned victimization survey and BJS’s overall research and 

implementation plans. 

CONCURRENT SESSION X-C:

ADVANCES IN AGRICULTURAL FRAME DEVELOPMENT AND USE

Utilizing an Alternative Sampling Frame to Produce Agricultural Survey Indications

Wendy Barboza and Mark Harris (National Agricultural Statistics Service, USA)

The U.S. Department of Agriculture’s (USDA) National Agricultural Statistics Service (NASS) conducts hundreds 

of surveys every year and prepares reports covering virtually every aspect of U.S. agriculture. For many surveys, data is combined from the list frame and area frame samples to produce multiple frame survey indications. NASS maintains a list frame containing names, addresses, telephone numbers, and other descriptive data 

on producers (and agribusinesses) and an area frame covering all land area in the U.S. To form multiple frame 

survey indications, data collected from the list frame sample are combined with data collected from the area 

frame operators who are not on the list frame. In this respect, the area frame accounts for the incompleteness 

of the list frame. For many years, NASS has partnered with USDA’s Farm Service Agency (FSA) to use their data 

as an administrative data source since producers report their planted crop acreages to FSA on an annual basis. 

Starting in December 2006, NASS initiated an operational pilot program to employ using FSA data as a sampling 

frame in the state of Nebraska. The goals of this new methodology were to reduce respondent burden, reduce 

data collection costs, and improve survey indications. After two years, the operational pilot program was discontinued because the objectives were not sufficiently achieved and efforts to maximize the use of FSA data 

were redirected to a research mode. This paper provides an overview of NASS utilizing an alternative sampling 

frame to produce agricultural survey indications.

Four-Digits or No-Digit Social Security Numbers: Impact on the National Agricultural Statistics 

Service Record Linkage Maintenance Processes

Denise Abreu, Kara Daniel, William Iwig, and Stan Hoge (National Agricultural Statistics Service, USA)

The National Agricultural Statistics Service (NASS) relies on Social Security Numbers (SSNs) and Employer Identification Numbers (EINs) as important matching variables for use in record linkage processing and other list 

aintenance activities that are conducted in order to maintain a high quality list of U.S. farmers and ranchers. 

Record linkage is used to match the NASS list frame to new lists and administrative data sources in order to 

identify new farmers and ranchers not previously identified by NASS. Record linkage is also used to match the 

list frame to administrative sources for maintenance purposes, for example updating telephone numbers on 

the list frame. However, maintaining nine digit SSNs/EINs on the list frame is a PII security concern for NASS. 

This research evaluated the potential impact of eliminating SSNs/EINs from the list frame or of only maintaining 

four digit SSNs/EINs on the various list building and maintenance record linkage processes. Overall, the results 
50

showed that using four digit SSNs/EINs for record linkage would result in missing 1 to 2 percent of the actual 

SSN matches and 4 to 8 percent of the actual EIN matches. However, having no SSNs/EINs for record linkage 

would result in missing 4 to 6 percent of the actual SSN matches and 9 to 13 percent of the EIN matches. In 

general, the percentage of missed matches will increase as the size of the data sets being matched increases. 

Since NASS processes several record linkage projects per state per year, the cumulative effect of either 

approach on the quality of the list frame is a concern.

Comparison of Methods for Updating Census Based Estimates of Number of Farms to 

Non-Census Years

Michael Bellow (National Agricultural Statistics Service, USA) and Phillip Kott (RTI International, USA)

The National Agricultural Statistics Service (NASS) conducts the Census of Agriculture (a complete count of U.S. 

farms and ranches) every five years. On an annual basis, sample surveys including the June Area Survey (JAS) 

are carried out to obtain estimates of many of the same agricultural commodities as the Census. Due to the 

complete coverage provided by the Census, its numbers are considered more accurate than those derived from 

the sample surveys. An interesting question is whether Census figures for specific commodities can be used 

in conjunction with survey data to improve estimation accuracy for non-Census years. Due to its relative 

stability over time, the commodity considered most likely to benefit from such an approach is number of 

farms in a state. 

We evaluate two proposed methods for projecting Census counts of number of farms to subsequent non-Census years. The first method updates the Census figure to the current year using JAS data only, while the second 

makes additional use of official NASS state level estimates of number of farms for the previous year. The two 

estimators are compared with area frame based and hybrid operational estimators for the 2003–06 periods in 

a study covering most of the lower 48 states. Variances are estimated using an extended delete-a-group 

jackknife method. 

CONCURRENT SESSION XI-A:

APPLICATIONS OF SAMPLING

Evolutionary Algorithms for Optimal Sample Design

Charles Day (Statistics of Income Division, Internal Revenue Service, USA)

Finding optimal stratum boundaries and optimal allocations to those strata has been a theme of recurring 

interest since the early 1950s. This paper presents continuing research on the use of Evolutionary Algorithms 

(EAs, sometimes called Genetic Algorithms) to do multivariate optimal allocation and stratification. Evolutionary 

Algorithms are inspired by the process of natural selection. As natural selection makes a species increasingly fit 

for its environment, an EA evolves candidate solutions to an optimization problem with monotone increasing 

scores on the problem’s objective function. Because of the flexibility of their objective functions and the many 

creative variations on the basic algorithms, EAs are capable of simultaneous solution of more than one related 

problem (coevolution) and finding multiple non-dominated solutions to a problem for the study of tradeoffs in 

the optimization process (multiobjective optimization). The author has applied these algorithms successfully to 

the solution of a large multivariate optimal allocation for a sample of tax returns. Possible future applications 

include the selection of balanced samples from moderate-sized populations.

Deviation of Sample Size Formula for Cluster Randomized Trials with Binary Responses Using a 

General Continuity Correction Factor and Identification of Optimal Settings for Small Event Rates

Majnu John and Madhu Mazumdar (Weill Cornell Medical College, USA)

Cluster randomized trials (CRTs) are comparative studies designed to evaluate interventions that operate at a 

cluster level. For evaluation of health care interventions, the clusters to be randomized could be communities, 

clinics, hospital wards or medical practices and not the individual subject. Use of CRTs is rapidly increasing.

 For comparison of binary outcomes, guidelines for sample size computation have been developed extensively 

for individually randomized trials (IRT) but for CRT, there are situations that still need further consideration. 

CRTs are usually less efficient than IRTs needing larger sample sizes. Therefore adequate design with proper 

sample size calculation is of utmost importance in trials utilizing CRT.

Exact methods for sample size computation are the gold standard, especially at the extreme range of the 

proportion metric. However, approximate methods are mostly utilized since the exact methods often involve 

burdensome computation. A series of approximation methods have been developed for IRT until one by Casagrande et al (Biometrics, 1978) had been found satisfactorily close to that obtained by exact methods. For CRT, 

although there are few approximate formulations published, the search for the best formulation continues. 
51

In this paper, we propose a new chi-square approximation formula with a general continuity correction factor 

and compare its operating characteristics to the available methods using statistical simulations. We also illustrate our methods by applying to the “Pathways to Health’ study which is a CRT of nicotine gum and motivational interviewing for smoking cessation in low-income housing (Okuyemi et al, Health Educ Behav 2007).

Sampling from Discrete Distributions: Application to an Editing Problem

Lawrence Cox (National Center for Health Statistics, USA) and Marco Better (OptTek Systems, Inc., USA)

At the 2001 FCSM Research Conference, Greene et al. introduced a problem in editing and imputation based 

on fire data. The editing problem consists of imputing values to cells in a 2x2 contingency table subject to 

extensive item and unit nonresponse. Mathematically, the nonresponse creates an incomplete 2-way table with 

partial counts for individual cells and marginal totals. Statistically, the problem is to adjust the partial table to 

a fully populated (imputed) table. Traditional imputation methods such as ratio adjustment and raking are ineffective, as they create imputed cell counts less than observed partial counts. Moreover, in addition to maximum 

likelihood estimates, it is often desirable to produce a probability sample of imputed tables from which confidence intervals and tests of hypotheses regarding potential missing data mechanisms can be obtained. We illustrate a method and efficient software for obtaining a probability sample for the editing problem and the general 

case of a partially-specified contingency table of network type, based on mathematical networks, and discuss 

statistical applications of this methodology.

CONCURRENT SESSION XI-B:

INCOME AND THE BUSINESS CYCLE

Sources of Earnings Volatility

Beethika Khan and David Beede (U.S. Department of Commerce)

Much effort has been expended studying long-term trends in the earnings volatility of workers. Less attention 

has been paid to analyzing sources of earnings volatility, such as the demographic and labor force characteristics of people experiencing such volatility. Focusing on the sources can help gauge how much is a voluntary 

component of the evolution of earnings over the lifecycle and how much is due to larger forces such as business cycles and longer-term secular changes in technology and globalization. Public policy responses to earnings volatility may differ considerably depending on its sources.

This paper will use data from the Survey of Income and Program Participation (SIPP) and the Current Population 

Survey to measure the contributions to year-over-year fluctuations in real earnings arising from: 

1. variation in volatility across industries and occupations; 

2. transitions between industries and occupations;

3. variations in hours and wage rates; 

4. transitions between part time and full time work; between part year and year-round work; and into and out 

of the labor force.

Preliminary analysis using the SIPP data suggests that earnings volatility varies quite a bit across industries 

and occupations. For example, earnings volatility is particularly high in administrative support, real estate and 

leasing, arts and entertainment, and accommodation and food services industries, and relatively low in utilities, 

manufacturing, and public administration. Higher earnings-volatility occupations include food preparation and 

serving, personal care, and building and grounds cleaning and maintenance, while science, engineering, and 

installation, maintenance, and repair occupations have relatively low volatility. Industries and occupations with 

higher earnings volatility may have greater variations in hours and wages than other sectors. The variation over 

time in industry- and occupation-specific earnings volatility may reflect longer-term changes in the economy.

The Impact of a Rapid Recession on the Statistical Process

Pieter Vlag (Statistics Netherlands)

In the Netherlands the so-called Business Cycle Tracer (BCT) is used for analyses of trends in conjuncture. The 

BCT consists of a combined set of confidence indices, production indices, consumption indices and indicators 

about the labor market. All these components are filtered out for short-term and erratic components. The BCT 

is intended as a tool to facilitate the analyses of medium-term economic developments. It has been empirically proven that the BCT described the Dutch business cycles between 1990 and 2008 quite well. However, 

the rapid economic decline during the 4th quarter of 2008 showed 1) the necessity of some additional shortterm indicators and 2) the need of improvements of the BCT (and underlying indicators). This is because the 
52

link between financial indicators and economic indicators was missed. Secondly, the amplitude of the rapid 

economic decline was well detected and the rapid decline itself was detected too late. The latter factors are 

probably related the to fact that confidence indices are the only early indicators for changes in conjuncture in 

the Netherlands. To resolve these problems we currently improve the timeliness of some existing short-term 

indicators, like consumption. Furthermore, in addition to the BCT, a factsheet is developed consisting of financial information and some fast short-term indicators about traffic volumes at transportation hubs (harbors and 

airports) and production volumes at essential parts of Dutch industry and commercial services. some examples. 

The results will be shown at the presentation.

Government Policy Strategies over the Current Business Cycle: Should New Business Start-Ups or 

Existing Businesses Expansion Be Favored

Ting Zhang and David Stevens (University of Baltimore, USA)

The worsening recession makes it critical for all levels of government to adopt strategic business development 

policies. The question addressed in this paper is: Do new business start-ups create more jobs than growth of 

current businesses? We use a longitudinal file of Maryland Quarterly Census of Employment and Wages (QCEW) 

administrative data to examine this question. Previous literature has investigated the employment impact of 

establishment (or firm) size with controversies. One challenge has been the chosen definition of establishment 

(or firm) size. Our paper responds to this challenge using longitudinal measures and tests the importance of 

establishment (or firm) age as a hypothetically complementary predictor of employment growth. The employment impacts of establishment (or firm) age and size are investigated in the context of local unemployment 

rates and industry sectors. We use a multistage panel data regression model with autocorrelation correction 

to test the roles of establishment (or firm) age and size as job growth engines. Administrative record quality 

considerations, as well as the rationale for using the above statistical model, are discussed. The paper 

concludes with policy implications and future research directions.

CONCURRENT SESSION XI-C:

EFFECTS OF QUESTIONNAIRE AND SURVEY DESIGN ON RECALL AND RESPONSE

Establishment Respondents as Survey Managers: Using Survey Design Features to Empower 

Respondents to Find and Recruit Knowledgeable Company Personnel for Assistance

Alfred Tuttle (U.S. Census Bureau)

The new Business R&D and Innovation Survey (BRDIS) replaces a previous survey in existence since 1957. In 

2009 it was pilot-tested in a dress rehearsal with the full sample of 40,000 U.S. for-profit companies. The BRDIS 

represents a substantial expansion over its predecessor in terms of types of information it collects. BRDIS 

requests information related to R&D finance and strategy, intellectual property, human resources, and R&D contracts, requiring specialized knowledge beyond that possessed by the finance or accounting area respondents 

typically responsible for the previous survey. During pre-testing of the questionnaire, we learned of difficulties often faced by respondents at multi-unit companies in 1) understanding the appropriate company personnel to provide the requested types of information, 2) finding those people among the myriad company units, 

and 3) obtaining the assistance of those people. The BRDIS questionnaire provides explicit instructions for a 

“survey coordinator” to assist in understanding the types of data requested and the need to obtain assistance 

from personnel with specialized knowledge, and the company units in which those personnel are likely to be 

found. These and other questionnaire design features are intended to encourage respondents to find specialized assistance rather than complete certain sections of the form themselves, and to address the concerns of 

other company personnel, in order to maximize the participation of appropriate company personnel. This paper 

will offer some preliminary findings from the pilot test evaluating the effectiveness of various design features 

in empowering respondents in this way, based on results from respondent debriefings and initial analysis of 

reported data. We will also use the findings to evaluate Willimack’s (2007) organizational model of establishment survey response.

Using Negatively Framed Questions to Evaluate Nursing Home Care

Elizabeth Frentzel (American Institutes for Research, USA), Judith Sangl (Agency for Healthcare Research and 

Quality, USA), Julie Brown (Rand, USA), Carol Cosenza (University of Massachusetts, USA), and Chris Evensen 

and San Keller (American Institutes for Research, USA)

Research Topic: The CAHPS® Nursing Home Survey-Family Member Instrument consists of 51 questions, with 

nine questions being were negatively framed (NF) items; it is unclear if NF questions induce respondent bias. 

Data Sources: The Agency for Healthcare Research and Quality and the CAHPS team developed a nursing 
53

home survey for family members based on a literature review, focus groups and expert input. We revised the 

survey based on cognitive testing. The survey was piloted with 1,444 family members of residents of 15 

nursing homes. 

Methods: We developed a draft survey with 88 questions and 15 NF questions. The survey was cognitively 

tested to assess content, comprehension, recall, and flow, resulting in a decrease to 13 NF questions. A pilot 

survey using mail with phone-follow-up with family members of residents (who had been in the nursing home 

at least 30 days) was conducted in October 2006 through February 2007. We evaluated item internal consistency, nursing home-level reliability, intent of the items, and qualitative information from consumers and other 

stakeholders to determine which items should be included in the final survey. After analyzing the pilot data, 

nine NF questions remained.

Conclusions: Our testing concluded that using NF questions, balanced with positive questions, can effectively 

evaluate the care of nursing homes. Focus groups and cognitive testing helped to determine the value of the 

items because respondents provided feedback to indicate item importance, comprehension, and item response 

retrieval. Industry was concerned that the NF questions would bias the respondent to respond negatively. 

Statistical analyses based on the pilot study further refined the survey by determining which items were not 

useful because of low levels of reliability and confirmed the value of the NF questions.

The Effects of Recall Length and Reporting Aids on Household Reporting of Health Care Events in 

the Medical Expenditure Panel Survey

Samuel Zuvekas (Agency for Healthcare Research and Quality, USA)

The Medical Expenditure Panel Survey (MEPS) is a widely used nationally representative survey of the levels 

and determinants of health care use and spending by U.S. households. Although the MEPS uses a number of 

procedures to enhance household reports of use, concerns remain about their completeness. Zuvekas and Olin 

(under review) found that Medicare beneficiaries in MEPS households accurately reported inpatient hospital 

stays, but that all socioeconomic groups systematically underreported office-based and emergency department 

visits. We build on this previous study to examine the effect that length of recall period and the use of reporting 

aids have on the accuracy of household reporting. 

The data are drawn from a sample of Medicare fee-for-service beneficiaries in the 2001–2003 MEPS who were 

matched to their Medicare claims and enrollment files under a Data Use Agreement with the Centers for Medicare and Medicaid Services (CMS). A propensity-score re-weighting procedure was used to further adjust MEPS 

sampling weights for differential non-response and matching. We compare the number of Medicare covered 

office visits and emergency department visits reported by respondents to the number of visits recorded in the 

Medicare claims for each of the five rounds of MEPS interviews (n=6927 person-rounds). We estimate Kappa 

and Lin’s concordance statistics in bivariate analyses of reporting accuracy by length of recall period, interview 

round, and whether the household used calendars, diaries, and billing paperwork in MEPS interviews. We also 

estimate multivariate logistic regression models of the probability of under- (and over-) reporting, controlling 

also for a number of sociodemographic characteristics. All analyses adjust for the complex design of the MEPS 

using the method of balanced repeated replication. Preliminary results suggest that reporting accuracy declines 

with length of recall period and there are positive effects of some recall aids. 

Response Order Effect and Social Desirability Bias in the New Dutch Security Monitor

Thomas Kraan, Jan van den Brakel, Bart Buelens, and Harry Huys (Statistics Netherlands)

In the Netherlands, perceived and factual security is measured by the Security Monitor (SM), a survey which is 

performed nationwide with a net sample size of about 20,000 respondents. Local authorities collect, parallel 

to the SM, related data on these topics on a regional level, which gives rise to inconsistent data about security feelings and crime victimization. Therefore it was decided in 2008 to combine the SM and these regional 

surveys into one Integrated Security Monitor (ISM). Next to other differences between the methodologies of the 

two surveys, the ISM employs web survey and paper and pencil interviewing in addition to computer assisted 

personal interviewing and computer assisted telephone interviewing used by the SM. In Autumn 2008, the ISM 

was performed for the first time with a net sample size of 62,803 respondents. To assess the effect of the modified methodology, in parallel the SM has been executed with a limited net sample size of 6,113 respondents. 

Comparison of ISM with SM results reveals that the survey transition has caused significant discontinuities. For 

attitude questions related to perceived security (evaluation of the environment and the police) these can be 

traced back in part to the different impacts of mode effects as a result of the different sets of response modes 

employed in the two surveys. In particular, the differing influences of the recency effect and social desirability 

bias and their interaction are important in this tentative explanation of the discontinuities as a consequence of 

the survey transition from SM to ISM.
54
55
56
57

NAME SESSION

Katharine Abraham Keynote Speaker

Denise Abreu X-C

Tamara Adams Technical Demo

Mark Asiala V-C 

Dale Atkinson X-A

Wendy Barboza X-C

Nancy Bates VII-A

Dan Beckler III-B

David Beede XI-B

Michael Bellow X-C

Gary Benedetto VII-C

Jeffrey Beranek III-B

Marco Better XI-A

Barbara Bibb Technical Demo

Stacey Bielick IV-C

Scott Boggess V-C

John Boland VI-B

Jonaki Bose X-B

Michel Boudreaux VII-B

Pat Dean Brick II-A

Julie Brown XI-C

Bart Buelens XI-C

Vicki Burt V-B

James Buszuwski Technical Demo

Maryan Cammarata V-B

David Cantor II-A

Lisa Carley-Baxter II-A

James Carr II-A

Kari Carris IX-C

George Carter III VII-B

Rachel Caspar II-A, VII-A

Patrizia Cella VI-B

Chris Chapman X-B

David Chesser VIII-B

Moon Jung Cho VI-C

Colleen Clark III-C

Stephen Clark IV-C

Richard Clayton IV-B

Paul Clemm Technical Demo

Carol Comisarow VII-A

Frederick Conrad II-A, IX-B

Kennon Copeland VII-C

Stephen Cornman III-B, VI-A

Theresa Cosca IV-B

Carol Cosenza XI-C

Mick Couper IX-B

NAME SESSION 

Lawrence Cox III-A, XI-A

Lester Curtin V-C

John Czajka VII-B

Ali Dadpay VIII-B

Brian Dahlin VIII-B

Ramesh Dandekar III-A, VI-A

Kara Daniel X-C

Filipponi Danila VI-B

Michael Davern IX-A

Paul Davies V-A

Charles Day XI-A

Angela Debello IX-C

Abdellatif Demnati VI-C

Gabrielle Denmead VII-B

Don Dillman IX-B

John Dixon VIII-A

Sebastien Djienouassi IV-A

Sylvia Dohrmann III-A

Irena Dushi V-A

John Eltinge VI-C, VII-C, IX-A

Alissa Emmel IV-B

Lawrence Ernst VI-C

Chris Evensen XI-C

Frank Farance X-A

Robert Fay II-A

Marco Ferreira II-C

Stan Freedman IX-B

Elizabeth Frentzel XI-C

Sirius Fuller V-C

Jenna Fulton IX-A

Guiseppe Garofalo VI-B

B. Dale Garrett V-C

Gerald Gates IX-A

Joseph Gfroerer X-B

Bula Ghose VII-C

Daniel Gillman X-A

Claude Girard VI-C

Serge Godbout VIII-A

Lee Goldberg Technical Demo

Jeffrey Gonzalez VII-C

Alfred Gottschalck VII-B

Charles Griffiths IV-A

Patricia Guenther VI-B

Paul Guerino VII-A

Charisse Gulosino II-C

Mary Hagedorn X-B

FCSM RESEARCH CONFERENCE AUTHOR AND COAUTHOR INDEX
58

NAME SESSION

Charlie Han VIII-B

Mark Harris X-C

Virginia Harris III-B

Dominique Haughton VI-B

Wen-Chau Haung IX-C

Michael Hawes VII-A

Michael Hayes VIII-A

Richard Heman-Ackah III-C

Steven Hill V-B

Stan Hoge X-C

Scott Holan VIII-B

Steven Honegger III-B

Antony Homan VIII-B

Irene Hudson VI-B

Harry Huys XI-C

Howard Iams V-A

Vince Iannacchione II-A

Michael Ikeda V-C

William Iwig III-B, X-C

Thomas Jacob X-A

Majnu John XI-A

Frank Johnson III-B

Susanne Johnson Technical Demo

Carey Johnston IX-C

Kimball Jonas V-C

Carrie Jones IV-C

Michael Jones X-B

WenYen Juan VI-B

Lynn Karoly V-A

San Keller XI-C

Joel Kennet X-B

Beethika Khan XI-B

Meena Khare III-C

Karen King V-C

Susan Kinsey III-C

Phillip Kott VI-B, X-C

Thomas Kraan XI-C

Sibylle Kranz VI-B

Tom Krenzke III-A, IX-C

Peter Kwok IX-C

Partha Lahiri II-C

Michael Larsen II-C

Lisa Lee II-A

Michael Levin V-B

Huilin Li II-C

Jianzhu Li II-A

NAME SESSION

Dan Liao VI-B

Youlian Liao IX-C

Kimberly Lochner III-B

Victoria Lynch VII-B

Stephen Mack IV-C

Kelly Maguire IV-A

Aaron Maitland Technical Demo

Laurence Mapondjou IV-A

Annie Louise Marshall IV-B

Shelley Wilkie Martinez IX-A

Madhu Mazumdar XI-A

Jaki McCarthy X-A

Stuart McDonald II-B

Tucker McElroy VIII-B

Cameron McPhee IV-C

Erik Meijer V-A

Peter Meyer IV-B, Technical Demo

Justin Mezetin Technical Demo

Pierre-Carl Michaud V-A

G. Edward Miller V-B

Kristen Miller Technical Demo

Lisa Mirel V-B

Krish Muralidhar III-A, VI-A

Joe Murphy Technical Demo

Padraic Murphy IX-C

Whitney Murphy IX-C

Susan Myers Technical Demo

Alfredo Navarro V-C

Charles Nelson V-C, VII-C

Phong Nguyen VI-B

Amber Noel III-B

Kenneth Notis II-B

Sharon O’Donnell II-B

Colm O’Muircheartaigh IX-C

Grace O’Neill IX-B

Laura Ozcoskun VIII-A

Dicy Painter X-B

Elsie Pamuk III-B

Yuling Pan VII-A

Van Parsons III-B

Joanne Pascale VIII-A

Emilia Peytcheva II-A

Lan Pham VII-C

Andrea Piani V-C

Valena Plisko X-A

Boris Rachev IX-B
59

NAME SESSION

Aroua Rachid II-B 

Michael Rand X-B

J.N.K. Rao VI-C

Ana Rappold II-C

Cleo Redline IX-B

Heather Ridolfo Technical Demo

J. Gregory Robinson V-C

Shep Roey III-A

Reid Rottach IX-C

Kalman Rupp V-A

J. Neil Russell III-A

Ben Jabeur Sami II-B

Judith Sangl XI-C

Rathindra Sarathy III-A, VI-A

Nathaniel Schenker III-B

Fritz Scheuren II-A, VII-C

Marilyn Seastrom VI-A, X-A

Hee-Choon Shin IX-C

Brian Sinclair-James VII-C

James Singleton V-C

Marla Smith IX-C

Philip Smith V-C

Jolene Smyth IX-B

Rachel Soloveichik II-B

Howard Speizer III-C

George Stamas IV-B

Michael Starsinic V-C

David Stevens XI-B

Matthew Stevens III-C

Martha Stinson VII-C

Alexander Strand V-A

Christopher Stringer V-C

NAME SESSION

Michail Sverchkov VI-C

M. Rita Thissen III-C

La Toya Thomas VIII-A

Randall Thomas IX-B

Roger Tourangeau II-A, IX-B

Joan Turek VII-C

Joanna Turner VII-B

Alfred Tuttle XI-C

Jan van den Brakel XI-C

Pieter Vlag XI-B

James Wagner IV-C

David Wagstaff VI-B

Edward Welniak VII-C

Andrew White VI-A

Stephanie Willson Technical Demo

John Wohlford V-C

Karen Woodrow-Lafield IV-A

Abera Wouhib III-C

Ke Yang IV-A

Cong Ye IX-B

Sungjin Youn VIII-A

Peg Young II-B

Ramses Zang IV-A

Cindy Zhang V-B

Ting Zhang XI-B

Zhiwei Zhang II-A

Lei Zhou III-B

Xiaoneng Zhu VIII-B

Marc Zodet V-B

Samuel Zuvekas XI-C
60

Reports Available in the Statistical Policy Working Paper Series

1. Report on Statistics for Allocation of Funds, 1978 (NTIS PB86-211521/AS)

2. Report on Statistical Disclosure and Disclosure-Avoidance Techniques, 1978 (NTIS PB86-211539/AS)

3. An Error Profile: Employment as Measured by the Current Population Survey, 1978 (NTIS PB86-214269/AS)

4. Glossary of Nonsampling Error Terms: An Illustration of a Semantic Problem in Statistics, 1978 

 (NTIS PB86-211547/AS)

5. Report on Exact and Statistical Matching Techniques, 1980 (NTIS PB86-215829/AS)

6. Report on Statistical Uses of Administrative Records, 1980 (NTIS PB86-214285/AS)

7. An Interagency Review of Time-Series Revision Policies, 1982 (NTIS PB86-232451/AS)

8. Statistical Interagency Agreements, 1982 (NTIS PB86-230570/AS)

9. Contracting for Surveys, 1983 (NTIS PB83-233148)

10. Approaches to Developing Questionnaires, 1983 (NTIS PB84-105055)

11. A Review of Industry Coding Systems, 1984 (NTIS PB84-135276)

12. The Role of Telephone Data Collection in Federal Statistics, 1984 (NTIS PB85-105971)

13. Federal Longitudinal Surveys, 1986 (NTIS PB86-139730)

14. Workshop on Statistical Uses of Microcomputers in Federal Agencies, 1987 (NTIS PB87-166393)

15. Quality in Establishment Surveys, 1988 (NTIS PB88-232921)

16. A Comparative Study of Reporting Units in Selected Employer Data Systems, 1990 (NTIS PB90-205238)

17. Survey Coverage, 1990 (NTIS PB90-205246)

18. Data Editing in Federal Statistical Agencies, 1990 (NTIS PB90-205253)

19. Computer Assisted Survey Information Collection, 1990 (NTIS PB90-205261)

20. Seminar on Quality of Federal Data, 1991 (NTIS PB91-142414)

21. Indirect Estimators in Federal Programs, 1993 (NTIS PB93-209294)

22. Report on Statistical Disclosure Limitation Methodology, 1994 (NTIS PB94-165305)

23. Seminar on New Directions in Statistical Methodology, 1995 (NTIS PB95-182978)

24. Electronic Dissemination of Statistical Data, 1995 (NTIS PB96-121629)

25. Data Editing Workshop and Exposition, 1996 (NTIS PB97-104624)

26. Seminar on Statistical Methodology in the Public Service, 1997 (NTIS PB97-162580)

27. Training for the Future: Addressing Tomorrow’s Survey Tasks, 1998 (NTIS PB99-102576)

28. Seminar on Interagency Coordination and Cooperation, 1999 (NTIS PB99-132029)

29. Federal Committee on Statistical Methodology Research Conference (Conference Papers), 

 1999 (NTIS PB99-166795)

30. 1999 Federal Committee on Statistical Methodology Research Conference: Complete Proceedings, 2000 

 (NTIS PB2000-105886)

31. Measuring and Reporting Sources of Error in Surveys, 2001 (NTIS PB2001-104329)

32. Seminar on Integrating Federal Statistical Information and Processes, 2001(NTIS PB2001-104626)

33. Seminar on the Funding Opportunity in Survey Research, 2001 (NTIS PB2001-108851)

34. Federal Committee on Statistical Methodology Research Conference (Conference Papers), 2001 

 (NTIS PB2002-100103)

35. Seminar on Challenges to the Federal Statistical System in Fostering Access to Statistics, 2002

36. Seminar on the Funding Opportunity in Survey Research, 2003

37. Federal Committee on Statistical Methodology Research Conference (Conference Papers), 2003
61

38. Summary Report of the FCSM-GSS Workshop on Web-Based Data Collection, 2004

39. Federal Committee on Statistical Methodology Research Conference (Conference Papers), 2005

40. Federal Committee on Statistical Methodology Research Conference (Conference Papers), 2007

Copies of these working papers may be ordered from NTIS Document Sales, 5285 Port Royal Road, Springfield, 

VA 22161; Telephone: 1-800-553-6847. The Statistical Policy Working Paper series is also available electronically from FCSM’s Web site <http://www.fcsm.gov>. 
62

 NOTES
63

 NOTES
ACSD/07-10645

Issued November 2009

e on Statistical Methodology Research Conference Final Program and Abstract Booklet November 2–4, 2009
65

NOTES