The Process of NAS Panels

David Betson

Presentation at CE Workshop

October 16, 2012
Initiation of the Project

• Projects undertaken by the NRC are initiated 

by federal agencies to pursue a question they 

have or have been directed by Congress to 

answer

• The importance of the “Charge to the Panel”

• Selection of Study Director
Selection of the Panel of Experts

• Nomination of Chair

• Nomination of Panel Members to create 

diversity of expertise and perspective

• Confirmation and Appointment of Chair and 

Panel Members
The Panel on Redesigning of the CE

The Panel was composed of 13 members representing a 

variety of disciplines and skills 

– Sample design

– Data collection and technology

– Survey measurement and cognitive design

– Economics and Public Policy

– Past direct involvement with the CE’s design and 

implementation

– Users of the CE
The CE Panel on Redesigning the BLS 

Consumer Expenditure Surveys

• Don Dillman, Department of Sociology, 

Washington State University (Chair)

• David Betson, College of Arts and Letters, 

University of Notre Dame

• Mick Couper, Institute for Social Research, 

University of Michigan

• Robert Gillingham, Independent 

Consultant, Potomac Falls, VA

• Michael Link, The Nielsen Company, 

Marietta, GA

• Bruce Meyer, Harris School of Public Policy 

Studies, University of Chicago

• Sarah Nusser, Department of Statistics, 

Iowa State University

• Andy Peytchev, RTI International, Research 

Triangle Park, NC

• Mark Pierzchala, Independent Consultant, 

Rockville, MD

• Robert Santos, The Urban Institute, 

Washington, DC

• Michael Schober, New School for Social 

Research, New York City, NY

• Melvin Stephens, Jr. Department of 

Economics, University of Michigan

• Clyde Tucker, Independent Consultant, 

Vienna, VA

• Carol House, Study Director, Committee on 

National Statistics
Bias and Conflict of Interest

• Once a year, Panel Members are required to complete 

‘Conflict of Interest Statements’

• At the initial meeting of the Panel, a discussion is held 

where Panel Members discuss any potential biases and 

conflicts of interest they may affect how their views on 

the Panel’s work. At this meeting, the Panel decides 

whether additional Panel Members may be needed.

• While travel expenses are paid, the Panel Members 

volunteer their time and are otherwise not compensated
Gathering Information and Evidence

• Open Meetings – Presentations and Discussion (February 8, 

2011) 

• Workshops or Conferences

– Household Survey Producer Workshop (June 1 and 2, 2011) 

– Redesign Options Workshop (October 26 and 27, 2011)

• Panel Activities

– Survey of Existing Literature 

– Data Collection and Analysis

– Outreach (attended CE User’s Workshop)

– Gaining Experience (attempt to answer the CE survey)
Arriving at a Consensus Report

Closed Meetings, Emails, Web Conferences

First Meeting – February 9, 2011

Second Meeting after first workshop – June 3, 2011

Third Meeting after second workshop – October 29, 2011

 A lot of emails and phone calls, drafting of report

Fourth Meeting – January 25 and 26, 2012

Web Conference Call – March 2, 2012

Report sent to Review – June 8, 2012
What is a Consensus Report?

The set of conclusions and recommendations that address 

the questions posed in the Panel’s charge that each Panel 

Member can ‘live with’ but don’t necessarily represent their 

most preferred set of conclusions and recommendations.

‘Live with’ in the context of a NAS Panel means that in the 

Panel’s expert judgment, the Panel’s recommendations are 

supported by the best available evidence.
Report Review

• NAS appoints a Review Monitor and a Review Coordinator to act as 

‘editors’ in the review process

• Reviewers are selected by NAS to review the report to insure that 

the report is responsive to the Panel’s Charge and that all 

conclusions and recommendations are supported by evidence

• The Study Director and Chair with the assistance of the Panel 

prepare a response to review where reviewers comments are 

addressed by making changes to the text or providing an 

explanation of why no change is needed

• The Review Monitor and Coordinator must sign off on the adequacy 

of the Response to Review prepared by the Panel
Release of Report

• Sponsors are provided a prepublication version 

of the report and are briefed by Panel Chair 

and Staff Director

• Public Release of Report 

– NAS Web Site

– Public Briefing