# Presentation: 155857

## Workshop 4cData Collection

- Stephanie J. Battles
- Energy Information Administration
- Dhaka, Bangladesh
- September 22, 1999

## Data Collection

- Mail Survey
- Personal Interview
- Telephone Interview: Computer-Assisted Telephone Survey (CATI)
- Other Modes of Data Collection
- Multiple Modes of Data Collection
- Survey Supplement Using A Subsample
- Nonsampling Error
- Nonresponse
- Response

## Mail Survey(Self-Reported Data)

- Advantages
- Cost Effectiveness
- Small Staff
- Same Mailing Cost for 1 Question as it Costs for 100
- Easily Target Population of Interest
- Studies Show That More Answers Are Provided

- Disadvantages
- Lower Response Rates
- Need to Have Good Planning
- Need Complete Mailing List
- No Opportunity for Follow-up
- Need Questions that are Simple and Short
- Legible/Easy to Follow
- Correct Language
- Can Respondent Read?

**Design and Format Crucial**
**Future--CSAQ**

## Mail SurveyExample: 1998 MECS

**Mail Survey***Example: 1998 MECS*

- Three Different Forms, Forms A,B, and C
- Form A: Collects Basic Data
- Form B: Petroleum Refining
- Form C: Establishments Who Ship Energy Offsite
- 18,000 Establishments
- Average Burden per Response--8.24 Hours
- U.S. Census Bureau--Contractor
- Two Mail Follow-Ups
- Additional Telephone Follow-Up for the Largest Establishments Not Responding

## Personal Interview(Face-to-Face)

- Advantages
- High Response Rates Are Possible
- Survey Questions Can be More Complex
- Opportunity for Follow-Up Questions
- Can Assist Respondents Who are Disabled, Aged or Other Problem

- Disadvantages
- Costly
- Need a Large Staff or Contract Personnel
- Natural Disaster
- Lose Portion of Sample
- May Not Get Person Most Capable of Answering the Questions
- Sample May Be in Restricted area

## Personal Interview (Face-to-Face)

- Interviewer Training (Classroom/Self-study/Both)
- Introductory Letter Before Initial Contact
- Purpose of the Survey
- Why Participation is Important
- Confidential Responses
- Exhibit Cards
- Multicontacts
- Callbacks at Different Times of the Day
- Multiwave
- Reassigned Different Interviewers
- Use of Incentives

## Personal Interview Example-1997 RECS

**All Training Sessions Were Monitored by EIA Staff**

**Personal Interview** *Example-1997 RECS*

- Trainer Training
- Three Days
- Interviewer Training
- Three Days (2 Sessions)/220 Interviewers
- Washington D.C.
- Topics Covered:
- General Interviewing Techniques
- RECS Background
- Introduction to the CAPI system
- Practice With Mock Interviews
- Review of Administrative Requirements

## Personal Interview Example-1997 RECS

**Personal Interview** *Example-1997 RECS*

- Training Manual
- Locating Exercises
- Identifying Respondents and Getting In the Door
- Mock Exercises (one person is the interviewer..the other is the respondent)
- Instruction Manual
- Administrative Procedures
- Identifying RECS Households
- Conducting the Interview
- Question by Question Discussion
- Training Used Computer (CAPI)
- Letter Sent Before Initial Contact

## Personal Interview Example-1997 RECS

**Personal Interview** *Example-1997 RECS*

- Data Collection Results
- Average Time: 29 Minutes
- Interviewers Conducted an Average of 27 Interviews
- Median Number of Visits--3
- 20% of the Interviews Were Were Verified by Mail or Telephone
- Used Multicontact Approach
- Different Time of Day
- Used Multiwave Approach
- Changed Interviewer
- Mail Follow-Ups for Nonresponsive Households
- Short  Self-Administered Questionnaire
- Sent Pen as an Incentive

## Personal Interview Example-1997 RECS

**Personal Interview** *Example-1997 RECS*

- Data Collection Response
- 8,319 Listed Units--7,285 Eligible
- Ineligible
- Out of Scope, e.g. Business, Not Habitable
- Vacant or Seasonal
- 5,902 Completed (81% Response Rate)
- 5% of the Personal Interviews Completed After the Second Wave (299)
- 3 % of the Interviews Completed After the Mail Follow-up
- 1,421 Households--Returned 10 % (181)

## Personal Interview Example-1997 RECS

**Response Rates and Household Characteristics**

**Personal Interview** *Example-1997 RECS*

- High Response Rates
- South (81.1%)
- Midwest (80.2%)
- Rural Areas (85%)
- Single Family and Mobile Homes (80.5%)

- Low Response Rates
- Northeast (73.1%)
- Urban/Suburbs (76.4%)
- Large Apartment Buildings (72.1%)

## Advantages
Increase Quality
Speed
Programmed to Correctly Skip Automatically
Edit Replies
Produce Statistics Quickly 
Eliminates the Keying of the Data

- Increase Quality
- Speed
- Programmed to Correctly Skip Automatically
- Edit Replies
- Produce Statistics Quickly
- Eliminates the Keying of the Data

- Disadvantages
- Costly
- Savings When the Survey is Large and/or Frequent and Questions Remain the Same
- Lower Response Rates
- Answering Machines
- No Telephone
- Up Front Costs
- Programming

**Most Telephone Surveys are CATI**** **
**(Computer-Assisted Telephone Interviews****)**

**Telephone Interview*****(CATI)***

## Other Modes of Data Collection

- Touchtone Data Entry (TDE)
- Alternative to Mail Collection
- Questionnaire is on a Computer
- Respondent Answers Prerecorded Questions Using the Touchtone Buttons
- Useful for a Few Numbers
- Eliminates Key Entry
- Voice Recognition (VR)
- Recognizes Numbers “0 to 9”, “Yes” and “No”
- Useful for A Few Numbers
- Accent Recognition Problem

**BLS: Current **
**Employment**
**Statistics**

**Banking**

## Other Modes of Data Collection

**Other Modes of Data Collection**

- Facsimile Transmission (FAX)
- Eliminates Mail Delays
- Electronic Data Interchange (EDI) Collect Large Volumes of Data
- Electronic Transmission is in a Standard Format
- Errors are Reduced
- Data are Instantly Available
- Email Transmission
- Paperless Collection
- Eliminates Mail Delays
- Errors/Costs Are Reduced
- Confidentiality Issues

**EIA’s Financial Reporting System**

## Multiple Modes of Data Collection

- Survey Requires the Collection of Data from Different Respondent
- Increase Survey Response Rates
- Respondents May Favor One Mode Over Another
- Certify Data Collected Are Correct
- Reduce Costs

## Multiple Modes of Data CollectionExample: 1993 RECS

- Personal Interview
- Household/Housing Unit Data
- Authorization Form
- Mail Survey
- Household Mail Survey
- Obtain Data for Households Who Refused to Participate in the Personal Interview
- Energy Supplier Survey
- Natural Gas, Electricity, LPG, and Fuel Oil
- Obtain Energy Consumption and Expenditure Data from the Household’s Energy Suppliers

## Multiple Modes of Data CollectionExample: 1993 RECS

- Telephone Survey
- Rental Agent Survey
- Verify Information Received From Certain Households Who Did not Pay for All Their Energy or Paid A Third Party and Rented or Owned Living Quarters in A Multi-Unit Building

## Survey Supplement Using A Subsample

- Obtain Detailed Data
- Use if Too Costly for the Full Sample
- Use if Response Burden is a Concern
- Use Data to Develop a Model Based on Data Available From the Full Survey

## Survey Supplement Using A SubsampleExample: 1993 RECS Lighting Supplement

- Purpose of the Supplement
- Collect Data to Develop a Model to Predict Electricity Consumption for Lighting
- Proportional Sample of All U.S. Households
- 7% of the Completed Interviews
- Lighting Model Used in Electricity Regression Equation Along Other End-Use Models, e.g. Space Heating.

## Nonsampling ErrorUnit Nonresponse Error

**Nonsampling Error*****Unit***** *****Nonresponse Error***

- Unit Nonresponse
- Refusal/Failure to Return Questionnaire
- Sensitive Information
- Incorrect Address
- Not Have Resources or Time To Reply
- Not Interested
- Not Like the Sponsor
- Could Not Be Located
- Natural Disaster
- Recently Vacant/Destroyed
- Telephone Disconnected

_**Unit Nonresponse Error**_**: Unit nonresponse error occurs when a sampling unit does not participate in the survey , e.g. household, building or establishment.**

## Nonsampling Error Unit Nonresponse :Control of Error

**Nonsampling Error**** *****Unit Nonresponse :******Control of Error***

- Sampling Unit’s Record Carefully Labeled and Transmitted
- Prior Notice
- Follow-Up
- Incentives
- Questionnaire Design
- Confidentially Ensured
- Reweighting the Sample

## Nonsampling ErrorItem Nonresponse Error

**Nonsampling Error*****Item***** *****Nonresponse Error***

- Sensitive Question
- Confidentiality Issue
- Recall
- Aged or Disabled
- Fatigue/Burden
- Question Long and Complex
- Open Question
- Records Not Available
- No Advance Notice
- Interviewer Error
- Programming Problems

**Item Nonresponse Error: Item nonresponse error occurs when a particular item of interest is missing from a questionnaire.**

## Nonsampling ErrorItem Nonresponse:Control of Error

- Interviewer Training
- Confidentiality Reminder
- Use Memory Aids and Cues
- Question Shorts and Simple
- Questionnaire-Reasonable Length
- Personal Interview, e.g. 60 Minutes
- Understandable Skip Patterns
- Limit Open Questions
- Imputation

## Nonsampling Error Response Error

**Nonsampling Error**** *****Response Error***

- Satisficing Answer
- Doesn’t Care/Time Burden
- Inability to Answer--A Guess
- Deliberate Incorrect Response
- Sensitive Question
- Respondent Knows the “Correct Answer”
- Invalid/Out of Date Records
- Interviewer Incorrectly Records Response
- Interviewer Bias

_**Response Error**_**: Response error occurs when the item **
**response is answered incorrectly.**

## Nonsampling ErrorResponse Error:Control of Error

- Importance of the Survey and Accurate Responses
- Use Memory Aids/Examples
- Ensure Confidentiality of Responses
- Advance Notice
- Interviewer Training
- Locate the Respondent With the Proper Knowledge
- Quality Control of the Interviewing Process
- Allow Respondents to Ask Questions