# Presentation: 337646

- This is a working document prepared by the Energy Information Administration (EIA) in order to solicit advice and comment on statistical matter from the American Statistical Association Committee on Energy Statistics. This topic will be discussed at EIA's spring 2006, meeting with the Committee to be held April 6 and 7, 2006.

**Report of the**
**Internet Data Collection Team **
**	****to the EIA Goal 4 Subcommittee**
**February 14, 2006**

## IDC Team Members 	

- Betty Barlow, OIT
- Audrey Corley, OOG
- John Paul Deley, NEIC
- Stan Freedman, SMG (Chair)
- Alethea Jennings, SMG (Asst. Chair)
- Tom Lorenz, EMEU
- Paul McArdle, OIAF
- Rick Salkov, CNEAF
- Paula Weir, OOG

## Introduction			

- Objectives
- Progress to Date
- Next Step for Project Completion
- Content of the Briefing

**Notes:**
Objectives – Develop a functional requirements document. The functional requirements document for the prototype will recognize and accommodate changing technology by focusing on what is required rather than describing how to build the prototype.

Progress to Date
Researched what other agencies have done
 BLS
Census
Agriculture
Blaise
EIA

Developed specific recommendations in 6 functional areas
Data Collection
Interoperability
Performance Measures
Retention & Disposition
Essential Support Services
Security

Next steps
Finalize the recommendations
Prepare best practices from the IDC systems we reviewed
Develop an implementation strategy
Write report

Content
Summarize what we found from our review of IDC systems
Review recommendations in each of the functional areas
Raise some questions that the Goal 4 subcommittee should consider when they get our report

## Visits to Statistical Agencies,  Private Providers, and EIA: Demonstrations and Interviews

- 11 broad questions sent in advance of visit
- Visits:  BLS, Census (2), NASS, Westat/Blaise (2), EIA (4)
- Main finding: three approaches:1) fully centralized, limited flexibility (ex. Census Taker for
-      sponsor surveys) 2) centralized core structure w/survey flexibility and/or control
-      (ex. BLS, NASS)3) fully de-centralized (i.e. not common) (ex. Census Decennial, EIA,
-      Westat customized)
- Other findings: Little evidence of cost savings; evidence of higher quality data; implementations are few thus far; often started with paper image but quickly moved on; take-up is low (other than in EIA); help support underestimated; organizational culture and buy-in is critical (What’s in it for me?)

## Recommendations: Data Collection (Graphical User Interface - Single Sign On)

- Users have one way to be authenticated and authorized for all surveys so that:
- 	1) Respondents will only need to remember one userid, password, etc.
- 	2) Far less EIA time will be spent helping respondents with login issues
- Users establish their own credentials and are given information to enroll for surveys
- Users can change contact information and gain access if they forget their passwords
- Users enroll once; further survey access is automated
- Survey systems provide updated respondent lists to IDC

**Notes:**
Authentication—are you who you say you are (login)
Authorization—what surveys are you allowed to access
EIA currently has an SSO for external users
Heard in interviews about experiences with using certificates to authenticate people:  
      secure but cumbersome and not “single”
We’re dealing in proprietary data, not secret or more

## Recommendations: Data Collection (Graphical User Interface - Common Look and Feel)

- Specify and build elements of a common look and feel for all surveys so that:
- 	1) Respondents will not have to relearn how GUI looks for each survey
- 	2) Respondents will always know they are on an EIA site
- 	3) We can reduce design and development costs
- Survey manager chooses what to use but uses the common look if item is chosen

**Notes:**
Examples:
GUI is the part most important to the user usually because it is what he sees and uses
Getting this right will go a long way toward user acceptance
Will provide style sheets or their equivalent with:
EIA website logo on all pages 
Header information
Fonts, colors
common respondent info page with known information filled in 
One place for common items:  Help, print, navigational symbols, buttons, progress bar
Choice to Use:
Way to specify required items
Progress bar
Will assure 508 compliance

## Recommendations: Data Collection (Graphical User Interface - Functionality)

- Specify GUI functionality to be used for all surveys so that:
- 	1) Respondents will not have to relearn how GUI functions for each survey
-    2) We can reduce design and development costs
- Survey managers have choice to use some functionalities; if they choose to use one, they do it in the specified way

**Notes:**
Examples:
Survey manager chooses whether main body of survey is on one sheet or several
If survey manager chooses several, will be one way to effect that (side menu, tabs, etc.)
Common meanings for terms:  save, submit, cancel, continue
Users can have several sessions before submitting
Respondent can retrieve historical data
Common way to import data from respondent’s computer system using a common file format
Common way to attach supplementary materials
Certain types of edits will be edited immediately and other types on a button press
Edit messages will be located near question  
Provide capability for resubmissions if survey manager allows it

## Recommendations: Data Collection (“Front End” Editing) 

**Prevent errors** through drop downs
- Provide capability for “**hard**” edits for defined fields —**must correct** (ex: numeric, integer, field length, required field)
- Provide capability for other “**hard**” edits defined by survey —**must correct or comment**
- Provide capability for “**soft**” edits **—can correct, comment or bypass**
- Provide capability for **various types of edits**: within cell, cell-to-cell within form, cell-to-cell historic, cell-to-cell other form/survey
- Provide ability to **implement edit rules** that include consistency, range, comparisons, etc., and may involve calculations (such as market share) and/or parameters that are easy to modify by survey manager

## Recommendations: Data Collection (User Notification)	

- Provide dual notification that official submission received by EIA—on-screen at submission, follow-up email message
- Provide non-response notification: no submission when expected, submission insufficient as defined by the survey
- Provide email (respondent or non-respondent) that can be targeted to sub-groups as defined by the survey
- Provide notification of data not saved/submitted if browser closed prior to that action.

## Recommendations: Interoperability

- IDC/Respondent Communication
- 	1) Respondent imports data into IDC or performs data entry
- 	2) Respondent can attach supplementary materials to submission
- 	3) IDC displays contact information and (possibly) historical submissions in browser
- IDC/Processing System Communication
- 	1) File can be exported from IDC to Processing system
- 	2) Processing group could choose to use IDC database directly
- 	3) Processing system passes respondent list and (possibly) revised respondent data back to IDC

**Notes:**
Two way communication in each of the two cases
1-1  If import chosen, respondent must use file structure specified by EIA (and this structure will be used for all surveys)
1-1  imported data is displayed in browser

2-1  files passed will have common structure to cut down on cost of this option

## Recommendations: Performance Measures 	

**Measures about the Survey**
- 	1) Response Measures: number submitted and number insufficient, subgroup counts (including import vs entry, and respondent entry vs internal entry), historic--response rates defined by survey
- 	 2) Edit Failure & Correction/Bypass Measures: counts/%/impact by cell, 	rule, group, hard and soft, historic 3) Corrections/Resubmissions Measures: counts and % by group, impact, reason type
**Measures about the Collection Mode** 1) Access counts (respondents and internal)
- 	2) TBD (change in respondent burden, respondent evaluation/usability, system inacessible, support costs, number of registered users, etc)
**Measures on Contribution of IDC Mode to Overall Survey Product Performance**
- 	1) TBD (comparison of cost per respondent, comparison costs to other modes, comparisons of response rates to other modes, “back-end” edit failure/call back/corrections comparisons to other modes, contribution to meeting/exceeding dissemination deadlines

## Recommendations: Retention and Disposition 	

- The IDC will have the capability of:
- Scheduling data retention for IDC data
- Placing an identifiable metadata tag on records
- Providing access to IDC data records using search criteria  (for authorized users only)

## Recommendations: Essential Support Services 	

- Purpose:  Services, activities, or products that the IDC must have to be effectively used by EIA and EIA respondents.
- Two help desks:  (1) maintained by USC to assist with systems problems/questions/concerns, (2) maintained by the program office to assist respondents with survey specific questions.
- Usability Testing:  all surveys must undergo usability testing with  (1) a sample of actual survey respondents, (2) internal EIA users.
- Documentation:  1) The IDC shall contain online survey instructions and user guides for the respondents.  2) Systems documentation and other internally used documentation shall be maintained in a central location for all of EIA.

**Notes:**
2) USC help desk support is not survey specific;
    USC passes customers on to the program office help center if the assistance needed is survey specific.  
    Other office support should be expected (ex: some respondents may call NEIC who could help with generic questions and otherwise refer the call to USC or the program     office)

## Recommendations: Security 	

- The IDC shall comply with the EIA Information Systems Security Policy.
- Data transmitted between the client and the EIA server shall be protected from interception and be readable only by the intended recipient.
- A Security and Privacy statement shall be available for viewing on the Logon page of SSO.
- Passwords shall be hashed (thus unreadable) in the database.
- The client and the EIA server shall be authenticated to one another.
- The IDC design shall ensure that individual work sessions are closed to unauthorized users from other workstations.

**Notes:**
6)  purpose:  so that data cannot be intentionally or unintentionally tampered with

## Recommendations: Overview

- Recommend: Follow 4 basic guiding principles supported by detailed requirements.  The IDC:
- 	1) is designed so that it is **easy** to implement, modify, or migrate a survey to the IDC2) provides tools enabling the survey group to have **ownership of their survey application**3) provides **value added from the respondent’s perspective** (answers “What is in it for me?”)4) promotes **high quality data** through editing features, user notifications, clear navigation, etc
- Recommend: Form a **partnership** of OIT and the program offices where OIT develops and maintains the **overall IDC infrastructure and provides tools for program offices to implement/customize the application** and OIT assists in customer help through a **help center**
- Recommend: Initiate a pilot for a common EIA IDC

## Points for Consideration by the Goal 4 Subcommittee

- Coordination with other related efforts
- Will the IDC extend beyond what we normally think of as our surveys?
- Should we anticipate planning for the implementation of a pilot project in FY2007?