Editing for the EIA-906

[Background]{.underline}

> The contractor will develop, evaluate and implement two categories of
> edits for the revised form EIA-906. They are on-line, respondent level
> edits for the revised form EIA-906, and summary-level edits to be
> reviewed by Electric Power Division survey staff prior to publication.
> The contractor will also evaluate the revised browser-based EIA-906
> and make suggestions for improving the form's architecture and user
> interface.

1.  [Respondent-Level Edits]{.underline}

These edits would be available to respondents as they fill in the
on-line version of the EIA-906. (If respondents submit paper forms,
Electric Power Division staff would use the on-line version of the
EIA-906 as the data entry screen.) These edits would alert respondents
(and data entry personnel) to potential errors in their data prior to
submission of the data to EIA. Examples of these types of edits might
include, but not be limited to:

a.  Changes of respondent ID, name address and/or contact person. The
    system might ask respondent to verify change name, address, or
    contact person -- and incorporate changes, if needed. (Both old and
    new information must remain available.)

b.  Missing data checks -- are all the necessary fields completed

> c\. Zero -- non-zero:

-   Are fields that were zero in last month's report still zero?

-   Are fields that were non zero in last month's report still non-zero?

> d\. Consistency checks -- are responses logically consistent within a
> form
>
> e\. Reasonableness checks -- are responses within an expected range
>
> f\. Historical checks: Are there "large" changes from the previous
> period's response or from an exponentially weighted average of
> respondent's past data? ("Large" may be defined based on exponentially
> weighted mean absolute deviation of respondent's past data.)

With the input of EIA's subject matter experts and survey
methodologists, the contractor will develop a list of the edits proposed
for the browser- based EIA-906. This list may include current EIA-906
edits. The contractor will evaluate the edits by running them on current
EIA-906 data (perhaps augmented with outliers) to provide the
information needed to select between competing edits, and/or to
determine edit parameters, and based on this information will recommend
a set of edits to be incorporated into the EIA-906 system. The
evaluation report and contractor recommendations will be used by EIA to
select the edits and edit parameters to be incorporated into the EIA-906
browser based form.

The contractor will recommend error messages to be associated with the
edits selected for inclusion into the browser-based EIA-906 form and
will suggest the manner in which edit failures will be displayed to
respondent. Error messages should be clear and self-explanatory to the
respondents, and the respondents should have the capability to override
the edit flags and to provide an explanation for the override. The edit
history, overrides, and explanatory notes shall be available to EIA-906
survey staff when they review the data after the form has been submitted
to EIA.

The contractor will evaluate the EIA-906 and suggest changes in the
forms architecture and user interface with the goal of making the survey
more "user friendly" (i.e., clearer and less burdensome to the
respondent). Integral to this is how the respondent level edits will
appear and how the respondent will use them.

The contractor will advise and participate in the testing of the
selected respondent-level edits after they have been implemented into
the EIA-906 processing system.

2.  [Summary Edits]{.underline}

> Summary level edits are edits that rely on individual respondent level
> data from all (or many) respondents for the same reference month. They
> are run once a sufficient number of respondents have submitted data to
> EIA so that meaningful aggregates can be generated, and the data have
> been processed, adjusted, and imputed, if necessary. These edits are
> not available to respondents but are available to survey staff and
> analysts conducting quality reviews of the data prior to publication.
> This analysis will be used by Electric Power Division staff to
> identify unusual responses or "outliers" in the data, and other data
> problems. Examples of summary edits might include, but not be limited
> to:

a.  Time series comparisons of generation and fuel used to produce
    electricity to see if they are moving in the same direction if the
    proportions remain consistent.

b.  Consistency between similar types of respondents

c.  Comparisons with other surveys

d.  Proportion of volumetric contribution to geographic totals (market
    share)

e.  Evaluating regression relationships, and plot them -- such as
    regressing generation on fuel used to generate electricity and
    identifying outliers

f.  Develop other graphical procedures such as biplots, which display
    variables and observations on the same plots.

> With the input of EIA's subject matter experts and survey
> methodologists, the contractor will develop a list of summary edits
> proposed for inclusion in the EIA-906 processing system. This list may
> include some of the current EIA-906 edits. The contractor will
> evaluate edits by running them on current EIA-906 data to provide the
> information needed to select edit parameters, and to select which
> edits to include in the system. Once edits have been selected, the
> contractor will propose how the selected summary edits can be
> integrated into the Electric Power Division's EIA-906 processing
> system most easily.

.

> Error messages should be clear and self-explanatory to survey staff,
> and the survey staff should have the capability to override the edit
> flags, and to select from a few informative codes to explain the
> override. The edit history, overrides, and explanatory notes shall be
> available to EIA-906 survey staff even after the data have been
> published.
>
> The contractor will advise and participate in the testing of the
> summary level edits once thy have been implemented within the EIA-906
> processing system.
>
> Deliverable Time line

# [Step]{.underline} [Start Date]{.underline} [Stop Date]{.underline}

1\. Respondent Edits June 30, 2003 August 15, 2003

a.  Develop list of

> Respondent-level edits.

b.  Conduct evaluations August 1 August 30

> of respondent level edits
>
> using current 906 data.
>
> Deliver a report of the
>
> Evaluations and a
>
> specifications document
>
> that describes the edits
>
> and details the methodology
>
> used to construct those edits.
>
> c\. Participate in and advise August 1 September 30
>
> EIA on how to incorporate
>
> respondent edits into the
>
> EIA-906 dissemination and
>
> processing system. Document
>
> suggestions.

d.  Participate in the testing October 31 November 26

> (possibly with EIA-906
>
> respondents) and

demonstration of the

browser-based EIA-906

protototype. Deliver test

report.

e\. Participate in the final December 1 December 31, 2003

> revisions and testing.
>
> Deliver report of this.

1.  Summary Edits:

    a.  Develop list of summary June 30 August 15

> level edits.

b.  Conduct evaluations of August 1 August 30

> summary level edits using
>
> current 906 data.
>
> Deliver an evaluation report
>
> c\. Participate in and advise EIA August 1 September 30
>
> on how to incorporate
>
> summary level editing
>
> system into EIA-906. Deliver
>
> documentation on how
>
> summary edits will be
>
> incorporated into the
>
> EIA-906 system.
>
> The report will detail the
>
> specifications methodology
>
> used to construct those edits.

d.  Participate in testing and November 1 November 26

> demonstration of
>
> summary level edit for
>
> EIA-906 survey staff.
>
> e\. Participate in final December 1 December 31
>
> testing and revision.
>
> Deliver final product.

2.  Evaluation of EIA-906 survey.

    a.  Evaluate the draft EIA-906.

> Deliver a report outlining

suggestions for changes in

screen architecture and user

interface.