1 

Achieving Information Quality via Continuous Quality Improvement

1

Shawna Waugh 

U.S. Energy Information Administration

Shawna.Waugh@eia.gov

Executive Summary: This paper focuses on a recent initiative to redesign the Petroleum Marketing Program 

(PMP), using a hybrid framework for information quality and data quality and the Continuous Quality Improvement 

(CQI) cycle. A CQI cycle was implemented to achieve data and information quality in the Office of Petroleum and 

Biofuels Statistics (PBS) of the U.S. Energy Information Administration (EIA). The CQI cycle consists of four 

stages - plan, implement, monitor and evaluate. 

When designing a new survey or program, one starts with the planning phase, however for existing programs the 

first step is the evaluation phase. An evaluation was conducted in 2011 of the PMP to identify and select 

recommendations to implement in the 2013 PMP during the Office of Management and Budget (OMB) forms 

clearance process. Many of the recommendations identified from several evaluations have been adopted during the 

planning phase, in preparation for implementation of the program in 2013. Additional recommendations may be 

implemented in the future as the CQI process continues from one survey cycle to the next one. 

PBS manages two information data collections – the PMP and the Petroleum Supply Program (PSP). Combined 

these two programs provide weekly, monthly and annual statistics pertaining to petroleum supply, demand, and 

price, including prices for crude oil and petroleum products in the United States.

A comprehensive review of the entire program is conducted every three years during the Office of Management and 

Budget (OMB) forms clearance process. This process started with compiling, reviewing, evaluating and prioritizing 

recommendations intended to enhance information quality and data quality. A review was conducted of recent and 

previous reports of the program and associated surveys, both internal and external. The review included reports from 

outreach activities with internal and external customers, internal self-assessments by survey managers and external 

evaluations of the program, including peer review of the survey questionnaires. 

A hybrid framework combining existing information quality and data quality frameworks was used to sort 

recommendations into several dimensions of quality related to the program, to the products and to the surveys. 

OMB’s Information Quality Guidelines was used to sort recommendations pertaining to the program. The seven 

dimensions of information quality introduced by Gordon Brackstone from Statistics Canada were used to sort 

recommendations pertaining to publications. The framework from OMB’s Statistical Policy Working Paper 

Number 31, Measuring and Reporting Sources of Error in Surveys, was used to sort recommendations pertaining to 

sampling and nonsampling errors.

A team of survey managers, methodologists and contractors evaluated which of the 200 potential recommendations

to adopt, of which many would have been adopted had resources not been a constraint. Forty of the

recommendations were adopted, many of which involved modifications to the survey questionnaire and instructions 

to enhance consistency across surveys in the program.

There are several challenges when planning and implementing the 2013 PMP. One challenge involves integrating 

ten surveys into a comprehensive program and another integrating this program with the PSP which provides an 

overview of the petroleum flow in the United States. Another challenge is providing publications for a variety of 

customers - policy makers and analysts with federal/state/local government agencies, petroleum and other industries, 

along with the media and public. These customers use select data for a variety of purposes. 

Consequently, a new product proposed for 2013 is a Product Profile for each PBS publication. A separate Product 

Profile will supplement the existing technical notes in the publications. The Product Profile will provide additional 

documentation regarding sources of data and their limitations, survey methods and appropriate uses. 

 

 

1

The analysis and conclusions contained in this paper are those of the author(s) and do not represent the official position of the 

U.S. Energy Information Administration (EIA) or the U.S. Department of Energy.
2 

Text Box 1: Overview of Surveys in the Petroleum Marketing Program

The Petroleum Marketing Family of Surveys consists of three sub-families of surveys and a sampling frame. The 

geographic coverage for these surveys is the United States (US), which in most cases includes the 50 states and the 

District of Columbia. As stated below the geographic coverage in a few cases also includes the U.S. territories and 

possession. All of the surveys in the program are mandatory. The monthly surveys are a census whereas the weekly 

and annual are sample surveys.

The first sub-family is composed of the following three monthly surveys which collect data on crude oil acquisition 

costs and the volumes acquired. 

• The EIA-182, “Monthly Domestic Crude Oil First Purchase Report,” collects data from firms which take or 

retain ownership of domestic crude oil leaving the lease on which it was produced within the US including the 

Outer Continental Shelf.

• The EIA-856, “Monthly Foreign Crude Oil Acquisition Report,” is a census of two populations: those firms that 

reported data as of June 1982 on the Transfer Pricing Report (ERA-51) and those firms acquiring more than 

500,000 barrels of foreign crude oil during the reporting month for importation to the US and its territories and 

possessions.

• The EIA-14, “Refiners’ Monthly Cost Report,” collects data from firms who own or control refining operations 

in the United States and its territories/possessions.

The second sub-family is composed of the following monthly and annual surveys which collect data on petroleum 

product sales volumes and prices for the refined products. 

• The EIA-782A, “Refiners’/Gas Plant Operators’ Monthly Petroleum Product Sales Report,” collects data from 

firms who directly or indirectly control a refinery or a gas plant located in the United States. Firms report by 

state on their total sales volume (in thousands of gallons) and average price per gallon (excluding sales taxes) 

for each petroleum product by sales type by user category.

• The EIA-782C, “Monthly Report of Prime Supplier Sales of Petroleum Products Sold for Local Consumption,” 

collects data from petroleum product suppliers who make the first sale of specified petroleum products and then 

deliver that product into a state for consumption in that state.

• The EIA-821, “Annual Fuel Oil and Kerosene Sales Report,” collects data from companies that deliver or sell to 

end users distillate and/or residual fuel oils or kerosene. The data being reported are average prices, which are 

computed using volume measures derived from the survey itself once a year. 

Firms report sales volume by state and product type (motor gasoline, diesel fuel oil, distillates, residual fuel oil, and 

other product types), and in a few cases by energy end use (residential, commercial, industrial, and other categories).

The third sub-family is composed of the following three weekly surveys which collect price data for end-users of 

refined petroleum products where the reporting unit tends to be the individual outlets. 

• The EIA-877, “Winter Heating Fuels Telephone Survey,” is a sample survey of No. 2 heating oil and propane 

dealers in Eastern and Midwestern states. This survey is conducted from October 1 to March 15.

• The EIA-878, “Motor Gasoline Price Survey,” is a sample survey of retail outlets selling motor gasoline.

• The EIA-888, “On-Highway Diesel Fuel Price Survey,” is a sample survey by telephone of retail motor vehicle 

diesel fuel outlets.

Firms report weekly prices of sales of select petroleum products by state and product type on these surveys. The data 

reported on these weekly surveys are point-in-time estimates.

The EIA-863, “Petroleum Product Sales Identification Survey” is a census which occurs every four years and is used 

to build the frame for the EIA-821, the EIA-877, EIA-878, and EIA-888. The frame consists of resellers and 

retailers of No. 2 distillate, propane, and residual fuel oil, motor gasoline, and other select petroleum products.

These surveys in combination are used to monitor the petroleum volumes and prices as the commodity moves 

through the various stages from production/importation of the raw material to refining to create the finished products

to transfer/distribution from the refiner to the retail outlets to sales to ultimate consumers. 
3 

Introduction: The Petroleum Marketing Program (PMP) collects and publishes weekly, monthly and annual data on 

the characteristics, structure, and efficiency of petroleum markets at the national, regional, and state levels. These 

data are published in the Monthly Energy Review (MER), Annual Energy Review (AER), Petroleum Marketing 

Monthly (PMM), Gasoline and Diesel Fuel Update (GDFU) and other web publications and analysis.

The data integrated into these publications is compiled from ten surveys described in order of their place in the 

market chain in Text Box 1. A schematic displaying integration of the Petroleum Marketing Program data collection 

activities is provided below in Diagram 1. This displays the petroleum flow from imports and domestic reserves to 

refineries, from the refineries through the pipeline, barges or tankers to bulk terminal storage, and from the bulk 

terminals along to tanker trucks that deliver gasoline to retailers, resellers and outlets where consumers purchase 

gasoline. This diagram also displays the challenge of collecting data regarding crude oil and petroleum products 

from a variety of respondents, including importers, refineries, retailers, resellers and outlets that market gasoline and 

other products.

Diagram 1: Petroleum Marketing Data Collection

Source: EIA