# Presentation: 698640

## Energy Survey Techniques and Forecasting Methods

- Dr. Mark Rodekohr
- January 2005
- South Asian Regional Initiative –
- Energy
- Bhutan
- (This presentation is based on presentations made by Dr. Ruey-Pyng Lu and Ms. Susan Holte)

## Exit Polls for 14th Lok Sabha 

- Election 2004

| Media Group | NDA | Cong+ | Others |
| --- | --- | --- | --- |
| Sahara Samay | 263-278 | 92-102 | 171-181 |
| Star News | 263-275 | 174-186 | 86-98 |
| Aaj Tak | 248 | 189 | 105 |
| Zee News | 249 | 117 | 176 |
| NDTV-IE | 230-250 | 190-205 | 100-130 |
| Final Results | 185 | 220 | 137 |

- Most analysts believed the NDA would win the elections:
- this assessment was also supported by opinion polls. But ...

## Types of Survey Error

- Coverage error ---
- Nonresponse error --- the error due to failure of some respondents to submit information.
- Sampling error --- due to interviewing sample rather than entire population
- Measurement error
- Data processing error

## Designing a Survey

- Identify needed information
- Identify the survey population
- Access to the **frame** (e.g. a list of the population entities.)
- Watch for** Frame Errors** (omissions) -- usually contribute to understatement of totals
- Describe attributes of the population by conducting a survey
- A sample design with a probability mechanism?
**Sampling Error** results from estimating at total from a sample

## Collection and Processing

- Identify respondents
- Watch for **Measurement error **
- Design form, send to respondents, collect by mail or telephone
- Enter data into Processing System.
- Watch for **Processing Error**

## Total Survey Error

**Large variance, **
**small bias**

**Small variance,**
**large bias**

## Bias and Variance by Error Source

## A survey from a process perspective

- Define research objectives

- Choose mode Of Collection

- Choose sampling frame

- Construct and pretest
- questionnaire

- Design and
- Select sample

- Recruit and measure sample

- Code and edit data

- Make postsurvey adjustment

- Perform analysis

## Scope of Survey and Frame

- Define what you want to measure, determine  which part of industry is optimal, where data exists in company
- 	-- custody versus equity?
- 	-- company versus establishment?
- 	-- specific person who knows or has data
- 			knowledge versus willingness 			versus authority to cooperate
- 	-- when are data available?

**Notes:**
Work with experts to understand the issues
Work with industry to gain their approval

## Energy Industries Highly Skewed

- 11% of all establishments account for 70% of all business
- Sample design must take frame size (counts) and respondent size (skewness) into account
- Size is an important analysis variable
- May need more than name and address of energy companies – e.g. industry type, geography, size
- Difficult to track “births” and “deaths” of establishments
- When is an entity “in operation”?
- How should mergers be handled?
- Frames get out-of-date

## Types of Frame Errors

-  Coverage errors
- Missing units, duplicates, extraneous units
-  Classification errors
- Industry type (SIC, NAICS)
- Geography
- Size
- Contact errors
- Address incomplete or incorrect
- Contact name incorrect
- Phone number incorrect
- Current frame?

## Collection Mode

- Self-administered questionnaire most common
- Mail
- Electronic (including fax, email)
- Internet
- Telephone interviewing
- For survey with few questions
- For nonresponse followup

## Respondent Process

- Identify appropriate respondent at establishment.
- Send materials to respondent
- Conduct Cognitive test

## A fancy way to say talk to people and use the information they provide 

Work with the person who fills out the form to make sure they understand and have data

- Work with the person who fills out the form to make sure they understand and have data

- Cognitive Methods

**Notes:**
Identify appropriate respondent at establishment.
Send materials to respondent
Respondent knowledge, authority, desire to respond
Respondent’s assignment of priority for this response (voluntary versus mandatory, competing tasks, sensitivity of information)
Respondent’s comprehension of requested information (unit of response, specific data)
Ease of data retrieval from records, number of others involved in getting data
Company policy on response

## Types of Imputation

- Imputation -- Estimation of missing survey information.  May be used to replace data that fail edits.  In EIA used primarily to estimate for nonresponse.
- Model-based imputation
- Deck imputation
- cold deck – external donor
- Hot deck – internal donor
- Weight adjustment (for sample surveys)

## Types of nonresponse

- Unit nonresponse – failure for a respondent to send in a form
- If skewed population and periodic survey, use estimate based on past reported data from respondent as imputed value.
- If nonrespondent similar to respondents, use estimate based on other respondents (eg. Hot deck)
- Error is difference between imputed value and value respondent should have reported.
- Item nonresponse – respondent does not completely fill in form, leaves some items blank.

## To Minimize Nonresponse 

- Follow-up with nonrespondents that do not provide data
- Send reminder letter
- Followup call(s)
- Send letters to more senior people in company signed by more senior people in your agency

## Dealing with nonresponse

- Increasing contacts (Number of call backs, call scheduling)
- Increasing cooperation
- Advance letters and information, letters to higher authorities in company
- Adjust estimates (if necessary)
- Impute for missing data using past data or data from other respondents, as appropriate
- Can subsample nonrespondents to determine similarity to respondents

## Both kinds of nonresponse

- Higher if information requested is not available, question is difficult to understand, or data are difficult to find.
- Higher if data requested are considered sensitive.  Some price information is considered sensitive.
- Usually lower if surveys are mandatory

## Computing response rate R

- I = number completed interviews
- RF= number refusals
- NC=number noncontacts
- NI=number of other interviews not completed
- IE=number of ineligibles

## Example of Survey Response Rates

_**	**__**	**__**Unweighted**_
_**Survey**_**	**_**Year**_**	**_**Rates**_
**Agricultural Resources Management Study****	****1999****	****66.1**
**Agricultural Survey****	****1999****	****80.0**
**Annual Survey of Manufacturers****	****1998****	****72.6**
**Census of Agriculture****	****1997****	****86.2**
**Commercial Building Energy Consumption**
**Survey:  Building Characteristics****	****1995****	****87.5**
**Economic Census****	****1997****	****86.1**
**National Ambulatory Medical Care Survey****	****1998****	****68.0**

- Shimizu, I.M.  (2000)  “Response in Federal Establishment Surveys.”  Paper presented at the International Conference on Establishment Surveys, June 2000, Buffalo, NY.

## Summary of Response Rates and Nonresponse Bias

- Bias is a function of both:
- Response rate
- Difference between respondent data and imputed data.  (Can check by using simulation.)

## Inspection Methods for Data Processing Quality

- 1.	Inspect sample (or entire set) of items in the data base
- 2.	Compare to questionnaires, or contact respondents to verify
- 3.	*Optional:* correct nonconformities
- 4.	Pass or fail
- 5.	*Optional: *provide feedback to operators

## Continuous Quality Improvement Strategy

- Use of Pareto principle
- Focus on root causes
- Team approach

## The Generic Coding Process

**The Generic Coding Process**

## Survey Evaluation

| Evaluation Methods | Stage | Purpose |
| --- | --- | --- |
| Debriefings
Interviewers group
Respondent focus groups | Post-survey | Evaluate questionnaire and data collection procedures  |
| Post-Survey Analysis
Experiments
Comparisons | Post-survey | Compare alternative methods of data collection, estimate MSE components, validate survey estimates |
| Post-survey Data collection
Reinterview
Non-response followup
Record check studies | Post-Survey | Estimate one or more components of MSE |

## The Balance

- Quality dimensions conflict
- Accuracy vs timeliness
- Accuracy vs relevance
- Comparability vs accuracy
- Cost vs error
- Comprehensiveness vs timeliness

## Modeling and Forecasting Techniques

## Why Use a Model?

- Represents complex systems in an understandable form.
- Helps to organize large amounts of data.
- Provides a consistent framework for testing hypotheses.

## Scope of Energy Policy Models

- Energy System Models
- These represent the behavior of an entire energy system, for example, a state, nation, region, or the world.  Macroeconomic trends drive the model but are exogenous.
- Energy-Economy Models
- These represent impact of energy system on the wider economy, and the economic drivers change the energy system.
- Partial System Models
- These include sectoral models, life-cycle costs tools, and facility-siting tools.

## A Taxonomy of Energy Policy Models

- Optimization Models - Prescriptive
- Identify least-cost configurations of energy systems based on various constraints, such as fuel supply and emissions limits.
- Select technologies based on their relative costs.
- Simulation Models - Descriptive
- Simulate behavior of consumers and producers in response to signals, such as prices, incomes, and energy policies, but this may not be “optimal” behavior.
- Typically use iterative approach to find a market-clearing demand-supply equilibrium.
- Energy prices are endogenous.

##  A Taxonomy of Energy Policy Models

- Econometric Models
- Typically use relatively simple economic relationships between, for example, demand for energy and energy prices.
- Accounting Frameworks
- Rather than simulate the behavior of a system in which outcomes are unknown, these asks user to explicitly specify outcomes.
- Main function of these tools is to manage data and results.

##  Another Taxonomy

- Deterministic Models
- Use single estimates of input assumptions.
- Derive a single answer.
- Stochastic Models
- Use a range of estimates of key assumptions, often a probability distribution.
- Derive a range of outcomes.
- Complex to specify inputs and interpret outcomes.

## Timeframes for Energy Models

- Short-term Models – up to several years
- Typically econometric models to represent supply and demand changes in reaction to prices.
- Midterm Models – up to several decades
- May be more complex to represent structural and regulatory changes and capital stock turnover.

## Optimization Models

- Typically use linear programming to identify energy systems that provide the least-cost means of providing   energy services, which are usually specified exogenously.
- Optimization is performed under constraints, for example, technology availability, fuel supply, and/or emissions limits.
- Model chooses between technologies based on their life-cycle costs.
- Least-cost solution also yields estimates of energy prices (the “dual” solution).
- Examples: MARKAL, EFOM, WASP (electric sector).

**Notes:**
EFOM is the Energy Flow Optimization Model, developed by the CEC, Commission of European Communities. It is a network of energy flows from source to service demands. One extension focuses on CHP systems for towns.

The Center for Energy, Economic, and Environmental Systems Analysis  (CEEESA) is an organization of Argonne National Laboratory (ANL). Argonne is a U.S. Department of Energy research and development laboratory and is operated by the University of Chicago. CEEESA is an administrative unit of ANL’s Decision and Information Sciences Division (DIS). CEEESA activities focus on the following broad strategic programmatic areas:
Power systems analysis
Energy systems analysis
Environmental systems analysis
Technology assessment
Natural gas systems analysis
Policy analysis
Economic analysis

WASP – Wien Automatic System Planning Package

## Optimization Models - Pros

- Structured and consistent approach for cost-based analyses, such as: *What will be the costs of meeting a certain policy goal?*
- Especially useful where many options exist: *What is the least-cost combination of efficiency, fuel switching, pollution trading, scrubbers, and lower-sulfur coal for meeting a SOx emissions limit?*

## Optimization Models - Cons

- Questionable fundamental assumption of perfect competition, that is, no monopolistic practices, market power, or subsidies, and all markets in equilibrium.
- Not well suited to simulating how systems behave in the real world.
- Assume energy is only factor in technology choice. *Is a Ferrari the same as a Ford? *
- Tend to yield extreme allocations, unless carefully constrained.
- Not well suited to examining policy options that go beyond technology choice or hard-to-cost options. T*o reduce CO**2** you can either use a large hybrid car or drive a smaller car. *
- Relatively complex, opaque, and data intensive. Hard for less-expert users to modify and understand, so less useful in capacity-building efforts.

## Simulation Models

- Simulate behavior of energy consumers and producers under various signals, such as price, income levels, and the rate of stock turnover.
- Pros:
- Not limited by assumption of “optimal” behavior.
- Do not assume energy is the only factor in technology choice. For example, BALANCE uses a market-share algorithm based on price and  “premium multipliers” indicating quality of energy services.
- Cons:
- Tend to be complex and data intensive.
- Behavioral relationships can be controversial and hard to parameterize.
- Future forecasts can be very sensitive to starting conditions and parameters.
- Examples: ENPEP/BALANCE, Energy 2020

**Notes:**
Show the basic VENSIM diagram. 
ENPEP – Argonne National Labs. 
NEMS (Economy Market Equilibrium) 
Dale Nesbit; Jim Pierrenbum

## Accounting Frameworks

- Physical descriptions of energy system, with costs and environmental impacts optional.
- Rather than simulating *decisions* of energy consumers and producers, modeler explicitly accounts for *outcomes of decisions.* Instead of calculating market share based on prices and other variables, accounting frameworks simply examine the implications of a scenario that achieves a certain market share.
- Explore the resource, environment and social cost implications of alternative future “what if” energy scenarios. *What would be the costs, emissions reductions, and fuel savings if we invest in more energy efficiency and renewables instead of fossil fuel power plants?*
- Examples: LEAP, MEDEE, MESAP

**Notes:**
EPRI with the automated RES; Change rules at each node to see the effect on the model results. 

## Accounting Frameworks

- Pros:
- Simple, transparent, flexible, with less required data.
- Capable of examining issues that go beyond technology choice or are hard to cost.
- Especially useful in capacity building applications.
- Cons:
- Does not automatically identify least-cost systems: less suitable where systems are complex and a least-cost solution is needed.
- Does not automatically yield price-consistent solutions (e.g. demand forecast may be inconsistent with projected supply configuration).

## Econometric Models

- Typically, incorporate relatively simple equations relating energy demand and/or supply to energy prices.
- May include many equations.
- Usually do not include structural elements of energy system, such as technologies and capital stock.
- May be limited in ability to address energy policies.

## Top-Down Models

- More *economically-focused* models, in which the energy sector is one part.
*Computable general equilibrium *(GGE) models of national or global energy systems represent an entire economy by a relatively small number of aggregate variables and equations.
- Production function parameters are calculated for each sector so that inputs and outputs reproduce a single base historical year.
- In policy runs, the mix of inputs required to produce one unit of a sector’s output is allowed to vary according to user-selected elasticities of substitution.

**Notes:**
DRI Econometric
Answers are at the top and the other results/solutions flow down from there. 
Focus on financial flows.
Capture interaction between energy policies and macroeconomic performance of the economy.
Top-down models on the other hand often include variables that account for macroeconomic factors and are easier to link to macroeconomic modes to obtain feedback results.
Technology: the autonomous energy efficiency index (AEEI) and the elasticity of substitution (ESUB).

## Bottom-Up Models

*Technology-explicit *models that focus on the energy sector of an economy.
- Energy-using technologies are identified by a detailed description of the inputs, outputs, unit costs, and several other technical and economic characteristics.
- A sector is represented by a usually large number of technologies, linked together by their inputs and outputs,which may be energy forms or carriers, materials, and/or emissions.
- The unit of sectoral output, for example, vehicle kilometers of heavy truck service, is produced using a mix of individual technologies’ outputs.
- The production function of a sector is *implicitly *constructed.

## Top-Down vs. Bottom-Up 

- Top-down models encompass macroeconomic variables beyond the energy sector, such as wages, consumption, and interest rates.
- Bottom-up models have a rich representation of the variety of technologies (existing and/or future) available to meet energy needs, and, they often have the capability to track a wide variety of traded commodities.

**Notes:**
Issues of aggregation, endogenization, technology representation. Also intangible costs and market barriers. 
Models that endogenize behavior are able to predict actual outcomes, whereas models that exogenize behavior are better suited to simulate effects of changes in historical patterns 

## National Energy Modeling System (NEMS)

- Developed by the U.S. Department of Energy’s Energy Information Administration.
- General equilibrium energy-economic model of U.S. energy markets with energy-related emissions.
- Energy supply, demand, and conversion sectors of domestic energy markets, plus international and macroeconomic modules.
- Designed for U.S. regional characteristics and regulatory structure.
- www.eia.doe.gov

## NEMS Energy Modules

- Residential Demand - 16 end-use services, 3 housing types
- Commercial Demand - 10 end-use services, 11 building types
- Industrial Demand - 15 industries
- Transportation Demand – personal vehicles, trucks, ship, rail, air
- Electricity Markets - fossil, renewable, and nuclear technologies
- Renewable Fuels - wind, geothermal, solar thermal, solar photovoltaic, municipal solid waste, ethanol, wood and other biomass, conventional hydropower
- Oil Supply - onshore, offshore, enhanced recovery
- Natural Gas Supply - onshore, offshore, coalbed methane, shale, tar sands
- Natural Gas Transmission and Distribution – domestic, pipeline imports, LNG
- Petroleum Markets – refining and pricing
- Coal Markets – sulfur, mercury, and thermal content, transportation

## NEMS Technology Detail

- Residential - 34 end-use technologies
- Commercial - 10 distributed generation technologies, 64 end-use technologies
- Industrial – technology detail for cogeneration and motors
- Transportation - 6 car and 6 truck size classes, 63 light-duty vehicle fuel-saving technologies, 6 advanced aircraft and 37 advanced freight truck technologies
- Electricity - 11 fossil; 7 renewable; conventional and advanced nuclear
- Petroleum Refining – more than 40 technologies

## MARKAL (MARKet ALlocation)

- Initial model is a partial equilibrium bottom-up energy system technology optimization model using perfect foresight and linear programming.
- Numerous model variants:
- Demand response to price (MACRO (non-linear)) and elastic demand (MED))
- Uncertainty (stochastic)
- Endogenous technology learning (ETL)
- Linked multi-region models.
- New variants under development
- Multi-criteria analysis (goal programming)
- Myopic execution (System for the Analysis of Global Energy - SAGE).

## MARKAL/SAGE

- Energy sector total life cycle costs, including upstream activities such as mining, import/export, conversion and distribution.
- End users in industry, commercial and service, households, agriculture, and transportation.
- Demand is exogenous under MARKAL and endogenous under MARKAL-MACRO/MED.
- MARKAL uses a comprehensive framework that addresses interactions and competition between all energy forms and technologies.

## MARKAL/SAGE

- MARKAL has been used in over 50 countries and 110 institutions. Government and research institutions from countries participating in ETSAP are the heaviest users of the model, along with a growing number of transition and developing countries. It is increasingly being used with a regional focus: Central America, ASEAN, APEC, Nordic countries, global.
- SAGE is the DOE EIA version.
- MARKAL addresses mid- to long-term “what if” questions.
- MARKAL can accommodate a variety of policy objectives, such as program/portfolio goals, energy security, emission reduction, and market analyses that identify optimal energy technologies to meet these objectives.

## Typical Data for Bottom-Up Models

- Macroeconomic Variables
- GDP, value added, population, income, industry production
- Energy Demand
- Fuel use and service demand by sector/subsector
- End-use and technology characteristics, such as existing stock and technology cost and performance
- Price and income elasticities
- Energy Supply Data
- Characteristics of energy supply, transport, and conversion facilities - capital and O&M costs, efficiencies, capacity factors, etc.
- New capacity on-line dates, costs, characteristics
- Energy resources and prices
- Reserves of fossil fuels; potential for renewable resources
- Emission Factors per unit energy consumed, produced, or transported

## Other Sources of Information

- Energy Modeling Forum at Stanford University
- Energy Technology Systems Analysis Program
- Tellus Institute
- Carnegie Mellon University
- Massachusetts Institute of Technology Energy Laboratory
- Lawrence Berkeley National Laboratory
- Resources for the Future
- Argonne National Laboratory
- Energy Policy Journal

## Short-Term Models

- Short-term energy models typically
- Address a period of up to five years
- Include changes in energy consumption to factors such as price, income, and weather or changes to energy supply in response to demand and price.
- Are econometrically-based with a lagged-adjustment structure
- Models may or may not be equilibrium models.
- Eviews is a commercial econometric modeling package.

**Notes:**
The student version costs about $40 but has limitations with regard to some more advanced econometric techniques.

## Short-Term Models

- Pros:
- Relatively transparent
- Not data-intensive
- Quick to develop, maintain, and run
- Cons:
- Little structural detail
- Limited in addressing policy issues, such as tax credits and efficiency standards
- Does not explicitly address changes in energy technology and stock turnover