# Presentation: 858670

## Update: Results of Simulation Study for the EIA-914

- Preston McDowney
- Statistics and Methods Group
- U.S. Department of Energy
- preston.mcdowney@eia.doe.gov

## 2004 Spring Meetings Recap

- Inderjit Kundra:
- EIA was in the process of developing a new survey, the EIA-914, to collect monthly production of natural gas in the United States
- EIA was considering a Probability Proportion to Size (PPS) sample.
- The goals were to estimate total US production with a coefficient of variation (CV) of 1% and regional production with a CV of 5%.

## Regions

-  Texas (TX)
-  Federal Gulf (FG)
-  Louisiana (LA)
-  New Mexico (NM)
-  Oklahoma (OK)
-  Wyoming (WY)
-  Others (excluding Alaska)

- 2004 Spring Meetings Recap

## How 

- Sample size was determined by using Presumed Optimum Allocation formulae

- Where CSS	denotes calculated Sample Size
- 	h	denotes stratum in a region
- 	_N(h)_	denotes number of operators in a stratum
- 	S(h)	denotes standard Deviation in stratum h
- 	CV	denotes coefficient of variation
- 	y(hi)	denotes the production for operator i in stratum h

- 2004 Spring Meetings Recap (cont.)

##  Estimated Sample Sizes  

**1**** An operator within a region having a measure of size greater than or equal to total production divided by 2n was selected with certainty.****  **

| 1% CV |  |  |  |
| --- | --- | --- | --- |
| Region | Certainty Group1 | Certainty Coverage | Proposed Sample Size |
| US | 95 | 82% | 176 |

| 5% CV |  |  |  |
| --- | --- | --- | --- |
| Region | Certainty Group1 | Certainty Coverage | Proposed Sample Size |
| TX | 52 | 74% | 96 |
| FG | 12 | 72% | 20 |
| LA | 34 | 72% | 57 |
| NM | 15 | 80% | 28 |
| OK | 35 | 72% | 63 |
| WY | 25 | 93% | 31 |
| OTHERS | 38 | 73% | 63 |
| SUBTOTAL | 211 | - | 358 |

## Four Estimation Procedures

- Probability Proportion to Size (PPS) Estimator
- Weighted Least Squares (WLS) Estimator
- Ordinary Least Squares (OLS) Estimator
- Difference Estimator

- 2004 Fall Meeting Recap

## Variations of the WLS and OLS

- Two Methods
- Using the non-certainty group only in the estimation procedure
- Using both the certainty and non-certainty groups in the estimation procedure
- Three Variations
- Using all of the operators in the method
- Removing outliers
- Removing outliers and overly influential observations

## Simulation Results,Percent Error Estimates for Total Natural Gas Production * 

- 2004 Fall Meeting Recap (cont.)

## Fall 2004 Conclusions

- The PPS sampling did not provide the desired accuracy
- ASA Committee Recommended Using the Hajek Estimator
- Decided to use a cut off sample with 90 percent coverage at the national level
- John Wood Presented the need for a Bias Adjustment

## Spring 2005 Meeting Recap	

- John Wood Presented Preliminary January and February 2005 Data
- Demonstrated the Estimation Procedure and Adjustment For Negative Bias
- Reported 99.4% Response Rate (Volume Weighted)

## Stage 2- Major Differences

- New Frame
- Hajek Estimator
- Stratified Sample
- *Focus on 2 regions TX and FG

## Frame Details

- _Stage 1_
- Annual Data
- Sources:
-  EIA-23 Survey Data
- State Data
- Multiple Third Parties

- _Stage 2_
- Monthly Data
- Sources:
- Single Third Party

## Stage 1

- Integrated
- Texas (TX)
-  Federal Gulf (FG)
-  Louisiana (LA)
-  New Mexico (NM)
-  Oklahoma (OK)
-  Wyoming (WY)
-  Others (excl. Alaska)

- Individual
- Texas (TX)
-  Federal Gulf (FG)

- Stage 2

- Regional Detail

## Sample Sizes

- Stage 1

- Stage 2

## Sampling Methodologies and Estimators

- Probability Proportion to Size (PPS)
- PPS Estimator
-  Weighted Least Squares (WLS)*
- Ordinary Least Squares (OLS)*
- Difference Estimator*
- Hajek Estimator
- Random Stratified Sample
- Stratified Sample Estimator

- Model Based Methods *

**Notes:**
The pps estimator uses the inverse of the probability of selection to weigh the responses of the operators in the pps sample

The weighted least squares estimator is the classic ratio estimators

The ordinary least squares estimator

And difference estimator, where the ratio 1, and uses the 2000 values of the nonsampled operators as their estimate in 2002.

## Hajek Estimator

- Y=Estimate
- wi= Weight
- yi= Observation
- n=non-certainty sample size

## Stratified Sample

- Random Stratified Sample with Replacement
- 6 non-certainty strata in Texas
- 2 non-certainty strata in the Federal Gulf

## Review of Summary Results 

- Compare the Percent Errors of the Estimated Regional Annual Totals (summation of monthly estimates)

## Texas Stage 2:  Annual Percent Error of Estimated Total

**Notes:**
The Strat. Estimate largest negative bias of all the estimators, but the  smaller range of errors than the Hajek and PPS.  Difference and Cert&NC estimators look good

## Federal Gulf Stage 2: AnnualPercent Error of Estimated Total

## Conclusions

- In General
- Probability Methods Had Largest Variation And In Many Cases Negative Bias
- Difference Estimator and Model Based Estimators with Certainty and Non-Certainty Operators Had the Smallest Variation and Bias
- Outlier and Influential Observations Detection Made Little Difference

## Slide 22

## Texas New Simulation Annual ResultsPercent Error of Estimated Total (outliers and influential variables removed)