## Assessing Effects of Government Regulations on Manufacturers Discussion with Department of Energy April 6, 1994 Arthur D. Little, Inc. on behalf of Air Conditioning and Refrigeration Institute, Association of Home Appliance Manufacturers, Gas Appliance Manufacturers Association Reference: 44856 

## Assessing Effects of Government Regulations on Manufacturers Discussionwith Department of Energy April 6, 1994 Arthur D. Little, Inc. on behalf ofAir Conditioning and Refrigeration Institute, Association of Home Appliance Manufacturers, Gas Appliance Manufacturers Association Reference: 44856

### Situation 

## Department of Energy requires a method to assess effects of appliance efficiency regulations on manufacturers. Understand financial impacts of regulations
Utilize currently accepted theory/modeling techniques Currently based on Manufacturers Impact Model (MIM) 

## developed by Lawrence Berkeley Laboratories (LBL)

ES44856aq033094 

### Situation 

## Department of Energy requires a method to assess effects of applianceefficiency regulations on manufacturers. Understand financial impacts of regulations
Utilize currently accepted theory/modeling techniques Currently based on Manufacturers Impact Model (MIM) 

## developed by Lawrence Berkeley Laboratories (LBL)

ES44856aq033094

### Situation 

## An appropriate model must be credible, usable, accurate and meet additional criteria requested by DOE. Credible

Usable
Accurate
DOE criteria 

## ArtJurD Little • Measure majorforces impacting effects on manufacturers • Account for all significant effects on manufacturers • Be based on sound underlying principles • Be verifiable and understandable to all major users

• Utilize readily available data • Use reasonable, predictable assumptions • Provide clear, understandable output

• Produce accurate forecasts of future market behavior • Result in correct and repeatable impact measures

• Contain predictive capability for prices, quantities, etc.

ES44856aq033094 2 

### Situation 

## An appropriate model must be credible, usable, accurate and meet additional criteria requested by DOE. Credible • Measure majorforces impacting effects on manufacturers • Accountfor all significant effects on manufacturers • Be based on sound underlying principles • Be verifiable and understandable to all major users • Utilize readily available data 

### Usable • Use reasonable, predictable assumptions • Provide clear, understandable output
• Produce accurate forecasts offuture market behavior 

### Accurate • Result in correct and repeatable impact measures

• Contain predictive capability for prices, quantities, etc. 

### DOE criteria
ES44856aq033094 2 

## ArtlurD Little

### Credibility 

## The MIM does not effectively measure majorforces impacting manufacturers. Claim • Consumer price and life cycle cost drive demand • MIM assumes: Q=a(P+ff*F)B 

### Implicit discount rates necessaryto make model function are improbable: • Microwave ovens - 250°;'0 • Cooktops and ovens - 37% • Room air conditioners - 49% • Direct heating equipment - 50% 

### • Water heaters - 106% 

### Empirical data demonstrate that other factors have major involvement: Non-price/energy factors

Source: WasherAnalysis

ES44858aq033094 3 

### Credibility 

## The MIM does noteffectively measure majorforces impacting manufacturers. Claim • Consumer price and life cycle cost drive demand • MIM assumes: Q=a(P+ff*F)B 

### Implicitdiscount rates necessaryto make modelfunction are improbable: • Microwave ovens - 

### 250% • Cooktops and ovens - 37% • Room airconditioners - 49% • Direct heating equipment - 50% 

### • Water heaters - 106% 

### Empirical data demonstratethat other factors have major involvement: Non-price/energy factors**~** Source: WasherAnalysis

ES44856aq033094 3

### Credibility 

## The MIM does not effectively account for all significant effects on manufacturers. Claim • Return on equity is the proper measure of manufacturer impact • Long-run and short-run models capture all major effects


### Short-run model misses key aspects of transition to implementation:

• Annualized investments omit timing considerations • Investment requirements • Capital availability • Measures only second-order revenue effect • Does not measure effects priorto regulation date 

## ArtlurD LittJe Cash flow is a more sophisticated and revealing tool:

• Return on equity misses several key effects on manufacturers • Cash flow is more revealing of timing and other issues • Cash flow allows easier assessment 

### of model performance

ES44856aq033094 4 

### Credibility 

## The MIM does noteffectivelyaccountforall significanteffects on manufacturers. Claim • Return on equity is the proper measure of manufacturer impact • Long-run and short-run models capture all major effects


### Short-run model misses key aspects oftransitionto implementation:

• Annualized investments omittiming considerations • Investment requirements • Capital availability • Measures only second-order revenue effect • Does not measure effects priorto regulation date 

## ArtlurDLittle Cash flow isa more sophisticated and revealing tool:

• Return on equity misses several key effects on manufacturers • Cash flow is more revealing oftiming and other issues • Cash flow allows easier assessment 

### of model performance

ES44856aq033094 4

### Credibility 

## MIM short-run ROE analysis only measures revenue and profit declines but misses major investment requirements.. 1993 Refrigerator Standards Estimated Net Income 1993 Refrigerator Standards Estimated Industry Cash Flows
100 80 80 70 60 60 c: 50 c 40 0 .2 

### 'e 20 § 40 E 

## ... th 30 0 20 -20 10 -40 0 -60 1992 1993 1994 1995 1996 1997 1998 0 

# .. (II 1990 1991 01 01 01 01 Cll 01 

### ... 

# .. .... 'Ot Cll '" 

### ... III Cll Cll .... co 

### '" 01 ....

ES44856aq033094 5 80 70 60 c: 50 .2 § 40 E (,4 30 20 10 0 1990 

### Credibility 

## MIM short-run ROE analysis only measures revenue and profit declines but misses major investment requirements.

1991 1993 Refrigerator Standards Estimated Net Income

1992 1993 1994 1995 c: 0 

### 'e



## ...

1996 1997 1998 100 80 60 40 20

0 ·20 ·40 ·60 0 

### ... Ol Ol Ol en 

### ... ... 1993 Refrigerator Standards Estimated Industry Cash Flows

(II Ol en 

### ...

ES44856aq033094 5Credibility 

## The application of economic theory in the MIM is seriously flawed.
Claim • The MIM is a straightforward application of economic theory
The MIM is based on at least twenty-three economic or other assumptions, in addition to data requirements, all of which must be true for the MIM to be accurate:

• Characteristics of firms (4) • Economic theory (6) • Cost/financial characteristics (10) 

### • Measurement (3)


## ArUurD Little • Virtually none of these assumptions are verifiable from external facts • Existing factual data tends to 

### disprove many assumptions

ES44856aq033094 6 Credibility 

## The application ofeconomictheory in the MIM is seriouslyflawed.
Claim • The MIM is a straightforward application of economic theory
The MIM is based on at leasttwenty-three economic orother assumptions, in addition to data requirements, all of which must be truefor the MIM to be accurate:

• Characteristics offirms (4) • Economictheory (6) • Cost/financial characteristics (10) 

### • Measurement (3)


## ArUurDLiHle • Virtually none ofthese assumptions are verifiable from external facts • Existing factual datatends to 

### disprove many assumptions

ES44856aq033094 6

### Credibility 

## The application of economic theory in the MIM is seriously flawed. (continued) Claim • Price is a markup over long term variable cost to cover necessary investments


### Initial assessment of markups from theoretical structure were excessively high (source - LBL):

• Potential measurement errors in prices and marginal costs could invalidate approach • LBL resorted to Cournot theory in attempt to support approach 

### Markups must be adjusted using questionable assumptions to achieve reasonable values:

• Cournot oligopoly result is based upon highly restrictive assumptions • No empirical evidence exists to support LBL assertion that this yields a good approximation of actual behavior • Actual data on number of firms is not congruent with Cournot theory

ES44856aq033094 7 

### Credibility 

## The application ofeconomictheory in the MIM isseriouslyflawed. 

## (continued) Claim • Price is a markup over long term variable costto cover necessary investments


### Initial assessmentof markups from theoretical structurewere excessively high (source - LBL):

• Potential measurement errors in prices and marginal costs could invalidate approach • LBL resorted to Cournot theory in attemptto support approach 

### Markups mustbe adjusted using questionableassumptionsto achieve reasonable values: • Cournot oligopoly result is based upon highly restrictive assumptions • No empirical evidence exists to support LBL assertion thatthis yields a good approximation of actual behavior • Actual data on numberoffirms is not congruent with Cournottheory

ES44856aq033094 7

### Credibility 

## The MIM is not verifiable or understandable to all major users.
Claim • MIM is a sophisticated model which is empirically verified and documented


### Model is not verified: • MIM does not generate predictions which can be empirically tested • Monte Carlo simulation does not address validation between MIM output and real world



## Artlur I)

## Little Calculations are not understandable: • Model process is opaque • Documentation is highly technical • Calculations take complicated approach to a simple problem • Cash flow analysis yields much more direct assessment

ES44856aq033094 8 

### Credibility 

## The MIM is notverifiable or understandabletoall major users.
Claim • MIM is a sophisticated model which is empiricallyverified and documented


### Model is notverified: • MIM does not generate predictions which can be empiricallytested • Monte Carlo simulation does not addressvalidation between MIM output and real world



## ArtJurDLittle Calculations are not understandable: • Model process is opaque • Documentation is highlytechnical • Calculations take complicated approach to a simple problem • Cash flow analysis yields much more direct assessment

ES44856aq033094 8

### Usability 

## The MIM does not utilize readily available data.
Claim • DOE admits that key assumptions are not based on available data 

### Long run fixed and variable cost division is not available: • Long run variable costs are key economic assumption • Long-run variable/fixed cost division is unknown to manufacturers (DOE NOPR page 10480)

ES44856aq033094 9 

### Usability 

## The MIM does not utilize readily available data.
Claim • DOE admits that key assumptions are not based on available data 

### Long runfixed andvariable costdivision is notavailable: • Long run variable costs are key economic assumption • Long-run variable/fixed cost division is unknown to manufacturers (DOE NOPR page 10480)

ES44856aq033094 9 

## D

### Usability 

## The MIM does not use reasonable, predictable assumptions.
Claim • Input values for the MIM are readily obtainable



### The MIM requires many input variables orvalues, most of which are not available in the form required by the MIM. Examples include:

• Implicit consumer discount rates • Price elasticity • Operating cost elasticity • Base case markup • Shape parameter • Economic profit • Accounting assumptions 

## ArtlurDLittle • One-time capital cost • Short-run price response to demand • Prototypical firm cost structure 

### • Equitycapital cost • Many variables require accuracy to four significant figures

ES44858aq033094 10 

### Usability 

## The MIM does not use reasonable, predictable assumptions. Claim • Inputvalues forthe MIM are readily obtainable



### The MIM requires many inputvariables orvalues, mostofwhich are not available intheform required bythe MIM. Examples include:

• Implicit consumer discount rates • Price elasticity • Operating cost elasticity • Base case markup • Shape parameter • Economic profit • Accounting assumptions 

## ArtlurD LiHle • One-time capital cost • Short-run price response to demand • Prototypical firm cost structure 

### • Equitycapital cost • Manyvariables require accuracy to four significantfigures

ES44856aq033094 1 0Usability 

## The MIM does not provide clear, understandable output.
Claim • The MIM is a usable understandable tool open to public assessment and review
MIM operations are unclear: MIM output is obscure and confusing:
• No effective documentation of operations • No guide for users on functioning of model



## ArtlurD LittJe • ROE is limited measure • Other results difficult to find and interpret

ES44856aq033094 11 

### Usability 

## The MIM does not provide clear, understandable output.
Claim • The MIM is a usable understandable tool open to public assessment and review


### MIM operations are unclear: • No effective documentation of operations • No guide for users on functioning of model



## ArtlurD Little MIM output isobscure and confusing: • ROE is limited measure • Other results difficultto find and interpret

ES44856aq033094 11Accuracy 

## The MIM does not generate correct and repeatable impact measures.
Claim • The MIM provides an appropriate, consistent base for assessing effects on manufacturers

MIM results are highly sensitive to changes in long-run fixed/variable cost assumptions:

• Changes in assumption change magnitude of impact • Changes in assumptions reorder options



## ArtlurD Little Several key variables require estimates to several significant digits:

• Short-run price response • Shape parameter

ES44856aq033094 12 Accuracy 

## The MIM does not generate correct and repeatable impact measures.
Claim • The MIM provides an appropriate, consistent base for assessing effects on manufacturers

MIM results are highly sensitiveto changes in long-run fixed/variable cost assumptions:

• Changes in assumption change magnitude of impact • Changes in assumptions reorder options



## ArtlurD Little Several key variables require estimates to several significant digits:

• Short-run price response • Shape parameter

ES44856aq033094 12Forecasting Capability 

## The MIM does not have valid, sophisticated forecasting capability.
Claim • The MIM has a theoretical foundation as a predictive model

MIM starts with standard economic theory, but predictions diverge from actual situation:

• Predicted returns on equity/markups much higherthan observed



## ArtlurD Little Cournot theory invoked to correct situation:

• Restrictive theory • Actual number of firms in most segments violate theory

ES44856aq033094 13 Forecasting Capability 

## The MIM does not have valid, sophisticated forecasting capability.
Claim • The MIM has a theoretical foundation as a predictive model

MIM starts with standard economic theory, but predictions divergefrom actual situation:

• Predicted returns on equity/markups much higherthan observed



## ArtlurD Little Cournottheory invoked to correct situation:

• Restrictive theory • Actual numberoffirms in most segments violate theory

ES44856aq033094 13

### Forecasting Capability 

## The MIM does not have valid, sophisticated forecasting capability. (continued) Claim • The MIM can provide adequate forecasts of key output variables from readily forecastable input variables


### MIM requires "correcting" variables which are neitherforecastable nor, often, known: • Numberof firms - arbitrarily adjusted • Shape parameter - four adjusted significant figures • Implicit discount rates improbable • Industry elasticity - unproven, unverified estimate

ES44856aq033094 14 

### ArtJurD LittJe Forecasting Capability 

## The MIM does not havevalid, sophisticatedforecasting capability. (continued) Claim • The MIM can provide adequate forecasts of key output variables from readily forecastable inputvariables


### MIM requires "correcting"variableswhich are neitherforecastable nor, often, known: • Numberoffirms - arbitrarily adjusted • Shape parameter - four adjusted significantfigures • Implicit discount rates improbable • Industry elasticity - unproven, unverified estimate

ES44856aq033094 14 

## ArtJur"LittJeForecasting Capability 

## The MIM does not have valid, sophisticated forecasting capability. (continued) Claim • The MIM can be used to forecast prices, quantities, and markups for products under regulatory scenarios

MIM cannot be validated versus observed market conditions:

• Normalized output • Non-verifiable input data



## ArtlurD Little Key factors driving forecasts/analysis are inconsistent with actual situations:

• Return on equity • Markups • Number of firms • Implicit discount rates

ES44856aq033094 15 Forecasting Capability 

## The MIM does not have valid, sophisticated forecasting capability_ (continued) Claim • The MIM can be used to forecast prices, quantities, and markups for products under regulatory scenarios

MIM cannot be validated versus observed market conditions:

• Normalized output • Non-verifiable input data



## ArtlurD Little Key factors drivingforecasts/analysis are inconsistentwith actual situations:

• Return on equity • Markups • Numberof firms • Implicit discount rates

ES44856aq033094 15

### Forecasting Capability 

## The MIM is not a useful predictive tool for prices, quantities, and markups. Claim • The 

### MIM is all there is and is betterthan nothing


### Computer outputfrom a bad model is no more accurate or usable than no output. The MIM confuses model complexity computer generation and manipulation of large quantities of data with realism.

• Questionable theory • Continuing adjustments to theory/variables in orderto approximate known data • No empirical validation • Input data less easily estimated than estimating desired results directly • Key data not consistent with known situation

ES44856aq033094 16 

## Artlur I) 

## Little

Forecasting Capability 

## The MIM is nota useful predictivetoolfor prices, quantities, and markups. Claim • The 

### MIM is all there is and is betterthan nothing


### Computeroutputfrom a bad model is no moreaccurate or usablethan no output. The MIM confuses model complexity computergeneration and manipulation of largequantitiesofdatawith realism.

• Questionabletheory • Continuing adjustments to theory/variables in orderto approximate known data • No empirical validation • Input data less easily estimated than estimating desired results directly • Key data not consistentwith known situation

ES44856aq033094 16 

## ArtlurI)

## Little

### Conclusion 

## DOE should completely abandon the MIM. It serves no useful purpose in assessing impact on manufacturers and defeats DOE's obligation to make such an assessment.

• MIM does not meet minimum requirements for 

## anymodel • MIM does not meet DOE's requirement of forecasting capability • MIM is excessively cumbersome and presents a confusing approach to a relatively simple problem

ES44856aq033094 17 

## Artlur I)

## Little Conclusion 

## DOE should completely abandon the MIM. Itserves no useful purpose in assessing impacton manufacturers and defeats DOE's obligationto make such an assessment. • MIM does not meet minimum requirements for 

## anymodel • MIM does not meet DOE's requirement offorecasting capability • MIM is excessively cumbersome and presents a confusing approach to a relatively simple problem

ES44856aq033094 17 

## Artlur"Little