metadata: authors: 0: author: Iimi,Atsushi
count: Japan
docna: 0: docna: Multidimensional auctions for public energy
            efficiency projects : evidence from the Japanese ESCO market
docty: Policy Research Working Paper
subtopic: Public Sector Administrative & Civil
            Service Reform,Energy Consumption,Energy Policies & Economics,Energy Conservation & Efficiency,Hydrology,Environment and Energy Efficiency,Energy Demand,Energy and Environment,Energy and Mining,Democratic Government,Public Sector Administrative and Civil
            Service Reform,De Facto Governments
teratopic: Environment,Macroeconomics and Economic Growth,Energy,Finance and Financial Sector Development
abstracts: cdata!: Competitive bidding is an important
            policy tool to procure goods and services from the market at
            the lowest possible cost. Under traditional public
            procurement systems, however, it may be difficult to
            purchase highly customized objects, such as energy
            efficiency services. This is because not only prices but
            also other nonmonetary aspects need to be taken into
            account. Multidimensional auctions are often used to
            evaluate multidimensional bids. This paper examines the
            bidding strategy in multidimensional auctions, using data
            from public energy service company projects in Japan. It
            shows that multidimensional auctions work well, as theory
            predicts. The competition effect is significant. In
            addition, strategic information disclosure, including
            walk-through and preannouncement of reserve prices, can also
            promote energy savings and investment. Risk sharing
            arrangements are critical in the energy service company
            market. In particular, the public sector should take
            regulatory risk.
geo_reg_key: 517198,119225,517191
repnme: repnme: Multidimensional auctions for public energy
            efficiency projects : evidence from the Japanese ESCO market
display_title: Multidimensional auctions for public
            energy efficiency projects : evidence from the Japanese ESCO market
listing_relative_url: /research/2013/06/17874375/multidimensional-auctions-public-energy-efficiency-projects-evidence-japanese-esco-market
topicv3: Water m1326522 137,Energy and Extractives m1326595 507,Energy Efficiency m1326852 533,Power and Electricity Sector m1327094 515,Public Administration m1327723 723
publishtoextweb_dt: 2013-06-01T00:00:00Z
historic_topic: Environment,Macroeconomics and Economic Growth,Energy,Finance and Financial Sector Development
no_of_pages: 49
author_keys: 0: author_key: 914707
model_extractions: WPS6485 Policy Research Working Paper 6485 Multidimensional Auctions for Public Energy Efficiency Projects Evidence from the Japanese ESCO Market _Atsushi Iimi_ The World Bank Africa Region Sustainable Development Department June 2013
Policy Research Working Paper 6485

# **Abstract**

Competitive bidding is an important policy tool to procure goods and services from the market at the lowest possible cost. Under traditional public procurement systems, however, it may be difficult to purchase highly customized objects, such as energy efficiency services. This is because not only prices but also other nonmonetary aspects need to be taken into account. Multidimensional auctions are often used to evaluate multidimensional bids. This paper examines the bidding strategy in multidimensional auctions, using data from public energy service company projects in Japan. It shows that multidimensional auctions work well, as theory predicts. The competition effect is significant. In addition, strategic information disclosure, including walk-through and preannouncement of reserve prices, can also promote energy savings and investment. Risk sharing arrangements are critical in the energy service company market. In particular, the public sector should take regulatory risk.

This paper is a product of the Sustainable Development Departmen, Africa Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at aiimi@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **MULTIDIMENSIONAL AUCTIONS FOR PUBLIC ENERGY EFFICIENCY**

**PROJECTS: EVIDENCE FROM THE JAPANESE ESCO MARKET**

Atsushi Iimi<sup>†</sup> Sustainable Development Department Africa Region The World Bank Key words: Multidimensional auction; Energy efficiency; Risk sharing. JEL classification: D44; D82; C36.

> † I would like to express my special thanks to the Jyukankyo Research Institute (JYURI) and the Japan Association of Energy Service Companies (JAESCO) for their collaboration in data collection. I am grateful to the Ministry of Economy, Trade and Industry (METI) and many seminar participants at the Institute of Developing Economies-Japan External Trade Organization (IDE-JETRO) and the Japan International Cooperation Agency (JICA) for their insightful comments on an earlier version of this paper. I also acknowledge Matias Herrera Dappe, Antonio Estache, Marianne Fay, Jas Singh, Luis Tineo and Yabei Zhang for their various suggestions.
- 2 -

# **I.  Introduction**

Competitive bidding is an important policy instrument for governments to procure goods and services at the lowest possible cost. Auction theory indicates that intense competition allows for the identification of the lowest-cost vendor or contractor in the market because the probability of winning a public contract declines for a firm as it faces more contenders in the auction (Wilson, 1977; Milgrom and Weber, 1982; Wolfstetter, 1996). As theory predicts, competitive bidding normally works well if an object to be purchased is simple and standardized, such as office supplies and computers.

In practice, however, traditional public procurement systems may not function well when purchases are complex and highly customized, such as infrastructure projects and consulting services. This is due to the fact that traditional systems by and large follow an input-based, least-cost approach, in which auctioneers need to determine all technical standards and other specifications before calling for tenders. Potential vendors or contractors, then, compete only in the price arena.

Multidimensional auction is one practical solution when various aspects—both monetary and nonmonetary—need to be evaluated at once. The theory exists (Che 1993; Cripps and Ireland 1994; Branco 1997; Asker and Cantillon, 2008) but there has been little empirical evidence to show how it actually works. In theory, a fundamental problem of multidimensional auctions is that there is a tradeoff between price and “quality,” which is often a catch phrase for anything that the auctioneer cares about other than prices. There is asymmetric information: procurers do not ex ante know how price and quality are related to each other, and therefore, have to ask bidders to propose what the best combination is. Then, bidders propose their solutions as a combination of price and quality.

This paper examines the bidding strategy in multidimensional auctions, using data from the public energy service company (ESCO) market in Japan. ESCO projects are
- 3 -

multidimensional service contracts aimed at improving energy efficiency of buildings or facilities. In principle, procurers prefer to minimize investment costs, but social and environmental benefits may also need to be taken into account. The interdependence of price and quality bids is also examined, along with the effect of market competition on bidding strategy. The paper will also cast light on the possible effects of information disclosure and risk sharing schemes on auction outcomes.

The rest of the paper is organized as follows: Section II provides an overview of public procurement practices in ESCO projects. Section III discusses our empirical approach. Section IV provides a summary of the data. Section V presents the main estimation results, and Section VI examines statistical issues, robustness, and some policy implications. Then, Section VII concludes.

# **II.  ESCO projects and multidimensional auctions**

The general purpose of ESCO projects is to reduce energy consumption of buildings and facilities through the installation of energy-efficient equipment and systems, such as compact fluorescent lights (CFLs) and power cogeneration systems. ESCO projects are generally estimated to have significant potential for greenhouse gas emission reduction because buildings consume about 40 percent of the world’s final energy (IEA 2008; World Bank 2010; Worldwatch Institute 2009).

Public-sector buildings, such as administrative offices and hospitals, are good candidates for ESCO projects because they are often old and energy-inefficient. In Brazil, the National Electrical Energy Conservation Program retrofitted government buildings and saved 140GWh of energy per year (UNEP 2010).

In practice, however, ESCO procurement is not necessarily an easy task, because ESCO projects are complex service contracts, including a number of components, such as engineering design, installation of goods and equipment, development of new energy sources,
- 4 -

operation and maintenance of facilities, and financial services.<sup>1</sup> Technologies adopted in ESCO projects are not complex, e.g., energy-efficient lights, chillers, and water boilers (Table 1). However, asymmetric information still exists: Procurers do not know ex ante how these technologies could be combined to achieve the highest energy savings at the lowest cost. On the other hand, bidders know some solutions.

Multidimensional auctions help procurers to identify the best service provider, taking a variety of (often-competing) objectives into account. Thus, the procurer only sets a broad project description, such as an energy savings target, and calls for proposals. To evaluate different types of proposals, the evaluation rule, such as weights assigned to criteria, is established. The evaluation criteria may vary across projects.<sup>2</sup> In the case of Japanese ESCO projects, more than 30 criteria are used. Price or payment to ESCO is merely one of many criteria involved.

Given the project description and evaluation rule, ESCOs are asked to prepare technical and financial proposals, normally including energy savings guaranteed and contract period. The amount of energy savings guaranteed is crucial for ESCO projects, because this has many implications to other components of the bid. More energy-efficient technologies usually cost more. Longer-term performance guarantees are a risk for ESCOs.

A distinguishing characteristic of ESCO projects is that they are not one-time acquisitions but long-term service contracts. ESCOs are responsible for maintaining and operating the systems for five to 30 years. The average payback period is 11 years in Japan. Therefore, how to finance the long-term investment is one of the important evaluation criteria in ESCO projects.<sup>3</sup> Governments may prefer a shorter contract period, however such an approach

> 1 See Annex I for further discussion.

> 2 The weighting of criteria differs from project to project. For instance, a Czech ESCO project attached six percent of weight to bid prices. In Germany, the weight of the monetary component is as high as 75 percent (Singh et al. 2010).

> 3 In theory, ESCO projects could pay for themselves because asset owners could reduce energy spending, if the financial markets were perfect.
- 5 -

makes each annual payment larger. It is ESCOs, not governments, who specify the investment plans with all these conditions taken into consideration.

**Table 1. Major Energy** **<u>Savings Technologies Adopted in Japanese ESCO Projects</u>**

|Adopted technology|Share of ESCO<br>contracts using this<br>technology (%)|Avg installed<br>capacity if<br>adopted (kW)|
|---|---|---|
|Lighting|79|4,479|
|Air conditioning|84|1,109|
|Hot water boiler|31|335|
|Cogeneration system|27|515|

_Source_ : Author’s survey data.

# **III.  Empirical model**

ESCO projects are procured under the multidimensional auction system. Scoring auctions are a commonly used format. The bidder whose score is highest wins the public contract (i.e., first-score auctions). In scoring auctions, the bid evaluation rule is disclosed to bidders prior to calling for proposals. In addition, scoring auctions allow each bidder to submit a single offer. By contrast, in menu auctions, bidders can submit a schedule composed of various combinations of solutions.

Multidimensional auction theory tells that economic efficiency is achieved if the auctioneer preannounces his true utility function determining the scoring rule and if there is no commitment problem (Milgrom, 2000; Asker and Cantillon, 2008). A simple scoring auction model suggests that the bidding strategy can be written by a single parameter if the private information is one-dimensional (Che 1993; Branco 1997; Naegelen 2002).<sup>4</sup> Under the private value paradigm, the firm’s bidding strategy essentially depends on three factors: (i) the scoring rule, i.e., weights, (ii) the number of bidders, and (iii) bidders’ private information, i.e., underlying cost parameter.<sup>5</sup>

> 4 Asker and Cantillon (2008) show that a single pseudotype of contractor is sufficient to characterize the equilibrium outcome in any multidimensional auction with more than one dimension of private information. 5 See Annex II for a formal multidimensional auction model.
- 6 -

Our empirical approach is to estimate a reduced-form model, because of limited data (see the following sections for details). Given the sample size of about 100, structural estimation is difficult to perform. In addition, the auctioneer’s objective function is actually far more complex than theory assumes. It is highly nonlinear and is likely to involve more than 10 criteria.

To make the estimation feasible, the current analysis focuses on four major criteria that are commonly used in the Japanese ESCO market: (i) initial investment (denoted by _INV_ ), (ii) annual energy savings ( _SAVE_ ), (iii) fee payment to ESCO ( _PAY_ ), and (iv) contract duration ( _DUR_ ). These criteria constitute the foundation of the net present value of an ESCO project.<sup>6</sup> Note that this specification has already ignored many other evaluation criteria, such as comprehensiveness of the proposed plan and uniqueness of technologies. These are difficult to quantify and compare across different projects.<sup>7</sup> As theory suggests, the bidding strategy is composed of these bid components. Importantly, they are related to one another as a package. Therefore, the following system of equations is considered: _N_ is the number of bidders at auction _i_ . _w_ is a vector of weights on the four criteria.

> 6 The procurer’s net present value of energy savings investment can be written by:

> 7 See Annex IV for more detailed discussion.
- 7 -

In the equation, it is assumed that the bidding strategy is independent between auction _i_ and _j_ , that is, _E_ ( ε _im_ ε _jn_ ) = 0 if _i ≠ j_ . On the other hand, correlation between the equations is not always equal to zero, i.e., _E_ ( ε _im_ ε _jn_ ) = σ _mn_ if _i = j_ , because the bidding strategy is a combination of _INV_ , _SAVE_ , _PAY_ , and _DUR_ .

To control for heterogeneity across auctions, project- and bidder-specific characteristics are included in a covariate matrix, _X_ (see the following section for more details). Of particular note, bidder-specific characteristics are essential, because the bidding strategy must depend on the bidders’ privately held information (i.e., cost parameters).

The difference in procurement and contract methods is also controlled by _X_ . Unlike ordinary acquisition of simple goods, ESCO projects involve uncertainty and are therefore sensitive to the risk allocation and information disclosed.<sup>8</sup> It is worth noting that about 40 percent of the Japanese ESCO projects failed to meet the energy savings targets. This failure is primarily attributable to a lack of information about buildings, equipment, and operations. Each building has its own history, and some of it cannot be known before the actual project implementation.

To reduce such uncertainty while promoting competition at auctions, a variety of procurement procedures are used in the ESCO market (Figure 1). For example, some municipalities predetermine energy savings targets, and others do not. Some local governments define a detailed risk allocation mechanism upfront, and others do not. The following analysis will examine the possible effects of these procedures on auction outcomes, such as energy savings and payments.

8 ESCO projects are normally implemented under performance-based contracts.
- 8 -

**Figure 1. Information Provision Practices in Japanese ESCO Procurement**

<!-- Start of picture text -->
1.0 0.99 0.98<br>0.92<br>0.9 0.83 0.85<br>0.8<br>0.68<br>0.7<br>0.6<br>0.5<br>0.4<br>0.30<br>0.3<br>0.2<br>0.1<br>0.0<br>Source : Author’s survey data.<br>provided<br>Share of ESCO projects with such information<br><!-- End of picture text -->

One empirical challenge in estimating Equation (1) is that the number of bidders is potentially endogenous (e.g., McAfee and McMillan, 1987; Levin and Smith, 1994; Li and Perrigne, 2003; Li and Zheng, 2009; Ohashi, 2009).<sup>9</sup> According to data collected for this study, the firms’ entry decision indeed looks to be a dynamic process. On average five firms showed initial interest, but only 3.5 firms actually applied for the formal process. Some of them decided not to remain in the competition. As the result, only about three firms submitted final proposals (Figure 2).

To deal with the endogeneity problem, the three-stage least squares (3SLS) model (Zellner and Theil, 1962) is used, instead of the ordinary least square techniques, such as seemingly unrelated regression (SUR). The instrumental variable is the number of firms that submitted an initial expression of interest, denoted by _NEOI_ . This is the maximum number of contenders that can participate in each competition. It is clearly related to the actual number of bidders _N_

> 9 Endogenous entry auction theory suggests that given a fixed positive cost of participating in an auction, bidders will enter until their expected profits are driven to the entry cost. At this level no more firms can expect nonnegative profits from new entry.
- 9 -

but not directly relevant to the bidding strategy, because this does not represent the intensity of competition that bidders actually face.<sup>10</sup> The Hausman (1978) exogeneity test technique is used to examine whether bidder participation is endogenous or exogenous. If the Hausman exogeneity cannot be rejected, the conventional SUR estimator is more efficient to estimate Equation (1).

**Figure 2. Average number of firms applying for a public tender**

<!-- Start of picture text --> 6<br>5 4.7<br>4<br>3.5<br>2.9<br>3<br>2<br>1<br>0<br>Number of firms that Number of firms that Number of firms that<br>expressed interest applied made presentation<br>Source : Author’s survey data.<br><!-- End of picture text -->

# **IV.  Data**

The sample data were collected from the public ESCO projects in Japan.<sup>11</sup> The market began developing in the mid-1990s and has been steadily growing until recently. The total market amounted to JPY40 billion or about US$350 million in 2007. About half were implemented in the industrial sector; another half in the residential sector. The public sector accounts for some 20 percent of the total market. The intensity of market competition has increased in

> 10 The underlying idea is the same as the use of the number of plan holders or eligible bidders in the existing literature (e.g., De Silva et al. 2008; Li and Zheng 2009). 11 See Annex III for more data on the Japanese ESCO market.
- 10 -

Japan. Some of the ESCOs have already decided to exit the market in recent years, because of the excess competition.

More than 200 ESCO projects implemented in the past 15 years (1998-2012) were initially identified. The questionnaire survey was conducted from November 2011 to January 2012. The questionnaire was sent out to 111 governmental offices in 29 prefectures. The effective response rate turned out to be about 55 percent. Detailed procurement data were collected for about 100 ESCOs. The data include a wide range of public facilities, from administrative offices to schools and hospitals. Public pools and gyms are also included. Because of lack of relevant information, the following regression analysis uses only about 70 ESCO cases.<sup>12</sup> The summary statistics are shown in Table 2. The previous section has already defined the four dependent variables, _INV_ , _SAVE_ , _DUR_ and _PAY,_ used in the analysis. The average investment amount is about JPY270 million or US$3 million, which in return reduces energy spending by JPY32 million per annum. The contract duration is about 120 months or 10 years on average. Asset owners, i.e., local governments, pay a service fee of JPY30 million to ESCOs every year.

For the weight matrix, _w_ , the original scores are normalized to a scale of 0 to 100 to make them comparable across auctions. Then, several evaluation criteria aimed at the same objective are aggregated to one weight. For example, the weight on energy savings, _wSAVE_ , is a summation of the scores that are attached to four criteria: energy savings guaranteed, compliance with efficiency targets, annual energy spending reduction, and CO2 emission reduction.

The competition effect, _N_ , is defined by the number of firms that actually applied for the selection process. They participated in a question-and-answer and walk-through session and made the final presentation. The ESCO industry is not necessarily thick even in developed

> 12 In total 171 buildings and facilities are covered by these 100 ESCO projects, because some projects are composed of more than one building (i.e., packaged contracts).
- 11 -

countries, such as Japan. Only 3.2 firms participate per auction. As discussed, the number of firms that submitted an initial expression of interest is used as an instrumental variable to deal with the endogeneity of bidder participation. The average is 4.5 firms.

To control for heterogeneity across projects, the following project characteristics are included in _X_ : energy use ( _KWH_ ), area of floor ( _AREA_ ), and building age ( _AGE_ ). Expected energy savings are normally large for energy-intensive facilities, such as hospitals in our case. Larger and older facilities are also likely to have more potential for energy savings, because they allow ESCOs to have flexibility in proposing significant changes in technologies, including energy sources.

The potential energy savings are also dependent on types of buildings and nature of technologies to be adopted. Three dummy variables represent building types: _dOFFC_ for administrative offices, _dHOSP_ for hospitals, and _dSCHL_ for schools, universities and museums. These account for three-fourth of the total sample. Other types of facilities, such as gyms and pools, are used as a baseline.<sup>13</sup> Four main technologies are used in the Japanese ESCO market: lighting ( _dLIT_ ), air conditioning ( _dAC_ ), hot water boiler ( _dHOTW_ ), and cogeneration system ( _dCGS_ ). These are included as dummy variables.

Different ESCOs have different advantages and disadvantages. To take into account possible bidder-specific heterogeneity, the number of ESCO contracts that each firm won in the past is constructed ( _AWAD_ ). This is expected to capture the (dis)advantages that contractors may have. Experienced firms may know technologies better and have more financial and human resources. In the Japanese market, there are 35 firms that won at least one public ESCO contract.<sup>14</sup> One company won 11 contracts, and another secured seven. Most of the other companies were awarded one or two contracts.

> 13 The potential effects of these facilities are found to be insignificant, thus these are altogether used as a baseline.

> 14 The figure counts only leading companies if firms form a bidding coalition, which is a common business practice in the ESCO market.
- 12 -

Particularly in the ESCO market, firms have different backgrounds. Four types of firms exist in Japan: engineering firms, lease companies (often subsidiaries of large banks), manufacturers of energy-related systems, and energy supply companies, such as electricity utilities. While engineering firms are used as a baseline, the rest are included as dummy variables (i.e., _dLEAS_ , _dMANU_ , and _dENEG_ ).

Other unobserved characteristics of bidders are controlled by a set of dummy variables representing their origin prefectures, which may result in some locational advantages. For instance, half of the winning firms are based on Tokyo, in which an abundance of commercial services and human resources are available. This is generally advantageous for firms.

Regarding procurement design, two interesting methods exist in the ESCO market.<sup>15</sup> First, governments can bundle more than one building or facility into one ESCO contract. There is a view that energy efficiency projects are individually too small for ESCOs to invest in and bear a fixed transaction cost (Singh et al., 2010).<sup>16</sup> In this research sample, some governments, though not many, packaged several facilities. These are denoted by _dPCKG_ .

Second, procurers can decide who should be responsible for investment. Two schemes exist: shared savings and guaranteed savings. The former is used as a baseline ( _dSSC_ ). Determining which scheme is a better fit for ESCO projects is a traditional question. Under the shared savings scheme, ESCOs are responsible for investment. Thus, asset owners can transfer not only the project risk but also the financial risk to the private sector, although they may have to pay relatively high service fees to the contractor.<sup>17</sup>

> 15 Note that these are the auctioneer’s choice, not bidders’. Thus, unlike the bidder participation, there is no endogeneity issue from the bidding strategy point of view. 16 Energy efficiency projects are building-specific investments. By nature, therefore, the transaction cost of reviewing the request for proposal and preparing bids tends to be high.

> 17 The shared savings scheme may have other practical advantages. For instance, it may allow the release of governments from some budgetary and operational duties, because ESCO projects can fix energy spending for the long term. In addition, governments do not have to be concerned about daily operations of energy facilities. ESCOs are responsible for operating installed equipment and fixing any mechanical breakdowns.
- 13 -

In theory, the selection between the shared and guaranteed savings schemes must depend on whether public or private finance is cheaper. In the sample used here, about 70 percent of the ESCO projects were based on the shared savings scheme (Figure 3). In the United States, the two models have been used equally, because subnational governments particularly prefer to finance through low-cost instruments, such as tax-exempt bonds.<sup>18</sup> Finally, seven dummy variables are also included to examine the impacts of information disclosure and risk sharing. Four types of information sharing are particularly considered: energy savings target ( _dTAGT_ ), reserve price ( _dRESV_ ), risk sharing matrix ( _dRISKMT_ ), and site visit or walk-through in the facility ( _dWALK_ ).<sup>19</sup> In the literature, the preannouncement of reserve prices is found to make bidders submit more aggressive offers. In Oklahoma highway auctions, for example, the publication of government engineering cost estimates lowered road procurement costs by 4.6 percent (De Silva et al. 2008).<sup>20</sup> Walk-through is not exactly a piece of information but aims at allowing bidders to obtain both tangible and intangible information that they need.

When the risk-sharing matrix is predetermined, it is also important to know who would bear what risk. Three types of risks are examined: (i) factual errors in the request for proposal ( _RISKTYPO_ ), (ii) regulatory risk, such as changes in national energy standards and building codes ( _RISKREG_ ), and (iii) inflation risk ( _RISKINF_ ). These dummy variables are set to one if the public sector is responsible for such a risk.

> 18 The shared savings scheme is considered to be unsustainable if ESCOs have to incur too much risk and debt. 19 See Annex V for an example of the risk sharing matrix that is actually used.

> 20 Similarly, the City of Montreal is also using disclosure of reserve prices to improve outcomes in snowremoval auctions (Flambard et al. 2007). In French timber auctions, it is estimated that the winning bids would be twice as high if optimal reserve prices were set and announced, rather than using the random reservation prices (Li and Perrigne, 2003). However, there seems to be a general hesitation in Japan, perhaps because there is a perceived risk of fostering collusion among bidders (e.g., Saijo _et al_ . 1996; Ohashi 2009).
- 14 -

|**Table 2. Summary Statisti**|**cs**||||||
|---|---|---|---|---|---|---|
|Variable|Abbr.|Obs|Mean S|td.Dev.|Min|<br>Max|
|Investment amount (JPY million)|_INV_|68|265.6|265.0|17.8|1738.5|
|Annual savings of energy spending (JPY million)|_SAVE_|68|32.3|35.6|0.9|171.0|
|Annual payment to ESCO (JPY million)|_PAY_|68|30.1|32.0|1.0|145.0|
|Contract duration (months)|_DUR_|68|121.9|42.1|36.0|180.0|
|Score attached to investment criteria (on a scale of 0 to 100)|_wINV_|68|23.7|6.0|12.5|<br>44.4|
|Score attached to energy savings criteria (on a scale of 0 to 100)|_wSAVE_|68|24.2|4.8|12.5|<br>37.5|
|Score attached to contract duration criteria (on a scale of 0 to 100)|_wDUR_|68|3.8|3.2|0.0|<br>8.3|
|Score attached to payment criteria (on a scale of 0 to 100)|_wPAY_|68|0.6|2.1|0.0|<br>8.3|
|Number of ESCOs applying for public tender|_N_|68|3.2|2.0|1.0|<br>11.0|
|Dummy variable for shared saving contracts|_dSSC_|68|0.79|0.41|0|<br>1|
|Dummy variable for packaged contracts|_dPCKG_|68|0.09|0.29|0|<br>1|
|<br>Annual electricity consumption (GWh)|_KWH_|68|5.0|6.8|0.2|<br>41.9|
|Age of building (years since the first establishment)|_AGE_|68|27.9|12.2|11.0|<br>85.0|
|Area of floor (1,000 m2)|_AREA_|68|32.3|33.5|2.1|171.2|
|Dummy variable for administrative offices|_dOFFC_|68|0.35|0.48|0|<br>1|
|<br>Dummy variable for hospitals|_dHOSP_|68|0.24|0.43|0|<br>1|
|Dummy variable for schools, university and museums|_dSCHL_|68|0.18|0.38|0|<br>1|
|Dummy variable for lighting systems|_dLIT_|68|0.91|0.29|0|<br>1|
|Dummy variable for air conditioning|_dAC_|68|0.97|0.17|0|<br>1|
|<br>Dummy variable for hot water boilers|_dHOTW_|68|0.37|0.49|0|<br>1|
|<br>Dummy variable for cogeneration systems|_dCGS_|68|0.29|0.46|0|<br>1|
|Number of ESCO contracts that were secured in the past|_AWAD_|68|1.31|2.17|0|<br>9|
|Dummy variable for lease companies|_dLEAS_|68|0.06|0.24|0|<br>1|
|Dummy variable for equipment manufacturers|_dMANU_|68|0.26|0.44|0|<br>1|
|<br>Dummy variable for oil, gas and electricity companies|_dENEG_|68|0.22|0.42|0|<br>1|
|Dummy variable for energy saving target announcement|_dTAGT_|68|0.87|0.34|0|<br>1|
|Dummy variable for reserve price announcement|_dRESV_|68|0.88|0.32|0|<br>1|
|<br>Dummy variable for risk sharing matrix disclosure|_dRISKMT_|<br>68|0.26|0.44|0|<br>1|
|<br>Dummy variable for walk through implementation|_dWALK_|68|0.93|0.26|0|<br>1|
|Dummy variable for typo errors born by public sector|_RISKTYPO_|68|0.97|0.17|0|<br>1|
|Dummy variable for regulatory risk born by public sector|_RISKREG_|68|0.24|0.43|0|<br>1|
|Dummy variable for inflation risk born by public sector|_RISKINF_|68|0.16|0.37|0|<br>1|
|Number of ESCOs expressingtheir initial interest|_NEOI_|68|4.5|5.1|1.0|<br>38.0|
- 15 -

**Figure 3. Shares of Guaranteed Savings and Shared Savings Contracts**

<!-- Start of picture text --> 100%<br>90% Shared savings<br>Guaranteed savings<br>80%<br>70% 0.55 0.60<br>0.69 0.67<br>60% 0.75<br>0.91 0.88<br>50% 1.00<br>40%<br>30%<br>20% 0.45 0.40<br>0.31 0.33<br>10% 0.25<br>0.09 0.13<br>0% 0.00<br>2004 2005 2006 2007 2008 2009 2010 2011<br>Source : Author’s survey data.<br><!-- End of picture text -->

# **V. Main estimation results**

The 3SLS estimation is performed; the results are shown in Table 3. The results indicate that, by-and-large, multidimensional auctions are functioning well in the ESCO market. The competition effect is found to be significant to increase energy savings. The elasticity is estimated at 0.569. At the same time, however, intense competition may increase payment for the contractor as well. The elasticity is 0.376. Therefore, given a particular increase in competition, expected savings are likely to be larger than expected payment increases.<sup>21</sup> This is consistent with the existing auction literature (Kessel, 1971; Gupta, 2002). Market competition is still an important driver of procurement efficiency even in the multidimensional context.

The weight variables have largely consistent coefficients with prior expectation. The weights are interdependent on each other, but the own score impacts may be of particular interest. The coefficient of _wPAY_ is found to be significantly negative at -0.24. This means that if more weight is put on the low payment criterion, the proposed service fees would decline, i.e., be more competitive. Similarly, the coefficient of _wDUR_ is significantly negative in the duration equation.

> 21 Recall that energy savings and annual payment are almost the same in many ESCO projects, this means that intensified competition can bring net benefits to asset owners.
- 16 -

Regarding the energy savings potentials, energy-intensive facilities are found to receive the greater amounts of investment; the coefficient of _KWH_ is positive and significant in the investment equation. At the same time, the greater amount of investment naturally results in larger energy savings; the coefficient of _KWH_ is also significantly positive in the energy savings equation.

Older buildings and facilities also seem to have larger potential for investment; the coefficient is 1.28. However, the resulting energy savings are not necessarily clear. The estimated coefficient is -0.381 in the savings equation, which is insignificant.

By type of building, hospitals and schools are found to have positive potential for energy savings. From the technical point of view, two technologies are found to be particularly important for energy savings: lighting and cogeneration systems. With these technologies adopted, larger energy savings can be expected, though they are costly, too. The coefficients of _dLIT_ and _dCGS_ are significantly positive in the savings and payment equations. The finding looks consistent with a general view that significant energy savings can be achieved if the source of energy is changed, for instance, using cogenerating systems.

Regardless of prior expectation, the past award history has no significant impact on proposed energy savings and investment plans. Instead, the ESCO projects implemented by experienced firms tend to be costlier; the coefficient of ln _AWAD_ is positive at 0.529 in the payment equation. This may be interpreted to mean that firm resources are constrained. Firms that are engaged in other ESCO projects elsewhere are less competitive, because their marginal costs of undertaking additional work may be high.

By firm type, not surprisingly, energy supply companies, such as electricity and gas utilities, have the advantage of proposing high-energy savings. The coefficient of _dENEG_ is significant and particularly large at 0.84 in the savings equation. Energy supply companies can take advantage of their own businesses to propose cost-effective energy solutions.
- 17 -

Regarding institutional arrangements, the estimation results show that the choice between the shared and guaranteed savings scheme does not really matter to energy savings and investment. The coefficients of _dSSC_ are insignificant in both investment and saving equations.

Packaging also does not help much to increase energy savings, either. Rather, it may reduce energy savings; the coefficient of _dPCKG_ turned out negative in the energy savings equation. The possible reason may be that a bundled ESCO project is often composed of a number of very small facilities. Even if they are bundled, it seems difficult to improve energy efficiency in those facilities.

Of particular note, our results indicate that different pieces of information have different effects on the bidding strategy. Walk-through and the disclosure of reserve prices are found effective to increase energy savings. The coefficients of both _dRESV_ and _dWALK_ are positive and significant in the savings equation. Walk-through is also found to make the contract duration shorter and thus investment more efficient. The coefficient is estimated at -1.576.

On the other hand, the preannounced energy savings target and risk-sharing matrix play an important role in promoting investments in technologies. The estimated coefficients are 0.588 and 0.917 in the investment equation, respectively. However, the required payments are also likely to be greater to compensate for increased investment.

The impact of risk sharing seems to depend on type of risk. According to our estimation results, the regulatory risk is most critical to ESCO projects. If the governments take the regulatory risk, contractors are likely to propose larger investment plans and higher energy savings. The coefficient of _RISKREG_ is positive and significant in both investment and savings equations. No impact is found for other types of risk.

The policy implications of the above findings are straightforward: First, multidimensional auctions work well, as expected by theory. The market competition is important to promote
- 18 -

energy savings. Second, the public sector needs to take certain risks, such as regulatory risk. Third, information disclosure is essential to encourage more investment and energy savings. This is consistent with the earlier literature (De Silva et al. 2008; Flambard et al. 2007) and reconfirms the fact that a fundamental issue in the ESCO procurement is uncertainty in achievable energy savings. Providing more information, for example, through walk-through and energy auditing, can help to mitigate uncertainty and promote competition among contenders.
- 19 -

**<u>Table 3. 3SLS Estimation Result</u>**

|Dep. Var.|ln_INV_|ln_SAVE_||ln_DUR_||ln_PAY_|
|---|---|---|---|---|---|---|
||Coef. Std.Err.|Coef.|Std.Err.|Coef.|Std.Err.|Coef. Std.Err.|
|_dTAGT_|0.588 (0.236) **|0.237|(0.197)|0.058|(0.088)|0.495 (0.238) **|
|_dRISKMT_|0.917 (0.331) ***|-0.013|(0.276)|-0.169|(0.123)|1.743 (0.334) ***|
|_dRESV_|0.085 (0.258)|0.394|(0.216) *|-0.005|(0.096)|0.056 (0.261)|
|_dWALK_|1.187 (0.898)|1.928|(0.750) ***|-1.576|(0.334) ***|<br>0.988 (0.908)|
|_RISKTYPO_|0.002 (0.555)|0.327|(0.464)|-0.679|(0.207) ***|<br>-0.028 (0.561)|
|_RISKREG_|0.497 (0.215) **|0.786|(0.180) ***|-0.030|(0.080)|0.299 (0.217)|
|_RISKINF_|-0.004 (0.369)|-0.366|(0.308)|-0.420|(0.137) ***|<br>0.445 (0.373)|
|_wINV_|0.003 (0.025)|-0.093|(0.021) ***|-0.019|(0.009) **|0.021 (0.025)|
|_wSAVE_|0.045 (0.023) *|0.018|(0.019)|0.027|(0.009) ***|<br>0.053 (0.024) **|
|_wDUR_|0.041 (0.033)|-0.016|(0.027)|-0.033|(0.012) ***|<br>0.084 (0.033) **|
|_wPAY_|-0.101 (0.058) *|-0.156|(0.048) ***|0.076|(0.022) ***|<br>-0.240 (0.058) ***|
|ln_N_|0.322 (0.216)|0.569|(0.181) ***|0.266|(0.080) ***|<br>0.376 (0.219) *|
|_dSSC_|-0.236 (0.337)|-0.223|(0.281)|0.790|(0.125) ***|<br>0.242 (0.340)|
|_dPCKG_|-0.190 (0.305)|-0.517|(0.255) **|-0.088|(0.113)|-0.375 (0.308)|
|ln_KWH_|0.529 (0.179) ***|0.642|(0.150) ***|0.224|(0.067) ***|<br>0.222 (0.181)|
|ln_AGE_|1.287 (0.456) ***|-0.381|(0.381)|-0.190|(0.170)|1.749 (0.461) ***|
|ln_AREA_|0.120 (0.234)|-0.075|(0.195)|-0.354|(0.087) ***|<br>0.486 (0.236) **|
|_dOFFC_|-0.454 (0.289)|0.306|(0.242)|-0.011|(0.108)|-0.527 (0.292) *|
|_dHOSP_|-0.318 (0.321)|0.499|(0.268) *|-0.082|(0.119)|-0.159 (0.324)|
|_dSCHL_|0.261 (0.208)|0.605|(0.174) ***|-0.042|(0.077)|0.164 (0.210)|
|_dLIT_|0.932 (0.300) ***|0.516|(0.251) ***|-0.004|(0.112)|1.010 (0.304) ***|
|_dAC_|-0.816 (0.625)|-0.854|(0.522) *|-0.111|(0.232)|-1.318 (0.632) **|
|_dHOTW_|0.244 (0.192)|0.164|(0.160)|-0.027|(0.071)|0.166 (0.194)|
|_dCGS_|0.430 (0.187) **|0.693|(0.157) ***|0.055|(0.070)|0.529 (0.189) ***|
|ln_AWAD_|0.059 (0.037)|0.026|(0.031)|-0.075|(0.014) ***|<br>0.095 (0.038) **|
|_dLEAS_|0.067 (0.337)|0.462|(0.282) *|-0.186|(0.125)|-0.013 (0.341)|
|_dMANU_|0.189 (0.186)|-0.040|(0.156)|-0.185|(0.069) ***|<br>0.300 (0.188) *|
|_dENEG_|0.338 (0.294)|0.840|(0.246) ***|-0.098|(0.109)|0.322 (0.297)|
|constant|-5.485(2.750)**|6.689|(3.376)**|6.559|(1.502)***|<br>-2.978(4.084)|
|Obs|68||||||
|R-square|0.873|0.927||0.889||0.900|
|No. of dummy variables|||||||
|Bidder origin prefecture|<br>11|11||11||11|
|Contractyear|8|8||8||8|

Note: A system of four equations is estimated by the three stage least squares. The number of bidders is treated as endogenous. Standard errors are shown in parentheses. *, **, *** indicate the statistical significance at the 10%, 5% and 1%, respectively.
- 20 -

# **VI.  Discussion**

There are several statistical issues in the above estimation. First, one might consider whether the bidder participation is exogenous or endogenous. If it is not endogenous, the conventional seemingly unrelated regression (SUR) can provide a more efficient estimate. To test this, the Hausman (1978) exogeneity test is applied. The exogeneity hypothesis can be rejected easily with a test statistic of 102.65. Thus, the bidder participation is concluded to be endogenous. This validates the current paper’s empirical approach based on the 3SLS method.

Second, the system-of-equations approach may remain arguable, although it is consistent with multidimensional auction theory. Theory tells that price and other quality bids are interdependent. The Breusch-Pagan test of independence can be used to examine whether correlations exist among different equations. The independence hypothesis can be rejected with a chi-square test statistic of 129.3 (Table 4). Therefore, the data are supportive of the system estimation, rather than ordinary least squares.

As shown in Table 5, the estimated correlations are all positive, though numerically relatively small at 0.0005 to 0.1. As expected, the table indicates that the more investment, the more energy savings; and the more investment, the more service fees. Larger savings also require longer contract periods. These are a clear indication of the difficult tradeoffs inherent in the ESCO projects.

**<u>Table 4. System Heteroskedasticity Tests</u>**

|Equation|Test statistics|_p_-value|
|---|---|---|
|Overall system|<br>129.327|<br>0.000|
|ln_INV_|0.019|<br>0.890|
|ln_SAVE_|4.597|<br>0.032|
|ln_DUR_|0.244|<br>0.621|
|ln_PAY_|1.727|<br>0.189|

**<u>Table 5. Covariance matrix of residual across equations</u>**

||lnINV|lnSAVE|lnDUR|lnPAY|
|---|---|---|---|---|
|lnINV|0.134||||
|lnSAVE|0.095|0.094|||
|lnDUR|0.018|0.005|0.019||
|lnPAY|0.102|0.077|0.007|0.137|
- 21 -

One might also be concerned about the potential heteroskedasticity of the sample data. The sample projects vary in size (e.g., investment amount) by a factor of 100 (see Table 2), although the vast majority are relatively small, involving investment of less than JPY100 million. In our estimation, the possible size effects are controlled by two explanatory variables: energy use ( _KWH_ ) and area of floor ( _AREA_ ).<sup>22</sup> Not surprisingly, they are highly correlated to the investment amount; simple correlations are also high at 0.674 and 0.679, respectively.

In the estimation results, the homoscedasticity is also by and large accepted by the single equation Breusch-Pagan tests. The results are also shown in Table 4. The homoscedasticity hypothesis can strongly be rejected for the three equations: _INV_ , _PAY_ and _DUR_ . For the energy savings equation, the homoscedasticity can be rejected at the five percent significance level, but cannot be at the one percent level. Therefore, the potential heteroskedasticity is largely controlled by covariates in our model.

Finally, there may be general concern about overspecification. Our model in fact includes a large number of independent variables despite the fairly small sample size. This is unavoidable to a certain extent, given the complex nature of ESCO projects. They are different in many aspects.

The degree of freedom can be increased, though only slightly, by focusing on certain aspects of auction design. The main estimation results are found largely to hold regardless of the selection of independent variables.<sup>23</sup> Tables 6 and 7 show the estimation results when only the information disclosure effects and the risk sharing effects are considered, respectively.

> 22 Engineering cost estimates are often used in the literature. Unfortunately, such a normalization factor is not available in the ESCO sector, because procurers ex ante do not have a concrete view on what their energy efficiency projects could look like. This is the fundamental reason why the ESCO procurement relies on multidimensional auctions.

> 23 Bidder origin and contract year fixed effects cannot be omitted, because they are found important to specify the model. The hypothesis that these fixed effects are indifferent from zero can be strongly rejected. The estimated chi-square test statistics are 326.8 and 226.6, respectively.
- 22 -

The findings are broadly consistent with Table 3: Competition matters. Information disclosure is essential. In particular, walk-through is effective to promote energy savings. On the other hand, investments are stimulated by setting an energy saving target and declaring the risk sharing mechanism upfront. The public sector needs to take certain risks, such as regulatory risk.

||**Table 6. 3SLS estimatio**<br>|**n result with informatio**<br>|**n disclosure variables on**<br>|**ly**<br>|
|---|---|---|---|---|
|Dep. Var.|ln_INV_|ln_SAVE_|ln_DUR_|ln_PAY_|
||Coef.<br>Std.Err.|Coef.<br>Std.Err.|Coef.<br>Std.Err.|Coef.<br>Std.Err.|
|_dTAGT_|0.676 (0.212) ***|0.213 (0.191)|0.076 (0.081)|0.676 (0.216) ***|
|_dRISKMT_|0.943 (0.301) ***|0.229 (0.272)|-0.174 (0.114)|1.593 (0.307) ***|
|_dRESV_|-0.040 (0.244)|0.301 (0.220)|0.158 (0.093) *|-0.160 (0.249)|
|_dWALK_|0.841 (0.828)|1.725 (0.748) **|-0.964 (0.315) ***|0.284 (0.845)|
|_wINV_|0.019 (0.024)|-0.071 (0.022) ***|-0.011 (0.009)|0.030 (0.025)|
|_wSAVE_|0.049 (0.021) **|0.008 (0.019)|0.023 (0.008) ***|0.071 (0.021) ***|
|_wDUR_|0.035 (0.033)|-0.026 (0.030)|-0.021 (0.012) *|0.076 (0.034) **|
|_wPAY_|-0.092 (0.059)|-0.162 (0.053) ***|0.070 (0.022) ***|-0.217 (0.060) ***|
|ln_N_|0.471 (0.189) **|0.679 (0.171) ***|0.126 (0.072) *|0.615 (0.193) ***|
|_dSSC_|-0.048 (0.327)|0.043 (0.295)|0.918 (0.124) ***|0.332 (0.334)|
|_dPCKG_|-0.195 (0.316)|-0.486 (0.286) *|-0.051 (0.120)|-0.425 (0.323)|
|ln_KWH_|0.610 (0.161) ***|0.641 (0.146) ***|0.170 (0.061) ***|0.397 (0.165) **|
|ln_AGE_|1.504 (0.417) ***|-0.020 (0.376)|0.083 (0.158)|1.765 (0.425) ***|
|ln_AREA_|0.065 (0.193)|-0.031 (0.174)|-0.161 (0.073) **|0.276 (0.196)|
|_dOFFC_|-0.490 (0.270) *|0.202 (0.243)|-0.199 (0.102) *|-0.443 (0.275) *|
|_dHOSP_|-0.482 (0.286) *|0.251 (0.258)|-0.295 (0.109) ***|-0.188 (0.291)|
|_dSCHL_|0.172 (0.207)|0.552 (0.187) ***|-0.045 (0.079)|0.041 (0.211)|
|_dLIT_|0.894 (0.311) ***|0.465 (0.281) *|-0.061 (0.118)|0.999 (0.317) ***|
|_dAC_|-0.690 (0.631)|-0.839 (0.569)|0.009 (0.240)|-1.135 (0.643) *|
|_dHOTW_|0.323 (0.186) *|0.266 (0.168)|-0.143 (0.071) **|0.271 (0.190)|
|_dCGS_|0.334 (0.175) *|0.665 (0.158) ***|0.076 (0.066)|0.363 (0.178) **|
|ln_AWAD_|0.056 (0.036)|0.042 (0.032)|-0.070 (0.014) ***|0.075 (0.037) **|
|_dLEAS_|-0.078 (0.326)|0.401 (0.294)|-0.232 (0.124) *|-0.222 (0.332)|
|_dMANU_|0.225 (0.182)|0.059 (0.164)|-0.109 (0.069)|0.260 (0.186)|
|_dENEG_|0.164 (0.294)|0.645 (0.265) **|-0.137 (0.112)|0.169 (0.300)|
|constant|-6.896(3.206)**|5.616(2.894)*|3.204(1.218)***|-3.701(3.270)|
|Obs|68||||
|R-square|0.861|0.907|0.874|0.889|
|No. of dummy<br>variables|||||
|Bidder origin|||||
|prefecture|11|11|11|11|
|Contractyear|8|8|8|8|

Note: A system of four equations is estimated by the three stage least squares. The number of bidders is treated as endogenous. Standard errors are shown in parentheses. *, **, *** indicate the statistical significance at the 10%, 5% and 1%, respectively.
- 23 -

|**T**|**able 7. 3SLS estimatio**|**n result **|**with risk shar**|**ing variab**|**les only**||
|---|---|---|---|---|---|---|
|Dep. Var.|ln_INV_|ln_SAVE_||ln_DUR_||ln_PAY_|
||Coef. Std.Err.|Coef.|Std.Err.|Coef.|Std.Err.|Coef. Std.Err.|
|_RISKTYPO_|-0.463 (0.442)|-0.598|(0.361) *|-0.294|(0.178) *|-0.233 (0.504)|
|_RISKREG_|0.613 (0.226) ***|0.757|(0.184) ***|-0.032|(0.091)|0.506 (0.257) **|
|_RISKINF_|-0.422 (0.310)|-0.716|(0.253) ***|-0.131|(0.125)|-0.332 (0.353)|
|_wINV_|-0.013 (0.024)|-0.073|(0.019) ***|-0.035|(0.010) ***|-0.012 (0.027)|
|_wSAVE_|0.057 (0.019) ***|0.027|(0.015) *|0.021|(0.008) ***|0.084 (0.022) ***|
|_wDUR_|0.030 (0.035)|-0.020|(0.028)|-0.026|(0.014) *|0.066 (0.039) *|
|_wPAY_|-0.082 (0.046) *|-0.078|(0.037) **|-0.007|(0.019)|-0.207 (0.052) ***|
|ln_N_|0.568 (0.233) **|0.668|(0.190) ***|0.107|(0.094)|0.861 (0.265) ***|
|_dSSC_|-0.484 (0.301) *|-0.411|(0.246) *|0.605|(0.122) ***|0.202 (0.344)|
|_dPCKG_|-0.484 (0.305)|-0.368|(0.249)|-0.149|(0.123)|-0.891 (0.348) ***|
|ln_KWH_|0.561 (0.179) ***|0.529|(0.146) ***|0.253|(0.072) ***|0.365 (0.205) *|
|ln_AGE_|0.155 (0.382)|-0.828|(0.311) ***|0.093|(0.154)|0.236 (0.435)|
|ln_AREA_|-0.092 (0.236)|-0.170|(0.192)|-0.265|(0.095) ***|0.166 (0.269)|
|_dOFFC_|0.047 (0.282)|0.458|(0.230) **|-0.075|(0.114)|0.214 (0.322)|
|_dHOSP_|0.238 (0.294)|0.868|(0.240) ***|-0.266|(0.119) **|0.549 (0.336) *|
|_dSCHL_|0.584 (0.180) ***|0.610|(0.147) ***|0.106|(0.073)|0.467 (0.206) **|
|_dLIT_|0.877 (0.284) ***|0.471|(0.232) **|0.096|(0.115)|0.743 (0.324) **|
|_dAC_|-0.555 (0.510)|-1.140|(0.416) ***|0.107|(0.206)|-0.525 (0.581)|
|_dHOTW_|0.209 (0.155)|0.348|(0.127) ***|-0.029|(0.063)|0.042 (0.177)|
|_dCGS_|0.506 (0.172) ***|0.691|(0.140) ***|0.129|(0.069) *|0.474 (0.196) **|
|ln_AWAD_|0.004 (0.033)|0.011|(0.027)|-0.058|(0.013) ***|0.016 (0.037)|
|_dLEAS_|0.282 (0.306)|0.419|(0.249) *|0.057|(0.123)|0.045 (0.348)|
|_dMANU_|0.005 (0.180)|-0.028|(0.147)|-0.095|(0.073)|-0.011 (0.206)|
|_dENEG_|0.877 (0.248) ***|0.819|(0.202) ***|0.072|(0.100)|1.092 (0.283) ***|
|constant|1.486(3.440)|13.257|(2.808)***|2.969|(1.389)**|3.523(3.923)|
|Obs|71||||||
|R-square|0.844|0.914||0.856||0.852|
|No. of dummy variables|||||||
|Bidder origin prefecture|<br>11|11||11||11|
|Contractyear|8|8||8||8|

Note: A system of four equations is estimated by the three stage least squares. The number of bidders is treated as endogenous. Standard errors are shown in parentheses. *, **, *** indicate the statistical significance at the 10%, 5% and 1%, respectively.
- 24 -

From the policy point of view, an important question is how the governments could stimulate energy efficiency investments, such as ESCO projects. The above estimations have already provided some insight into how to design auctions (e.g., information disclosure). Subsidies are another way of motivating municipalities to take measures to improve energy efficiency in their assets. In Japan, indeed, the central government has several intergovernmental subsidy schemes to support energy efficiency investments by subnational governments. They normally finance one-third to one-half of the eligible investment components.

About 80 percent of ESCO projects in the sample received the central government subsidies. This does not matter to contractors from the financial point of view, because there is no difference in payment regardless of whether a project is financed by the central or local government. However, the application to the subsidy schemes does matter to the bidding strategy to the extent that only certain technologies are eligible for subsidies. Therefore, proposed technologies may be biased towards eligible components, and energy savings and investment amounts will be affected.

By including the amount of subsidy in the model, the subsidy impacts are estimated (Table 8).<sup>24</sup> The subsidies are found to promote energy savings and investment; the coefficient of ln _SUBS_ in the investment equation is positive and significant. The elasticity is estimated at 0.058, which is moderate. The impact on energy savings is also positive. The elasticity is also small at 0.039. In addition, increased investment brings about larger payments in a longer period. All the indications are that the financial subsidies may not be an effective policy option from the general point of view.

> 24 The sample size is limited for this specification, because the subsidy information is missing in some ESCO projects.
- 25 -

|**T**|**able 8. 3SLS Estimatio**|**n result with central g**|**overnment subsidies**||
|---|---|---|---|---|
|Dep. Var.|ln_INV_|ln_SAVE_|ln_DUR_|ln_PAY_|
||Coef. Std.Err.|Coef. Std.Err.|Coef. Std.Err.|Coef. Std.Err.|
|ln_SUB_|0.058 (0.009) ***|0.039 (0.009) ***|<br>0.018 (0.004) ***|0.047 (0.011) ***|
|_dTAGT_|0.416 (0.255) *|-0.014 (0.250)|0.127 (0.096)|0.491 (0.306) *|
|_dRISKMT_|1.538 (0.378) ***|0.787 (0.371) **|-0.221 (0.142)|2.587 (0.454) ***|
|_dRESV_|-0.975 (0.343) ***|-0.486 (0.336)|-0.159 (0.128)|-0.625 (0.411)|
|_dWALK_|1.027 (0.746)|2.347 (0.731) ***|<br>-1.454 (0.279) ***|2.310 (0.895) ***|
|_RISKTYPO_|1.121 (0.597) *|0.988 (0.585) *|-0.728 (0.223) ***|0.553 (0.715)|
|_RISKREG_|-0.062 (0.295)|0.293 (0.289)|-0.140 (0.110)|-0.047 (0.354)|
|_RISKINF_|0.571 (0.350) *|0.243 (0.343)|-0.482 (0.131) ***|0.929 (0.419) **|
|_wINV_|0.046 (0.026) *|-0.062 (0.025) **|-0.011 (0.010)|0.042 (0.031)|
|_wSAVE_|0.022 (0.018)|-0.010 (0.018)|0.018 (0.007) ***|0.012 (0.022)|
|_wDUR_|0.084 (0.031) ***|-0.015 (0.031)|-0.031 (0.012) ***|0.056 (0.038)|
|_wPAY_|0.046 (0.057)|-0.119 (0.056) **|0.096 (0.021) ***|-0.285 (0.069) ***|
|ln_N_|-0.143 (0.185)|0.269 (0.181)|0.168 (0.069) **|-0.013 (0.222)|
|_dSSC_|-0.891 (0.276) ***|-0.721 (0.271) ***|<br>0.610 (0.103) ***|-0.220 (0.331)|
|_dPCKG_|0.177 (0.244)|-0.166 (0.239)|-0.079 (0.091)|0.095 (0.293)|
|ln_KWH_|0.208 (0.168)|0.328 (0.164) **|0.218 (0.063) ***|-0.112 (0.201)|
|ln_AGE_|0.943 (0.347) ***|-0.454 (0.340)|-0.445 (0.130) ***|1.634 (0.416) ***|
|ln_AREA_|0.362 (0.193) *|0.202 (0.189)|-0.347 (0.072) ***|0.834 (0.231) ***|
|_dOFFC_|-0.435 (0.243) *|0.320 (0.239)|0.049 (0.091)|-0.470 (0.292) *|
|_dHOSP_|0.272 (0.243)|0.862 (0.238) ***|<br>0.097 (0.091)|0.174 (0.291)|
|_dSCHL_|0.309 (0.155) **|0.639 (0.152) ***|<br>-0.018 (0.058)|0.079 (0.186)|
|_dLIT_|0.436 (0.244) *|0.093 (0.239)|-0.188 (0.091) **|0.362 (0.293)|
|_dAC_|-0.061 (0.668)|-0.920 (0.655)|0.292 (0.250)|-1.761 (0.801) **|
|_dHOTW_|-0.041 (0.191)|-0.031 (0.187)|-0.147 (0.071) **|-0.182 (0.229)|
|_dCGS_|0.578 (0.190) ***|0.931 (0.186) ***|<br>-0.071 (0.071)|0.723 (0.228) ***|
|ln_AWAD_|0.177 (0.036) ***|0.135 (0.035) ***|<br>-0.061 (0.013) ***|0.221 (0.043) ***|
|_dLEAS_|0.054 (0.259)|0.313 (0.254)|-0.242 (0.097) **|-0.357 (0.310)|
|_dMANU_|0.603 (0.179) ***|0.313 (0.175) *|-0.222 (0.067) ***|0.613 (0.215) ***|
|_dENEG_|0.681 (0.213) ***|1.054 (0.209) ***|<br>0.005 (0.080)|0.561 (0.255) **|
|constant|-3.666(3.364)|8.658(3.297)***|<br>7.086(1.259)***|-0.816(4.033)|
|Obs|61||||
|R-square|0.944|0.956|0.951|0.937|
|No. of dummy variables|||||
|Bidder origin prefecture|<br>11|11|11|11|
|Contractyear|7|7|7|7|

Note: A system of four equations is estimated by the three stage least squares. The number of bidders is treated as endogenous. Standard errors are shown in parentheses. *, **, *** indicate the statistical significance at the 10%, 5% and 1%, respectively.
- 26 -

# **VII.  Conclusion**

Competitive bidding is an important policy tool to procure goods and services from the market at the lowest possible cost. However, it is not necessarily easy to design competitive bidding when an object to be procured is highly customized and involves uncertainty.

Unlike simple goods acquisition, energy efficiency services are complex contracts, involving various components. There are potentially a number of solutions that can achieve the same energy savings. Procurers ex ante do not know what the best solution is, and therefore ask bidders to propose a package of solutions.

In multidimensional auctions, procurers can balance potentially competing policy objectives by setting the scoring rule. Based on the evaluation rule, bids based on both price and other qualities are evaluated. The firm whose score is the highest wins the contract. Multidimensional auction theory exists, but there has been little empirical evidence to show how such auctions actually function.

This paper examined the bidding strategy in multidimensional auctions for ESCO projects in Japan. The results show that the competition effect matters to multidimensional auctions, as theory predicts. To promote energy efficiency investment, the risk sharing arrangements between the public and private sector are essential. In particular, the public sector should take the regulatory risk, such as changes in energy standards and building codes. This will help to ease uncertainty and allow ESCOs to propose competitive proposals. Information disclosure is also found important. While walk-through and preannouncement of reserve prices can promote expected energy savings, disclosed energy savings targets and risk sharing matrix are useful to encourage aggressive investment plans.
- 27 -

# **Annex I.  Energy efficiency projects in building sector**

Energy efficiency in buildings has a great potential to mitigate greenhouse gas emissions. Buildings are estimated to consume about 40 percent of the world’s final energy (IEA 2008; World Bank 2010; Worldwatch Institute 2009). In general, energy efficiency projects are cost-effective and pay for themselves (Figure 4). Importantly, they are a win-win solution for both asset owners and service providers. While asset owners can reduce their energy spending, ESCOs make profit on the sales of energy services.

The evidence is clearly supportive of significant potential of energy savings in buildings. For new buildings in Europe, the cumulative energy savings from building codes is about 60 percent more than for those built before the first oil shock in the 1970s (WEC 2008; World Bank 2010). In Brazil, the National Electrical Energy Conservation Program aims to retrofit government buildings and save 140GWh of energy per year (UNEP 2010). In the developing world, managing peak energy demand for buildings―namely, air conditioning―will be an important challenge, since a significant amount of energy is used for air conditioning, and this demand seems to continue increasing.<sup>25</sup> Notably, however, energy efficiency projects are not necessarily low-hanging fruits. There are many institutional barriers, particularly in the public sector. Singh et al. (2010) point out some major barriers: (i) lack of awareness and information about benefits and costs, (ii) lack of technical capacity for energy audits, project design and implementation, and operation and maintenance, (iii) limited incentives for governments to improve energy efficiency, reduce budget, and take risks of new technology, (iv) lack of public accountability for energy savings under proper reporting and monitoring systems, (v) restrictive procurement,

> 25 The electricity demand for air conditioning is very price-inelastic. Reiss and White (2005) show that the price elasticity for households with central or air conditioning is three times higher (in absolute terms) than that for those who have no air conditioning. As the result, the impact of air conditioning can be substantial, as in the United States during the late 1980s, when it was estimated to have reached about one-third of the total electricity peak demand (Andrew, 1989). In Hong Kong SAR, China, heating, ventilation and air-conditioning are found to be the single largest electricity end-user, accounting for 30 to 60 percent of the total electrical demand during the hot summer months of July and August (Lam et al., 2003).
- 28 -

contracting, and financial rules, (vi) lack of upfront funding for audits and investment, and (vii) high transaction costs associated with small size of projects.

Public procurement rules and guidelines are a particularly difficult obstacle to ESCO projects, because they are based on input-based price competition. By construction, procurers must know what the best technology is; all technical standards and other specifications need to be determined before issuing tender notices. However, this is not always the case, particularly when an object to be procured is technically complex and highly customized, as in the ESCO market.<sup>26</sup> The ESCO markets are full of uncertainties. Technology can evolve very rapidly nowadays (Figure 5). Of course, energy efficiency technologies commonly used, such as energy efficient lighting, chillers and cogeneration systems, are already well established. However, how to combine those technologies to achieve the maximum energy savings is still not obvious. The solution must be building-specific, and there are a number of ways of combining available technologies.

Uncertainty in cost of energy saving technology is also significant. Today’s most costeffective technology may not necessarily be the best one tomorrow. For instance, the cost of solar photovoltaic power declined from $25 per watt in 1979 to $4 per watt in 2001 (World Bank 2010). The cost of light-emitting diodes (LED) lamps has also declined in recent years, though they are still far more expensive than traditional and compact fluorescent light (CFL)

> 26 In the traditional public procurement systems, several procedures exist to purchase new technology. First, the two-stage approach can be used, in which firms are asked to propose new ideas, and the procurer then determines a specification and invites price bids from all participants. But there is little incentive for private firms to reveal their truly innovative proposals without any assurance of the final contract award from the procurer. Second, a new idea or design can be developed and identified through consulting services separately from the main work. However, this method does not motivate contractors to take advantage of the synergistic effect among design, construction, operation, and maintenance. Finally, the lifecycle cost-benefit analysis can be used if there is any bid evaluation formula that everyone can unambiguously agree on. Technically speaking, the current procurement guidelines often allow combining monetary and nonmonetary bids into a single measurement. However, it is difficult to agree on all the costs and benefits beforehand, when an auctioned object is complex and customized.
- 29 -

bulbs. But this may change in the future. Given the expected demand effect, cost effectiveness of new technologies can improve dramatically.

The choice between such technologies is theoretically dependent on the market energy and carbon prices. But these prices remain highly uncertain and volatile. There is no agreement on how much the carbon price should be.<sup>27</sup> All these uncertainties make the traditional public procurement systems less useful for promoting energy efficiency projects.

Because of flexible weighting, multidimensional auctions are considered to be an effective system for the simultaneous evaluation of various factors, such as contractors’ abilities and technical reliability of their proposals. This is essential for the ESCO procurement, because individual proposals are different and unique. However, it is also noteworthy that the evaluation method should be standardized to a certain extent for transparency purposes (Singh and others 2010).

One important key feature of ESCO projects is that they usually rely on performance-based contracting. In an ESCO project, any individual inputs, such as equipment and generators, are not a meaningful measurement to verify the completion of the contract. What really matters is the ultimate outcome, which is energy savings in this case. In other words, facility owners purchase energy efficiency services, not particular goods. ESCOs deliver energysaving solutions and are responsible for achieving the agreed target. As the result, facility owners can transfer many technical and commercial risks to the private sector through performance-based contracting.

> 27 There is no theoretical convergence of the social cost of carbon (SCC). The last IPCC report estimates the SCC at $3 to $95 per tCO2 (IPCC 2007). Tol (2005), surveying more than 100 estimates, shows that the estimates have a long tail of the distribution from nearly zero to over $100 per tCO2; the mean is $25. The UK government recommends an SCC of $28 per tCO2 in public decisions, with a range of $14 to $57 per tCO2 (Watkiss 2005). The United States proposes $21 per tCO2, with a range of $5 to $65 (U.S. Department of Energy 2010). The French government recommends a much higher price of $135 per tCO2 by 2030 (Quinet 2008).
<!-- Start of picture text --> Marginal mitigation cost ($/tC0,e) 7 : : - :<br>120<br>100 Efficiency in buildings: Land-use and land-use change, mostly in developing Advanced technologies: ———><br>80 residential and commercial; countries: reduced deforestation, grassland carbon capture and storage —><br>60 building envelope, heat & water management,changed agronomysoil restoration, afforestation,practices, livestock practices, \<br>4020 reduced/intensive| agricultural\ conversion——_<br>0<br>-20 = je \ I /<br>40 \ Small hydro and nuclear power<br>60 in developing countries Renewable energy: on- and off-shore wind,<br>-80 energyMore efficient co-generation; motors; solar photovoltaic energy, concentrated solar power<br>-100~120 ————- electricity= from- landfill. waste;: = Marginal- cost, all countries-<br>-140 Negative costs: Long-termee savings @ MitigationMitigation measure measure inin aa developing countryhigh-income country<br>-160 0 outweigh initial costs 10 0 20 40<br>Mitigation potential (GtC0,e/year)<br><!-- End of picture text -->

<!-- Start of picture text --> 10 10<br>9—Wind power 8= . 9<br>sees Fuel calls 5 ees ne<br>a — — Electric/Hybrid cars a 8<br>7 == LightingSolar PV EE : ‘4<br>reer Buildings EE = : r<br>=e All technology sectors .<br>6;[6<br>5,> é bh oa 5<br>4-4<br>3ES. .* 4<br>"4 7 “mle °<br>2 ee 2<br>aetnee ert le<br>Peeeeeeeee oon<br>0 . . 0<br>1980 1985 1990 1995 2000 2005<br><!-- End of picture text -->
- 31 -

# **Annex II.  A simple multidimensional auction model**

In a multidimensional auction, both monetary and nonmonetary bids are taken into account to select a winner. Theory suggests that a scoring auction is implementable (Che 1993; Branco 1997; Naegelen 2002; Asker and Cantillon, 2008).

Following Che (1993) and Naegelen (2002), a simple scoring auction model is considered: Suppose that a procuring agency purchases the good or service at price, _p_ , and value its quality, _q_ . Then, the procurer will maximize the following expected utility by specifying the scoring rule to put weights on _p_ and _q_ :

> The scoring rule, _S_ (: _p_ , _q_ ) → _s_ ,<sup>is often quasi-linear. Therefore, it can be written by:</sup>

Given its private information of project costs and the scoring rule, each contractor _i_ ∈{ ,1  , _N_ } submit a combination of price and quality ( _p_ , _q_ ) to maximize its expected profit: where _θi_ is _i_ ’s cost parameter distributed according to _F_ ( _θ_ ) over [ <u>θ</u> , θ ] . Since the scoring rule is quasi-linear to price, the optimal level of quality is determined by:
- 32 -

Given the optimal quality bid, _q_<sup>*</sup> , each contractor offers the optimal price _p_<sup>*</sup> , as in standard auction theory.

Che (1993) suggests that the optimal mechanism for maximizing government profits can be implemented in the two-stage, first- or second-score bid system. In the first stage, the auctioneer chooses the best bidder in terms of the score and then negotiates with the selected firm the optimal quality to be delivered. It is also shown that the procurer needs to be committed strongly to the optimal awarding rule in advance, because the selected weight on quality tends to deviate from the social optimum weight (also see Branco 1997; Naegelen 2002).

Note that a single pseudotype of contractor, such as _θ_ , is sufficient to characterize the equilibrium outcome in any multidimensional auction with more than one dimension of private information (Asker and Cantillon, 2008). Thus, the above model can be generalized and applied to many auction practices.
- 33 -

# **Annex III.  ESCO Market in Japan**

The Japanese ESCO market has been growing in recent years; the total amount of the public and private ESCO contracts reached JPY40 billion or about US$350 million in 2007 (Figure 6). About half were implemented in the industrial sector; another half in the residential sector. When other energy efficiency works, such as stand-alone investment without performance contracts and energy audit, are included, the size of the whole energy savings market was estimated at JPY64 billion or US$550 million per year.

This paper focuses on public-sector energy efficiency projects, which accounted for some 20 percent of the total market in Japan. According to the JAESCO survey, over 1,100 energy efficiency projects were carried out in 20001-2005, out of which 189 were implemented in the public sector (Figure 7). As far as ESCO projects―which this paper defines as performance-based energy savings contracts―are concerned, about 15 percent of them were public works.

The reasons for this paper’s focusing the public ESCO market have been twofold. First, the public sector is expected to have an important role in developing the ESCO market and catalyzing private investment in energy efficiency technology, especially in developing countries where the market has not yet developed. Second, there is an issue of data availability. It is generally difficult to obtain the details of private ESCO contracts in industry.

The Japanese public ESCO market has been matured, and thus, there are a considerable amount of transactions, of which the detailed data are available. The survey data for this study covers 171 sub-national government buildings and facilities, for which 100 ESCO projects were implemented during the period: 1998-2012. The data include a wide range of public facilities, from administrative offices to schools and hospitals (Figure 8).
<!-- Start of picture text --> 1<br>© Other<br>Auwdit + Energy e ficiency work<br>© Energy efficiency work only<br>© Energy sevice povider— Onsite generation<br>Mo oAwdit+ Energy efficiency work+ Performance contract<br>(PY billion) 637<br>a0 557 |<br>515 == 497 4g)<br>ara<br># —]<br>3 265<br>D» 170 186 .<br>0<br>LJ l —_36. 74 ” 303 Pes ( IPY40.6 billion<br>1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 FY<br><!-- End of picture text -->

<!-- Start of picture text --> rT<br>.<br>rT<br>:<br>rT<br>:<br><!-- End of picture text -->
- 35 -

**Figure 8. Number of Public Facilities under ESCO projects**

<!-- Start of picture text --> 40<br>35<br>30<br>25<br>20<br>15<br>10<br>5<br>0<br>Note: Some of the projects involve more than one facility.<br>Source : Author’s survey data.<br>Admin. office Hospital Community center School Pool Gym Library Hotel Museum Welfare facility Athletic field Research center University Skate link Other<br><!-- End of picture text --> The Japanese ESCO projects are mainly motivated for two objectives: (i) to reduce energy spending, and (ii) to contribute to reducing emissions (Figure 9). Some projects were partially motivated to support industrial policies, such as local job creation. These policy objectives are, by and large, reflected by the weighted evaluation criteria.<sup>28</sup> More weights are put on energy savings and technical and reliability aspects. The most popular criterion is about concreteness and reliability of technologies proposed, followed by the reliability of O&M, monitoring and verification, and the 15-year total revenue criterion.<sup>29</sup> ESCOs propose different technologies, depending on the evaluation criteria. Major technologies used are energy-efficient lighting and air conditioning systems. The total investment amounts to about JPY250 million on average (Table 9). Investment is financed directly by a facility owner (i.e., municipality) if the contract follows the guaranteed savings

> 28 See Annex Table for the list of criteria used in the Japanese ESCO market.

> 29 Despite the large number of criteria used, the weights are remarkably similar to each other. On average each criterion has a weight of 6.5 to 8 out of 100. This means that procurers’ preferences may not be represented by the attached weights but by the selection of criteria. For each project, the evaluation formula involves about 1215 criteria.
- 36 -

scheme. If it is based on the shared savings arrangement, investment is funded by a contractor.<sup>30</sup> The central government often subsidizes part of investment in ESCO projects in Japan. In most cases, one-third to a half of the eligible investment is supported by the central government. The average amount of subsidy is JPY63 million, if it is granted. It is often that 70-90 percent of the total investment is eligible for the central government’s subsidies. Thus, the amount of subsidy will be JPY50-80 million if the subsidy rate is one-third.

The energy efficiency investment is expected to bring about a 20 percent reduction of energy use. This does not seem different across types of facilities (Figure 10). Notably, these levels of energy savings are not particularly high by global standards. Many public buildings in developing countries can easily achieve 20 to 40 percent of energy savings by retrofitting existing equipment (Singh et al. 2010).

It is worth noting that the Japanese economy is generally energy-efficient, and so are public buildings. The survey data used herein show that the average building-specific energy use is about 150kWh per m<sup>2</sup> , and most facilities consumed less than 200kWh per m<sup>2</sup> even before the energy efficiency projects (Figure 11).<sup>31</sup> Hence, it is very likely that the expected energy savings would be modest, regardless of the relatively large amount of investment.

The reduced energy use has two implications: On one hand, facility owners can save their energy spending. This amounts to on average JPY30 million per annum. Notably, most of these expected savings will be paid to contractors as service fees. On the other hand, the reduced energy use also means a reduction of CO2 emissions, which amount to 790 tCO2 in a

> 30 The maximum investment in our sample amounted to JPY1.7 billion in the case of a hospital in the prefecture of Kanagawa, which has 12 floors and consumes more than 40GWh.

> 31 European average building-specific energy use is 203kWh per m2 (Kingspan Insulated Panel 2011).
- 37 -

Japanese public ESCO project.<sup>32</sup> The expected CO2 reduction rate is about 18 percent on average.

There are several important implications of these figures. First of all, energy saving potential is highly building-specific. Not surprisingly, large-scale, energy-intensive buildings, such as hospitals, can expect more savings in energy spending (Figure 12). For small facilities consuming less energy, such as administrative offices, the energy saving potential varies considerably.

The payback period of the public ESCO projects in Japan is much longer than a commercial norm. In the private sector, the maximum payback period that is acceptable is about eight to nine years, which is 60 percent of the statutory useful life of normal equipment (i.e., 15 years). However, the public-sector ESCO projects sometimes have a payback period of more than 10 years. In several cases, it exceeds 30 years (Figure 13).<sup>33</sup> This long time horizon is consistent with a common view in the public sector that renovation of old equipment is necessary anyway; efficiency in investment is less important.

The long payback period also implied that a sustainable contractual and legal framework is vital between the public and private sectors. The average contract period in our sample projects is 9.5 years with a maximum of 15 years, during which many unanticipated events can happen. Thus, it is essential to establish a solid institutional framework to deal with those events. In the developing world, this may be an important challenge to develop the ESCO market. It is still difficult to make a contract of more than five years in many developing countries.

> 32 This is equivalent to the total amount of CO2 emissions from 10,000 passenger cars that drive about 500km from Tokyo to Osaka under the assumption that a passenger car emits 150gCO2 per km. It can be considerable, given the fact that the daily traffic between the two cities was about 70,000 vehicles in 2011.

> 33 In such a case, municipalities make 10-year contracts with ESCOs and will recover their investment beyond the contract period.
- 38 -

The Japanese ESCO projects are costly investment to reduce CO2 emissions. The unit cost is about JPY60,000 or $750 per tCO2 for a 10 year period (Figure 14). In general, the existing literature shows that the cost of the normal energy efficiency investment is on average $76 per ton of oil equivalent or $11 per barrel of oil over a 10-year investment period (Taylor and others 2008). However, the Japanese ESCO investment corresponds to $320 per barrel of oil.<sup>34</sup> This is largely because Japanese public buildings are already efficient in energy use, as discussed above.

**Figure 9. Policy Objectives of Energy Efficiency Projects by Japanese Municipalities**

|90||||
|---|---|---|---|
|0<br>10<br>20<br>30<br>40<br>50<br>60<br>70<br>80<br>Reduce<br>energy costs<br>Reduce total<br>O&M costs<br>f filiti<br>Meet energy<br>efficiency<br>tdd<br>Contribute t<br>emissions<br>dti<br>Number of projects|o<br>Support<br>industrial<br>li|||
|o aces<br>sanars<br>reucon|pocy|||
|Note: Multiple answers.<br>_Source_: Author’s survey data.<br>**Table 9. Investment, Outcomes (target) and Subsidy in**<br> <br>|**Japanese E**<br> <br>|**SCO Pr**<br>|**ojects**<br>|
|Obs<br>Mean|Std. Dev.|Min|Max|
|Investment (JPY million)<br>86<br>244.9|<br>248.7|<br>17.8|<br>1,738.5|
|Energy cost savings target (JPY million)<br>92<br>31.4|<br>34.4|<br>0.9|<br>171.0|
|Payment to ESCO (JPY million)<br>84<br>30.3|<br>32.9|<br>0.7|<br>144.9|
|CO2 reduction target (tCO2)<br>83<br>793.0|<br>1,061.8|<br>24.0|<br>4,900.0|
|CO2 reduction rate target(%)<br>5<br>18.1|<br>9.2|<br>10.5|<br>30.7|

_Source_ : Author’s survey data.

34 It is assumed that carbon dioxide emissions per barrel of crude oil are 0.43 tCO2 per barrel.
- 39 -

**Figure 10. Average energy savings rate by type of facility**

<!-- Start of picture text --> 25<br>22.6<br>21.5 21.1<br>20 19.7 19.3<br>16.7<br>15<br>10<br>5<br>0<br>Office Hospital School, Community Athletic Multiple<br>library, center, field, gym, facilities<br>institute, hotel, pool<br>museum welfare<br>facility<br>Energy savings rate (target, %)<br><!-- End of picture text --> _Source_ : Author’s survey data.

**Figure 11. Distribution of building-specific energy use in the sample**

<!-- Start of picture text --> 0 100 200 300 400 500<br>Building-specific energy use (kWh/m2)<br>.008<br>.006<br>.004<br>Density<br>.002<br>0<br><!-- End of picture text --> _Source_ : Author’s survey data.
- 40 -

**Figure 12. Energy savings rate and energy consumption**

<!-- Start of picture text --> 50<br>Office<br>45 Hospital<br>School, library, museum, institute<br>40 Community center, hotel, welfare facility<br>Field, pool, gym<br>35 Multiple facilities<br>30<br>25<br>20<br>15<br>Annual savings<br>= JPY80 million<br>10<br>JPY40 million<br>5 JPY20 million<br>JPY10 million<br>0<br>0 200 400 600 800 1000<br>Annual energy spending (JPY million)<br>Source : Author’s survey data.<br>Energy savings rate (target, %)<br><!-- End of picture text -->

**Figure 13. Distribution of imputed payback periods in the sample**

<!-- Start of picture text --> 0 10 20 30 40<br>Payback period (year)<br>Source : Author’s survey data.<br>.15<br>.1<br>Density<br>.05<br>0<br><!-- End of picture text -->

**Figure 14. Distribution of unit cost of energy savings in the sample**

<!-- Start of picture text --> 0 50000 100000 150000 200000 250000<br>Unit cost of energy savings<br>for 10 year period (JPY/tCO2)<br>Source : Author’s survey data.<br>2.0e-05<br>1.5e-05<br>Density 1.0e-05<br>5.0e-06<br>0<br><!-- End of picture text -->
- 41 -

# **Annex IV.  Evaluation weights in Japanese ESCO projects**

Evaluation criteria are markedly different across ESCO projects. More than 30 criteria were used in the sample projects used in this study. Some criteria, such as energy savings and technology reliability, are common, and others are not. For instance, some municipalities required ESCOs to take into account implications for local employment (Table 10). It is an empirical challenge to construct a weight matrix, _w_ , for this analysis. This is first because some criteria are similar but not exactly the same. For example, a CO2 emission reduction criterion is closely related to an energy savings criterion. But they are not exactly the same. Second, more importantly, one criterion may have implications to more than one dimension of dependent variables. For example, a short payback period is dependent on the investment amount and annual payment. Finally, some criteria are difficult to quantify, such as uniqueness of technologies and comprehensiveness of the investment plan.

To overcome these difficulties, the original scores are first normalized to a scale of 0 to 100. Second, several relevant evaluation criteria that aim at the same objective are aggregated to one of the four criteria. Simple correlations are compared if a criterion can be categorized into more than one dimension.

The weight on investment, _wINV_ , includes various technology-related criteria, because any technical requirements, by and large, have implications to investment. The more advanced technologies, the more costs. Accordingly, the weight includes the criteria of concreteness and reliability of proposed technologies, environmental consideration of NOx, SOx and noise, innovation and uniqueness of technologies, good plan to remove existing equipment, and comprehensiveness and balance of the overall proposal.
- 42 -

The weight on energy savings, _wSAVE_ , is composed of the criteria of energy savings guarantees and targets, annual revenue from energy spending reduction, and CO2 emission reduction, which is almost linearly related to energy savings.

The weight on the contract duration, _wDUR_ , consists of two criteria: short contract and payback period criteria. Finally, a weight on payment, _wPAY_ , is defined by the low service fee criterion.

**<u>Table 10. Criteria Used in ESCO Procurement in Japan</u>**

|Criteria|Share of auctions with<br>this criterion adopted|Avg weight if adopted<br>(scale of 0 to 100)|
|---|---|---|
|Concreteness and reliability of proposed technologies|<br>0.91|7.53|
|High CO2 emission reduction|0.86|7.33|
|Reliability of O&M, monitoring and verification|0.86|7.29|
|Total revenue for 15 years|0.83|7.63|
|Energy savings target requirement|0.83|7.25|
|High energy savings guaranteed|0.79|7.59|
|Environmental consideration of NOx, SOx and noise|0.78|7.10|
|Good plan to adjust existing equipment|0.78|7.14|
|Applicability for subsidy|0.77|7.82|
|Overall comprehensiveness and balance of proposal|0.72|7.55|
|Good proposal of after sales services|0.65|6.92|
|Quality management and timely completion of work|0.61|7.31|
|Innovation and uniqueness of technologies|0.60|7.10|
|Annual revenue|0.58|7.17|
|Reliable financial arrangement|0.57|6.76|
|Short contract period|0.50|6.50|
|Installation without preventing operations|0.26|6.74|
|Low and reliable investment costs|0.25|6.61|
|Safety and emergency measures|0.15|6.23|
|Local business involvement|0.12|6.73|
|Experience and reputation of ESCO|0.10|8.20|
|Low O&M costs|0.09|6.72|
|Low service fees paid to ESCO|0.06|7.59|
|Environmental awareness activity|0.05|8.47|
|Reliability of baseline and energy savings calculation|0.03|7.72|
|Standards for facilities to be used at times of disaster|0.03|6.39|
|Long energy savings guarantee period|0.02|8.33|
|Short payback period|0.01|8.33|
|Goodplan to remove existingequipment|0.01|5.88|

_Source_ : Author’s data.
- 43 -

# **Annex V.  An example of risk matrix for an ESCO project**

**<u>Table 11. An example of risk matrix attached to RFP in the prefecture of Niigata</u>**

||Type of risk|Description|City|ESCO|
|---|---|---|---|---|
|Throughout all|Typo in RFP|Significant error in RFP|X||
|stages|Error in proposal|Failure to meet the agreed energy savings target||X|
||Safety|Safety in design, construction and O&M||X|
||Environmental<br>protection|Environmental protection in design, construction<br>and O&M||X|
||Institutional change|Change in VAT|X||
|||Other taxes||X|
||Cancelation and delay|City's order and decline by the city council|X|X|
|||Manifestation of neighboring residents|X|X|
|||Delay of issuance of construction and other permits||X|
|||Delay of issuance of construction and other permits|X||
|||because of city's mismanagement|||
|||Contractor's default||X|
|Planning/design|/ Force majeure|Design change, cancelation or delay by weather<br>conditions|X|X|
|construction|Price changes|Rapid inflation or deflation|X|X|
||Design change|City's order|X||
|||Contractor's order||X|
||Proposal cost|Cost of preparing proposal||X|
||Financial risk|Assurance of necessary financial resources||X|
|Construction|Project delay|Project delays and late delivery||X|
||Cost overruns|Cost increases by city's order|X||
|||Cost increases by contractor's mismanagement||X|
||Performance|Mis-specification||X|
||Damage before delivery|Damage of facilities before their delivery||X|
|Payment|Change of service fees|Administrative cost of changing service fees|X|X|
||Delay of payment|Delay of payment because of city's delayed process|X||
|||Delay of payment due to modification of revenue||X|
|||Delay of payment because of delay of monitoring<br>and verification||X|
|||Delay of payment of penalty||X|
||Interest rate|Change in commercial interest rates||X|
||Warranty against defect|Warranty against hidden defects||X|
|O&M|Change of facility use|Change of facility use by city's order|X||
||Increase in O&M cost|Increase in O&M cost for other reasons||X|
||Entry permission|Failure of O&M due to the lack of entry permission<br>in the facility|X||
||Damage of assets|<br>Damage by fire or accidents on purpose||X|
|||Damage by fire or accidents (force majeure)|X||
|Monitoring/|Quality of equipment|Failure to operate at the designed full capacity||X|
|Verification|Monitoring/Verification|Mis-report of monitoring and verification results||X|
|||No or late access to city's information of monitoring<br>and verification results|X||
||Utility cost|Change of utility unit costs|X||
- 44 -

||Baseline adjustment|Major change in facility and energy use compared to<br>the baseline|X||
|---|---|---|---|---|
|Guarantee|Performance|Mis-specification||X|
|||Damage of mis-specification to city's operations||X|