output criteria – low inflation and higher average GDP growth – are the same for all countries. The issue also creates significant practical problems in estimating the effectiveness or regulators on outcomes since it may cause problems in combining developed and developing country samples. (CEE and some CIS countries are, at least for electricity, in an intermediate position.)

# **5.1.2  Methodological Issues in Empirical Work**

These issues are serious but less fundamental than the problem of defining relevant output measures discussed above.

Empirical testing of the performance of regulation seems now to concentrate on the application of _panel-data econometrics_ . (See Fink, Mattoo and Rathindran (2002). Guasch, Laffont and Straub (2002) Gutierrez (2003) and Pargal (2003) for good recent examples.) Case studies provide interesting narrative pointers but cannot allow a statistically robust separation of the effects of other changes (e.g. commercialization, privatization, unbundling, the introduction of competition, etc) from the impact of the establishment and development of an independent regulatory agency. Cross-section econometric modelling faces huge problems of multi-collinearity between the relevant independent variables and simple time-series modelling per country is not relevant to the underlying question.

Panel-data modelling creates various problems regarding the quantity or quality of data (e.g. with missing observations, an unbalanced panel, etc.) Panel data also allow the possibility of taking explicit account of country-specific effects – fixed or random. These issues and others raise important questions for any estimation strategy on the appropriate specification of the error term. In consequence, the eventually obtained coefficient estimates, standard errors and test statistics depend on a number of critical econometric assumptions, the choice of which is not always obvious<sup>37</sup> .

Another major issue is the fact that countries are more likely to adopt and encourage effective regulatory agencies the more that they have a positive attitude to the separation of powers, the commercialization of economic activities and to private investment and market economics in general. Hence, one should not assume that independent regulators are exogenous. In all areas of the world, countries will select whether they want to go down the route of commercialisation and independent regulation. Early adopters in particular are likely to be heavily self-selecting.

This endogeneity problem is well-known in the ICB effectiveness literature<sup>38</sup> . It has surfaced in the regulatory effectiveness literature. Two recent examples are in Gual and Trillas (2002) and Guasch, Laffont and Straub op cit<sup>39</sup> .

Gual and Trillas discuss the impact of 37 telecom regulators and they instrument the degree of independence of the regulator which is measured by an index, following the ICB effectiveness literature. The degree of regulatory independence is estimated

> 37 See Fink, Matthoo & Rathindran op cit, Appendix 4, pp 32-3 for a good discussion of some of these issues in the context of telecom regulation.

> 38 See, for instance, the survey of ICB empirical studies in Berger, de Haan and Eijffinger (2000).  We will discuss this and related issues in Section 2.5 below.

> 39 See also Gutierrez (2003) which has a detailed investigation of endogeneity issues in telecom regulation.  He finds little or no change in  his estimates from allowing for endogeneity.

29