metadata: authors: 0: author: Hallward-Driemeier, Mary
1: author: Aterido, Reyes
count: World
docna: 0: docna: Comparing Apples with....Apples : how to
            make (more) sense of subjective rankings of constraints to business
docty: Policy Research Working Paper
theme: Analysis of economic growth
majtheme: Economic management
subtopic: Investment and Investment Climate,Judicial System Reform,Energy Policies & Economics,Crime and Society,Access to Finance
teratopic: Transport,Private Sector Development,Finance and Financial Sector Development
abstracts: cdata!: The use of expert or qualitative surveys
            to rank countries business investment conditions is
            widespread. However, within the economic literature there
            are concerns about measurement error and endogeneity based
            on characteristics of the respondents, raising questions
            about how well the data reflect the underlying reality they
            are trying to measure. This paper examines these concerns
            using data from 79,000 firms in 105 countries. The findings
            show that first, qualitative rankings correlate well with
            quantitative measures of the business environment, using
            both quantitative measures from within the survey and from
            external sources.  Second, there are systematic variations
            in perceptions based on firm characteristics - focusing in
            particular on size and growth performance. However, it is
            not that an optimistic view of the business environment is
            simply the expression of a firms own performance. Rather,
            firm size and performance affect the relative importance of
            certain constraints, particularly in areas such as finance,
            time with officials/inspectors, corruption, and access to
            reliable electricity. The results also show that much of the
            variation in subjective responses by firm types is largely
            due to differences in the objective conditions across firm
            types. There is little evidence that size and performance
            have non-linear effects in how constraining a given
            objective condition is reported to be. Overall, concerns
            about endogeneity remain in using business environment
            indicators to explain firm performance, but this stems
            primarily from the fact that who you are and how well you
            are doing can affect the conditions you face rather than
            whether the indicator used is qualitative or quantitative.
repnme: repnme: Comparing Apples with....Apples : how to
            make (more) sense of subjective rankings of constraints to business
display_title: Comparing Apples with....Apples :
            how to make (more) sense of subjective rankings of
            constraints to business
listing_relative_url: /research/2009/09/11075855/comparing-apples-withapples-make-more-sense-subjective-rankings-constraints-business
topicv3: Justice m1326106 840,Investment Climate m1326293 405,Power and Electricity Sector m1327094 515,Access to Finance m1327464 1249,Crime m1439217 3041
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Transport,Private Sector Development,Finance and Financial Sector Development
no_of_pages: 34
model_extractions: WPS5054 Policy Research Working Paper 5054

# Comparing Apples with...Apples

How to Make (More) Sense of Subjective Rankings of Constraints to Business _Mary Hallward-Driemeier Reyes Aterido_ The World Bank Development Research Group Macroeconomics and Growth Team September 2009
Policy Research Working Paper 5054

# **Abstract**

The use of expert or qualitative surveys to rank countries’ business investment conditions is widespread. However, within the economic literature there are concerns about measurement error and endogeneity based on characteristics of the respondents, raising questions about how well the data reflect the underlying reality they are trying to measure. This paper examines these concerns using data from 79,000 firms in 105 countries. The findings show that first, qualitative rankings correlate well with quantitative measures of the business environment, using both quantitative measures from within the survey and from external sources. Second, there are systematic variations in perceptions based on firm characteristics— focusing in particular on size and growth performance. However, it is not that an optimistic view of the business environment is simply the expression of a firm’s own performance. Rather, firm size and performance affect the _relative_ importance of certain constraints, particularly in areas such as finance, time with officials/inspectors, corruption, and access to reliable electricity. The results also show that much of the variation in subjective responses by firm types is largely due to differences in the objective conditions across firm types. There is little evidence that size and performance have non-linear effects in how constraining a given objective condition is reported to be. Overall, concerns about endogeneity remain in using business environment indicators to explain firm performance, but this stems primarily from the fact that who you are and how well you are doing can affect the conditions you face rather than whether the indicator used is qualitative or quantitative.

This paper—a product of the Growth and the Macroeconomics Team, Development Research Group—is part of a larger effort in the department to explore firm level data and the microeconomics of growth. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at mhallward@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Comparing Apples with....Apples: How to Make (More) Sense of Subjective Rankings of Constraints to Business**

Mary Hallward-Driemeier and Reyes Aterido The World Bank<sup>*</sup>

> * We wish to thank the participants of the Oxford Business and Economics Conference and anonymous referees for useful comments.   The corresponding author is Mary Hallward-Driemeier, World Bank MSN MC3-301, 1818 H St. NW, Washington, DC 20433, mhallward@worldbank.org.  The opinions are solely those of the authors and do not necessarily reflect the official views of the World Bank or its Executive Directors.
# **1. Introduction**

_In Brazil, 80 percent of managers have ranked access to finance as a major or severe constraint. But in Kazakhstan, 20 percent of managers did. Is access to finance a bigger problem in Brazil than in Kazakhstan?_ _Not necessarily. In Brazil, all but one of the 17 possible constraints are identified as major or severe by a higher proportion of respondents than identify the top constraint as major or severe in Kazakhstan. Switching from an absolute ranking to a relative one, finance is reported as the top constraint in both countries. Looking at more objective information on access to finance, only 13% of small firms in Kazakhstan are able to access formal external finance, while half of the small firms in Brazil are able to. Indeed, productive firms in Brazil complain less about access to finance than less-productive firms. However, in Kazakhstan, productive firms are more likely to complain about a lack of access to finance than less-productive firms, raising bigger questions about the available supply of external finance and how it is allocated among firms there._ Qualitative rankings are extremely popular among policy makers as well as members of the private sector. The World Economic Forum’s Global Competitiveness Report uses subjective ratings to rank countries, as does the International Country Risk Guide (ICRG) and the Heritage’s Index of Economic Freedom. However, what is ultimately of interest is how well these rankings reflect the underlying reality and its impact on business operations, and how much weight one should put on them in formulating priorities for change. Here there is much more skepticism among economists about their reliability.

Concerns about using qualitative or subjective indicators in analysis stem from possible measurement error that could introduce bias into the results.<sup>1</sup> This could reflect differences across individuals in their willingness to report negative (or positive) responses; differences across respondents in what they use as a benchmark or yardstick in

> 1 We use qualitative or subjective interchangeably to reflect the measures are ratings of conditions.

2
answering questions; and whether the respondents’ responses reflect their views on broader phenomena than the specific issue in the question. As subjective rankings implicitly combine an assessment of the issue being ranked with an assessment of how important that issue is to the respondent, endogeneity cannot be altogether eliminated. However, the concern that these responses simply reflect the individual’s own performance across the board (e.g. that poorly performing firms could complain more about everything) is a separate form of measurement error that can be addressed.

The World Bank’s Enterprise Surveys database (ES) provides an opportunity to address all of these concerns. It provides comparable data on 79,000 firms and 105 countries. There are four features of the data that are of relevance for this paper. First, there is a question that asks respondents to rank 17 dimensions of the business environment on how constraining they are to the operation and growth of their business. These provide the subjective perceptions data that are examined here. Second, the survey also has objective measures of the investment climate that correspond to these same issue areas.<sup>2</sup> Third, there is a rich set of firm characteristics and measures of firm performance in the dataset. Fourth, the same questionnaire is implemented in each country, with a standardized sampling methodology, making the data comparable across countries. This rich set of information on firm characteristics, including firm performance itself, allows

> 2  It may clarify the difference between the qualitative and quantitative data to provide an example. Looking at the issue of crime, the qualitative question asks respondents to rank on a scale of 0-4: ‘how constraining is crime to the operation and growth of your firm?’ In contrast, a quantitative question regarding crime would be “what share of sales is lost to theft or vandalism last year?”  To the extent all answers are reported by the respondent, they can be seen as having a subjective element to them.  The distinction we are trying to make, however, is that the quantitative questions are based on more objective criteria and ask for responses in terms of time or monetary costs, not a rating scale, and do not include any assessment as to whether the issue is of concern to the respondent or not.

3
for more direct tests of whether constraints vary systematically across categories of firms and what feedback, if any, there is between firm performance and perceived constraints. The analysis looks at two broad issues. First, it examines how well the subjective rankings correlate well with the realities they are trying to measure; e.g. are firms that experience more outages more likely to report a lack of access to reliable power a constraint? It uses both more objective data within the survey (e.g. the time and monetary costs of complying with regulations, frequency of power outages, losses from crime etc.) as well as outside data sources. Second, it examines whether subjective rankings vary systematically by respondent characteristics, whether what you say you care about depends on who you are. Attention is focused on two firm characteristics -- firm size and firm performance. We measure firm performance based on employment growth, using dummies for whether firms are expanding or contracting.<sup>3</sup> There are three channels through which firm characteristics could affect how constraining a dimension of the business environment is reported to be: i) conditions experienced may vary with firm size or performance (e.g. smaller firms report more limited access to finance or may be easier targets for officials seeking bribes). Indeed, there are concerns that small firms face additional burdens, particularly in developing countries, that may hold them back (Tybout 2000, Hallward-Driemeier and Aterido (2008), Aterido et al. (2009)); ii) relative priorities across dimensions of the investment climate may vary with firm characteristics (e.g. labor skills may be more important for expanding firms while labor regulations may matter more for contracting firms); and, iii)

> 3 Employment growth is correlated with sales growth, but the former has the benefit of not needing to be deflated and is not subject to changes in markups, so growth reflects real changes.  Growth rates are generally calculated over a 3 year period to smooth out fluctuations.

4
firm characteristics could have a non-linear impact of how constraining the same objective condition is reported to be (e.g. large firms may report the same number of outages as more constraining than a small firm).

Overall, the analysis shows that subjective rankings are significantly correlated with objective measures – taken from within the survey or outside sources. Variations in reported constraints thus do correspond to differences in conditions on the ground.

Firm characteristics also matter in determining reported constraints. However, the effects of size and performance are not to raise or lower overall levels of constraints. Rather, expanding and contracting firms both report higher overall levels of constraints than stable firms. Firm characteristics affect the _relative_ importance of different constraints; so which issues matter does depend on who you are and how you are doing. But the evidence indicates this reflects differences in objective conditions experienced by firms, _not_ how they translate those conditions into rankings of constraints. Thus, contracting firms that complain about electricity are doing so because they are experiencing more outages. There is little evidence that the same number of outages is reported as more or less constraining based on performance or size. This implies that endogeneity remains a concern in using firm based indicators to explain firm performance, but the source of endogeneity is from firm characteristics on the business environment they face, rather than whether the measure of the business environment used is subjective or objective.

The paper is organized as follows. Section 2 discusses the concerns raised in the literature about using subjective rankings. Section 3 describes the Enterprise Survey data used in this analysis. Section 4 examines how well subjective conditions reflect 5
differences in objective conditions using both measures from within the ES and from external sources. Section 5 then investigates whether there are differences across firms by size and performance – and whether these stem from differences in the objective conditions they face, or in how they translate those conditions into reported constraints. Section 6 concludes.

# **2.  Literature**

Subjective responses are used to look at a wide range of issues. The focus here is on issues related to the business environment or investment climate in which firms operate. This includes regulations, infrastructure and financial services, corruption and security – all dimensions of the broader environment in which firms operate and that affects their incentives and opportunities to invest, create jobs and grow. The importance of these dimensions in explaining long run growth has been rekindled with work by Engerman and Sokoloff (2002) and Acemoglu, Johnson, Robinson (2001). However, there has also been a growing literature that seeks to move the analysis to a more disaggregated level. This is both because the measures of institutions need to be more refined, but also due to the possibility that different firms within a country can face different conditions (World Bank 2004). The availability of new datasets has shown promising results on the importance of property rights, regulatory burden, corruption as well as access to finance and infrastructure services in explaining firm performance. While some papers have focused on the available objective measures (e.g. Dollar et al. 2005; Reinikka and Svensson 2006; Aterido et al. 2009), many papers have used the subjective ratings to rank countries or to give relative priorities to issues, e.g. Batra et al.

6
(2003) or Haltiwanger and Schweiger (2005); or to examine a particular dimension of the investment climate, e.g. corruption – Hellman et al. (2000); property rights -- Brunetti (1998); Johnson et al. (2002); finance – Beck et al. 2005.

One of the appealing features of firm rankings of constraints is that they implicitly combine an assessment of the severity of the conditions with the importance of that area to the firm’s operations. If long delays or costs are in a critical area, this should be reflected in the reported constraints. Thus perceived constraints incorporate a measure of the impact of an issue which should be important in identifying reform priorities. In this sense they are closer to Carlin et al. (2006)’s notion of subjective measures as Lagrangian multipliers. And the perceived impact of issues will influence firms’ decision whether to undertake investments, hire workers or expand production. Managers make these decisions with an eye to what they expect in the future, including their assessment of the investment climate. Perceptions can thus have real effects.

_Concerns about using subjective responses_ While the comparison of constraints thus has many appealing features, there are also several potential difficulties in making such comparisons across individuals, let alone countries (e.g. Sudman et al. 1996, Tanur (1992)). Simply comparing absolute scores can be misleading as the italicized paragraph at the beginning of the paper illustrates. Potential shortcomings in comparing subjective responses include:<sup>4</sup> _1. Specificity of responses: whether respondents answer the specific question or provide a broader evaluation of conditions._ To attribute the respondents to the

> 4 Additional limitations to relying solely on survey responses: they only include incumbent firms; reflect respondents’ views, not broader social interests; and they ignore the costs (absolute and relative) of addressing constraints.

7
particular issue being investigated, it is important to know that responses are not a general assessment or a proxy for overall concerns.<sup>5</sup> 2. _‘Kvetch’ factor: differences in the willingness to report that potential obstacles are constraining._ This effect can be at work across individuals, but it is also striking across countries where some cultures are more or less willing to report that potential obstacles are constraining. Bertrand and Mullainathan (2001) caution against the use of subjective variables, both as dependent and independent variables, due to likely measurement error based on such differences in willingness to report on an issue. Fortunately, this can be addressed, particularly if multiple issues are ranked simultaneously. If the ‘optimism/kvetch’ factor is likely to shift all of an individual’s responses up or down, one can capture that in an individual fixed effect. Subtracting off the individual’s mean level of reported constraints across all issue areas leaves the relative rankings between obstacles unaffected. Corruption is likely to be the area where willingness to be truthful may be particularly delicate, but Hellman et al. (2000) and Svensson (2003) find little evidence in bias in responses to questions of corruption that are also used in the ES given the phrasing of the questionnaire and the overall list of constraints.

3. _Reference point bias: respondents may use different yardsticks against which conditions are measured._ People may agree that a particular issue is a problem, but the same experience or condition (e.g. 3 power outages a month) may

> 5 The ordering of constraints may also matter: earlier elements in the list may get higher or lower rankings, more care may be taken to differentiate between them or the comparison may be made with the obstacle right before the current one. This has been tested –in a particular country the sample was divided in two groups with different ordering of questions on each survey but responses were not found to be different.

8
be rated as ‘severe’ by one person while it may be ‘moderate’ to another. (Schwarz (1985, 1991), Fowler (1995)).

4. _Performance bias—whether ratings actually reflect the environment in which the firm operates rather than the firm’s performance in the environment._ To interpret the results one would want to know that a constraint reflects difficulties in the external business environment and are not just a reflection of whether the respondents’ firms are performing well or not. The concern is that internal difficulties not be blamed inappropriately on external conditions (Senik (2005)). However, it should be noted that this potential source of bias could go either way. Poorly performing firms may blame external conditions for their difficulties. But, it may also be that it is precisely those firms that are doing well and trying to expand that may complain more, finding that weak investment climate conditions really are constraining them. So it is a matter of interest to know not just if there are performance biases, but also in which direction they might be working.

In addition to concerns about possible measurement error, some have questioned the practical relevance of subjective constraints in understanding firm performance.

Commander and Svejnar (2007) find little significant correlation between the subjective rankings and firms’ revenue efficiency using a sub-sample of the same data, i.e. 26 countries in Eastern Europe and Central Asia. However, they do not allow for non-linear effects. One of the results from this paper is that the effects of many issues are not monotonic; e.g. access to finance and corruption are reported as more constraining by both 9
expanding and contracting firms relative to stable firms. Tests for linear effects would miss these relationships.

# **3. Addressing these concerns with the Enterprise Survey data**

This paper uses the Enterprise Survey (ES) database to test for how significant these concerns are in practice across 79,000 surveys in 105 countries. In face-to-face interviews, the survey collects detailed information on many aspects of the investment climate in which a firm operates as well as information about the firm’s own characteristics and performance. The concept of investment climate compiles issues related to costs and risks of investing in and operating a business. The same questionnaire covering all these aspects faced by managers is implemented in each country, with a standardized sampling methodology, making the data comparable across countries (see data appendix for details).

Questions include both qualitative subjective questions on various potential constraints to business performance and quantitative measures. Managers are asked to rank how constraining a set of issues are for the operation and growth of their business. The list is found on Table 1, column 1, with higher average values indicating more firms report the issue as a significant constraint.<sup>6</sup> More than three-fifths of firms report tax rates as being above their average level of constraint. This is closely followed by policy uncertainty, access to finance, corruption, tax administration and informality. Issues that earned low rankings were telecommunications and transportation.

> 6 The constraints are grouped thematically, and ordered from most to least constraining.  The same ordering is then kept for all the tables.

10
In addition, the ES has more quantitative measures of the investment climate. These are listed in Table 1, colunm2, with variables chosen to match the corresponding qualitative measure. Column 3 provides summary statistics for measures drawn from additional external sources of data.

Having a large set of questions being asked together provides a means of addressing the ‘specificity’ concern as well as the “optimism or kvetch factor.” In the first instance, it requires the respondent to differentiate across a number of issues, making it much less likely that any one issue is a proxy for the broader business environment. Looking at the range of responses across respondents, less than one percent had no variation in their answers. Thus the first potential concern does not seem to be an issue here.

The second benefit to the ES set of questions is that individual fixed effects can be used to control for differences across respondents in their optimism/pessimism in answering questions. Demeaning responses at the firm level, subtracting each firm’s average of all the scores on its list, not only controls for the manager’s optimism/kvetch factor, it also removes the effect of any characteristics or location-specific circumstance that affects all the perceptions. This demeaning does not allow for the comparisons of the absolute level of constraints, but does look at the relative importance of a particular constraint compared to the others.<sup>7</sup>

> 7 There is also a question as to what is the appropriate level of responses to consider.  Commander and Svejnar (2007) argue that the absolute rankings of the business environment help explain firms’ revenue efficiency at the national level, but that the measures lose significance with the inclusion of country, sector, year dummies.  However, they do not allow for non-linear effects, which as illustrated here are very significant.  Carlin, Schaffer and Seabright (2006) do find meaningful within country variation.  However, we disagree with their interpretation of the source of variation.  They argue that the list of potential constraints (except access to finance) should be seen as public goods that do not vary within a country. They thus interpret between country differences as measures of differences in supply of the public goods, and

11
As has been noted, the richness of the Enterprise Survey data provides for direct ways to test for the links between quantitative and qualitative indicators, and to link them to firm characteristics and firm performance. This is done in the next section.

# **4.    How well do subjective responses reflect objective conditions?**

A first step in determining the validity of subjective responses is to see how well they correspond to more quantitative measures of conditions facing firms. One would expect that within an area, those facing worse objective conditions would complain more about that area. We run the following regressions for each of the relative measures of subjective investment climate constraints: Where Xi is a set of k dummy variables identifying firms’ characteristics: size, firm age, location , ownership (domestic, foreign and government), exporter (yes-no);  s and  c are sector and country dummies We find strong empirical evidence that perceptions reflect the real investment climate. Within the thirteen issues areas for which the survey has related quantitative follow-on questions,<sup>8</sup> we tested if the firms that report longer delays, greater costs or lower quality service are the same ones that report that issue as being more constraining. In each case, the answer is yes. The coefficients _ß1_ are extremely significant and with the appropriate sign.

> within country variations as indications of different firms’ demand for these goods.  They predict that more productive firms will likely complain more when comparing within a country, but that they will complain less when comparing across countries.   In this paper, country and sector dummies are included.   As the evidence presented shows, there is actually considerable variation within country in the quantitative measures that can be exploited and is meaningfully related to the subjective rankings reported.

> 8 The survey did include four other issue areas.  However, for two of the areas (macroeconomic instability and access to land) there are no additional objective indicators included in the survey.  For two others (courts and operative licenses) there are too few firms that either used the court system or applied for a new operative license in the year prior to the survey.

12
Correlations between the qualitative and quantitative indicators could simply reflect differences in the firms’ use or reliance on a condition, e.g. firms producing ice cream will need electricity, and thus will be more likely to report outages and report unreliable electricity as more constraining than a tailor shop that uses electricity intermittently. Thus, these regressions look at variations within sectors, controlling for firm size, export status, ownership, age and location ( _Xkit_ ). The aim is test among similar firms within the same sector, whether and how size and performance affect the qualitative rankings. Thus the results are not based on differences between ice cream makers and tailors, but between large and small (or growing and contracting) ice cream makers.

Table 2 reports the effect of a one standard deviation change in the objective measure on the probability that the issue is ranked as an above-average constraint for that firm. Each row represents a separate regression, with the full set of controls indicated at the bottom of the table. As the units for the various objective measures vary, the use of the one standard deviation facilitates comparisons.

The effects can be large; reducing the size of bribes by 1 standard deviation, lowers the probability a firm sees the constraint as above its average level by 23 percentage points. Improving the number of days without power would reduce electricity as an above average constraint by 19 percentage points. Lowering the time to get goods through customs is associated with a 13 percentage point reduction in customs being seen as an above average constraint.

Table 3 uses measures of the investment climate drawn from non-ES sources such as the World Development Indicators, Kaufmann-Kraay Governance Indicators and Doing Business. As the external indicators are only available at the country level, the firm 13
responses are aggregated to give the share of firms in the country that report the issue as above their own firm-average level of constraint. The correlations with the ES indicators are relatively large and significant for all but one indicator.<sup>9</sup> This reinforces the view that the qualitative measures of constraints are capturing meaningful variations across countries and within countries.

# **5.  Are there differences across types of firms?**

In the literature, measures of constraints are often reported for a country as a whole, even if they are based on underlying disaggregated information. However, Table 4 shows there are significant sub-national differences and variations across types of firms in the constraints they report. The two dimensions highlighted here in particular are firm size and lagged firm performance.<sup>10</sup> _IC_qualitativeit= ß1 + ß2 *Size(t-1)i + ß3*Performance(t-1)I + ∑ ßk*Xkit +_  s _+_  c _+ eit_ (3) Where Xi is a set of k dummy variables identifying firms’ characteristics: size, firm age, location, ownership (domestic, foreign and government),  s and  c are sector and country dummies 9 It should be noted that there are somewhat different patterns based on the external data source used for comparisons. The first source is the Kaufmann-Kraay Governance Indicators. They are composite measures based on the aggregation of existing data sources (can be 12-20 variables used to construct each index). These draw on a number of subjective or expert opinion polls as well as some more objective measures. The second data source is Doing Business. This provides _de jure_ measures of the time and costs of fully complying with all regulatory requirements. The correlations with these variables are still generally significant, but the sizes of the coefficients are considerably smaller. This can be understood in part from the differences in what Doing Business and the surveys measure. While Doing Business measures the formal requirements, the ES report what firms actually experience. But, what is supposed to happen is not always what actually does happen. Enforcement and implementation are ultimately what matters to the firms. To the extent that there is a gap between what is on the books and what happens on the ground, the correlations between the two will be weak. Other work shows that this gap can be considerable and is generally larger in countries with weaker rule of law or greater corruption (Kaufmann, Kraay 2007; Hallward-Driemeier 2006).

10 The regressions are run as probits on the probability that a firm reports the issue as being above their average level of complaint, with marginal effects reported.

14
The first column in Table 4 shows that the average level of complaint is not monotonically related to firm size; it is lowest for small firms. Exporters also have a significantly higher level of complaint, although controlling for export status, foreign firms do not. State owned firms are less likely to report constraints.<sup>11</sup> Subjective rankings do vary by firm size. Looking across issue areas, micro firms report being relatively more constrained by issues of finance, crime and informal competition. Larger firms report being relatively more constrained by most areas of regulations. Electricity is one of the few categories where there is no significant difference across sizes of firms.

Subjective rankings also vary with firm performance. However, it is not the case that performance simply shifts ratings up or down. Thus firms with better (or worse) lagged performance are not uniformly more or less optimistic. Both expanding and contracting firms report significantly higher levels of overall constraints compared to stable firms.<sup>12</sup> And looking across the issue areas, the same pattern of expanding and contracting firms reporting higher levels of constraints compared to stable firms holds; the constraints do not rise (fall) monotonically with performance. Non-linearities in the effect of performance on a constraint’s severity cautions against simply including firm performance as a control; the overall effect can be insignificant when there are important relationships in different segments of the data.

What is also striking is that the effect of performance has differential effects across issues. Performance thus affects _which issues_ are reported as most constraining. For

> 11 The wider set of firm controls were included in the regressions but not reported due to space constraints. The full set of results is available upon request.

> 12 Note, stable firms report more issues to be ‘not a constraint’, so that they have a slightly higher proportion of  constraints that are ‘below average’.

15
example, expanding firms report labor skills, crime and transportation as relatively more of an obstacle, while contracting firms see labor regulations, informality and policy uncertainty as relatively more constraining.

There are different underlying reasons why firms may vary in their assessment of the business environment. A reported constraint itself is a combination of an assessment of the conditions and an assessment of the importance of this issue to the firm. A third possibility is that there is an interaction between the two elements, that there are nonlinearities in how constraining the same objective condition is reported to be. The next sections thus try to sort out if micro firms complain more about access to finance whether this is because micro firms receive less external finance, whether access to finance is relatively more important for micro firms, or whether the same amount of external finance is reported as relatively more constraining for micro firms.

_i) Variations in objective conditions by size and performance_ The first question to examine is whether some groups of firms actually face more challenging conditions. Smaller firms may actually face greater obstacles in securing a loan or in accessing regular power supply. Table 5 reports the results from the following specification: _IC_quantitativeit= ß1 + ß2 *Size(t-1)i + ß3*Performance(t-1)I + ∑ ßk*Xkit +_  s _+_  c _+ eit_ (4) Where Xi is a set of k dummy variables identifying firms’ characteristics: size, firm age, location , ownership (domestic, foreign and government),  s and  c are sector and country dummies The results show that objective conditions do vary significantly by size. This is not just true in comparing large and small firms, but also between small and micro firms. For example, within regulations, an area that larger firms complained relatively more 16
about, they do have to spend greater time dealing with officials and inspectors. Looking at days of inspections, smaller firms spend considerably less time in inspections than the larger firms. However for finance, larger firms have at least double the share of investments financed by formal bank loans compared to the smaller firms. This is an area where micro firms complain more. On issues of corruption, larger firms are less likely to see bribes as being necessary to ‘get things done’, and the relative size of the bribe for larger firms is smaller. This would be consistent with bribes being a fixed amount, which in percentage terms represents a higher share of revenues of smaller firms. In terms of infrastructure, the middle size categories report more delays and interruptions. These differences in objective conditions thus match up fairly well with the size patterns in terms of what issues are reported as relatively constraining.

Objective conditions also differ by firm performance. Contracting firms are more likely to have experienced losses from crime and spend more time with officials and in inspections. In looking at performance, dynamic firms seem to have more interactions with officials, with declining firms having even more than expanding firms. For finance, both expanding and contracting firms are more likely to report actually having access to external financing. The former is the story of firms wishing to pursue an opportunity but not finding support from banks. The latter finding is consistent with firms in distress wishing to turn to banks to help them through the adjustment. Yet the objective data shows that these groups of firms are both more likely to actually have access to bank finance than stable firms.<sup>13</sup> Both groups are more likely to report paying bribes than stable firms too, with the effects larger for declining firms. Both groups are also more

> 13 If one looks at additional sources of finance, expanding firms are more likely to have longer-term finance and contracting firms shorter-term working capital.

17
likely to report unreliable infrastructure services. These findings all correlate well with the importance given to these issues in the relative rankings.

This evidence points to the need to look at variations in measures within countries and not to assume that there are national standards that all firms face in common.

_ii) Variations by size and performance on the impact of the same objective conditions on subjective rankings reported_ That there are differences in objective conditions by firm size and firm performance still leaves open whether the differences in subjective rankings reflect these conditions or whether they reflect how these conditions are translated into subjective rankings. If firms lack access to bank loans, there could be differences in access to alternative sources of funds by size or productivity that then affect how this translates into its impact on firms. Thus, even if the objective conditions do not vary across firms, the impact of the same conditions might. The results from Table 5 underscore that endogeneity concerns should be taken into account in trying to use subjective rankings to explain performance; the possibility of non-linearities in how the same condition is reported would further complicate the interpretation of rankings, with the same conditions leading to different rankings depending on who the respondent is and how they are doing.

Table 6 tests whether non-linearities in how objective conditions are reported is an issue, reporting the results from interacting the objective conditions with firm characteristics in explaining the patterns of reported subjective constraints.

_IC_qualitativeit= ß1*IC_quantitativeit + ß2* Size(t-1)i + ß3*Size(t-1)i *IC_quantitative + + ß4* Performance(t-1)I + ß5*Performance(t-1)i *IC_quantitativeit + ∑ ßk*Xkit +ei_ (5)

> Where Xi is a set of k dummy variables identifying firms’ characteristics:  size, firm age, location , ownership (domestic, foreign and government), exporter,  s and  c are sector and country dummies

18
The first thing to note is that the results for the objective conditions are still highly significant in all the specifications. Looking within sectors and controlling for other firm characteristics, firms that report longer delays, greater costs or lower quality service report that issue as being more constraining.

Second, as in table 4, the relative level of constraints also varies with firm size and performance, with the patterns largely unchanged with the inclusion of the objective conditions. This reinforces the interpretation that the relative importance of different dimensions of the business environment does vary by firm characteristic and that they reflect differences in priorities across firms.

Lastly, there is little evidence of non-linearities by either size or performance in how constraining objective conditions are reported as being. The exceptions, by size, are tax rates and tax administration. Thus, larger firms report the same quantitative gifts to tax inspectors as less constraining, reflecting that smaller firms are less in compliance and/or less able to navigate the system.

For performance, the interactions are only significant within some of the ‘rule of law’ variables, i.e. the amount – but not the incidence – of bribes. Both expanding and contracting firms report the same size bribe as relatively more constraining than for stable firms paying the same bribe. Again, this could reflect the ability to extract benefits from officials for the payments made. Contracting firms also experience the same degree of informal competition as more constraining. However, firm performance has very little effect on how constraining a condition is perceived to be – including in access to finance, the area where endogeneity concerns are most commonly raised (Carlin et al. 2006).

19
# **6.  Conclusion**

Overall, subjective rankings should reflect both an assessment of actual conditions on the ground and how important they are to the firm. Skeptics of subjective rankings are concerned that correlations with quantitative measures may not be very significant. The evidence presented here demonstrates qualitative measures do reflect differences in conditions experienced by firms. Rankings are significantly correlated with quantitative measures – taken from within the survey or outside sources. Skeptics also worry that performance may have level effects across the board, a concern that we address by looking at relative rankings. The results show that neither size or nor performance leads to systematically higher or lower complaints. Rather, size and performance affect the relative importance of different constraints, in ways that meaningfully reflect differences in which dimensions of the business environment matter for their different needs. Finally, with some exception for rule of law variables, who you are has little independent effect on how constraining the same objective conditions are reported to be. This should also help assuage concerns of possible measurement error due to endogeneity in how firm characteristics affect the reporting of conditions. Overall, concerns about endogeneity remain in using business environment indicators to explain firm performance, but this stems primarily from the fact that who you are and how well you are doing can affect the conditions you face rather than whether the indicator used is qualitative or quantitative.

20