metadata: authors: 0: author: Brunori,Paolo
1: author: Hufe,Paul
2: author: Mahler,Daniel Gerszon
count: World
docna: 0: docna: The roots of inequality : estimating
            inequality of opportunity from regression trees
docty: Policy Research Working Paper
abstracts: cdata!: This paper proposes a set of new methods
            to estimate inequality of opportunity based on conditional
            inference regression trees. It illustrates how these methods
            represent a substantial improvement over existing empirical
            approaches to measure inequality of opportunity. First, the
            new methods minimize the risk of arbitrary and ad hoc model
            selection. Second, they provide a standardized way to trade
            off upward and downward biases in inequality of opportunity
            estimations. Finally, regression trees can be graphically
            represented; their structure is immediate to read and easy
            to understand. This will make the measurement of inequality
            of opportunity more easily comprehensible to a large
            audience. These advantages are illustrated by an empirical
            application based on the 2011 wave of the European Union
            Statistics on Income and Living Conditions.
repnme: repnme: The roots of inequality : estimating
            inequality of opportunity from regression trees
display_title: The roots of inequality : estimating
            inequality of opportunity from regression trees
listing_relative_url: /research/2018/02/29610954/roots-inequality-estimating-inequality-opportunity-regression-trees
topicv3: Poverty m1326687 695,Trade m1326924 698,Education m1327048 470,Income Inequality m1503891 2668,Jobs m1503931 2670
publishtoextweb_dt: 2018-02-20T00:00:00Z
no_of_pages: 35
author_keys: 0: author_key: 1539953
model_extractions: WPS8349 Policy Research Working Paper 8349 The Roots of Inequality Estimating Inequality of Opportunity from Regression Trees _Paolo Brunori Paul Hufe Daniel Gerszon Mahler_ Development Research Group Poverty and Inequality Team February 2018
Policy Research Working Paper 8349

# **Abstract**

This paper proposes a set of new methods to estimate inequality of opportunity based on conditional inference regression trees. It illustrates how these methods represent a substantial improvement over existing empirical approaches to measure inequality of opportunity. First, the new methods minimize the risk of arbitrary and ad hoc model selection. Second, they provide a standardized way to trade off upward and downward biases in inequality of opportunity estimations. Finally, regression trees can be graphically represented; their structure is immediate to read and easy to understand. This will make the measurement of inequality of opportunity more easily comprehensible to a large audience. These advantages are illustrated by an empirical application based on the 2011 wave of the European Union Statistics on Income and Living Conditions.

This paper is a product of the Poverty and Inequality Team, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at dmahler@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
The Roots of Inequality: Estimating Inequality of Opportunity from Regression Trees<sup>*</sup> Paolo Brunori<sup></sup> , Paul Hufe<sup></sup> , Daniel Gerszon Mahler<sup>§</sup>

**_JEL-Codes:_** D31; D63; C38

**_Keywords:_** Equality of Opportunity; Machine Learning; Random Forests

> * We are grateful for comments received from participants during presentations held at the Institute for Social and Economic Research at the University of Essex, the Poverty and Applied Microeconomics Seminar at the World Bank, the Copenhagen Centre for Social Data Science at the University of Copenhagen, and the 13th Winter School on Social Choice Theory and Welfare at Canazei. Any errors remain our own.

>  Corresponding Author: University of Florence, Dipartimento di Scienze per l’Economia e l’Impresa, Via delle Pandette 32 - 50127 Firenze, Italy, `paolo.brunori@unifi.it` .

>  ifo Munich and LMU Munich, `hufe@ifo.de` .

> § University of Copenhagen and World Bank, `dmahler@worldbank.org` .
# **1 Introduction**

John Roemer’s (1998) seminal contribution, _Equality of Opportunity_ , has incited a flourishing empirical literature on the measurement of unequal opportunities. At the heart of Roemer’s formulation is the idea that factors that determine individual outcomes can be divided into two categories: factors over which individuals have control, which he calls _effort_ , and factors for which individuals cannot be held responsible, which he calls _circumstances_ . Individuals characterized by exactly the same exogenous circumstances are assigned to a circumstance _type_ . Members of a type have the same background conditions to transform resources into outcomes. Therefore, while within-type inequality, as caused by the differential exertion of effort, is morally irrelevant, between-type differences in achievements are inequitable and call for compensation. Thus, opportunity-equalizing policies have the objective of neutralizing the impact of circumstances on the distribution of the desirable outcome.

Following Roemer’s approach, a battery of methods to measure inequality of opportunity have been proposed (see Roemer and Trannoy, 2015; Van de gaer and Ramos, 2016, for recent overviews).<sup>1</sup> Today, well established empirical methods include summary indexes that quantify the extent of unequal opportunities (Alm ̊as et al., 2011; Bourguignon et al., 2007; Checchi and Peragine, 2010) as well as statistical tests that detect the mere existence thereof (Kanbur and Snell, 2017; Lefranc et al., 2009). In either case, empirical results are sensitive to critical choices of model selection which are under complete discretion of the researcher.

First, researchers have to make a decision on which circumstance variables to consider for estimation.<sup>2</sup> Observable circumstances beyond individual control are typically a subset of the real number of exogenous variables affecting individual outcomes. This issue has been largely discussed by the literature, and the prevailing view is that partial observability implies downwardbiased inequality of opportunity estimates (Ferreira and Gignoux, 2011). To counteract this downward bias, one strategy is to resort to high-quality datasets that provide very detailed information with respect to individual circumstances (Hufe et al., 2017). Naturally, the scope of improvement of this approach is limited by sample sizes. Consider for example the increasing availability of genetic datasets with billions of polymorphisms per person (Altshuler et al., 2015). While the genetic make-up of individuals clearly is beyond individual control and must be considered a circumstance, the number of circumstances exceeds the available degrees of freedom which forces the researcher to choose selectively from the available set of circumstances.

Second, the influence of circumstances may be dependent on the expression of other circumstance characteristics. For example, it is a well-established finding that the influence of similar child-care arrangements on various life outcomes varies strongly by biological sex (Garc ́ıa et al., 2017). In contrast to such evidence, however, many empirical applications presume that the effect of circumstances on individual outcome is fixed and additive (Bourguignon et al., 2007; Ferreira and Gignoux, 2011). On the one hand, analogous to partial observability, this functional form assumption forces a downward bias on inequality of opportunity estimates. On the other hand, limitations in the available degrees of freedom may prove the estimation of fully saturated models impractical. Again the researcher is left to her own devices in selecting the best model for estimating inequality of opportunity.

While the downward bias of inequality of opportunity estimates is prominently discussed in the extant literature, the reliability of estimates has been largely disregarded. Holding the

> 1Note that a number of contributions from the social choice literature on fair allocation had previously proposed similar methods (Fleurbaey, 1995, 2008; Van de gaer, 1993).

> 2Roemer does not provide a fixed list of variables that are to be considered as circumstances. Rather he suggests that the set of circumstances should evolve from a political process (Roemer and Trannoy, 2015). In empirical implementations typical circumstances are biological sex, socioeconomic background, race, or the area of birth.

2
sample size constant, increasing the type partition by including additional circumstances or relaxing the linearity assumption directly translates into reduced variation for estimating the relevant parameters. In fact, Brunori et al. (2016) show that overfitting the model may instill an upward bias on inequality of opportunity estimates.

This discussion highlights the non-trivial challenge in selecting the appropriate model for estimating inequality of opportunity. Scholars must balance between different sources of bias while trying to avoid ad-hoc solutions.

In this paper we propose the use of classification and regression tree methods to address the outlined shortcomings of current approaches. Introduced by Morgan and Sonquist (1963) and popularized by Breiman et al. (1984), classification and regression trees belong to a family of statistical methods that are commonly summarized under the labels of “machine learning” or “statistical learning” (Friedman et al., 2009). Originating from the fields of computer science and statistics, these methods are being increasingly adopted by economists (Athey, 2017; Mullainathan and Spiess, 2017; Varian, 2014). Classification and regression trees methods were developed to make out-of-sample predictions of a dependent variable based on a number of observable predictors. They let algorithms automatically segment the predictor space into nonoverlapping regions to find the best model for predicting the outcome of interest. In the context of estimating equality of opportunity, this means that we let an automated algorithm decide how to partition the population into mutually exclusive types for the purpose of calculating measures of inequality of opportunity in the spirit of Roemer’s theory. To be precise, within the class of classification and regression tree methods we focus on _conditional inference trees_ and _conditional inference forests_ , both of which bear a number of substantial advantages (Hothorn et al., 2006).

First, by drawing on a clear-cut algorithm one minimizes the degree of arbitrariness in model selection. In both trees and forests types are obtained in the attempt to explain outcome variability without assuming anything about which circumstances play a statistically significant role in shaping individual opportunities and how they interact. Thus, the partition of the population into Roemerian types is no longer a judgment call of the researcher but a nonarbitrary outcome of data analysis. Second, the conditional inference algorithm branches trees (and constructs forests) by using a sequence of hypothesis tests that prevent model overfitting. Therefore, by using the conditional inference algorithm we can both derive a test for the null hypothesis of equal opportunity and avoid the potential upward bias of inequality of opportunity estimates. As a consequence of avoiding upwards and downwards biases, and in contrast to the current approaches, our estimates are better suited for comparisons across time and between countries when sample sizes differ.

Aside from those shared merits, trees and forests bear some distinct advantages which the researcher needs to trade off when selecting her preferred approach within the class of conditional inference methods. Trees, on the one hand, have intuitive appeal and their graphical illustrations are instructive tools for longitudinal or cross-sectional comparisons of opportunity structures. Forests, on the other hand, perform better in trading off the different sources of bias outlined above. In fact, we will show that conditional inference forests outperform other prevalent estimation techniques in terms of out-of-sample prediction accuracy.

To be sure, just as the literature on intergenerational mobility (Black and Devereux, 2011), scholars of equality of opportunity are reluctant to give their estimates a causal interpretation. The ambition of the literature is to understand how much variation in outcomes can be attributed to root causes that are commonly perceived as unfair. It is precisely the _prediction_ character of these empirical exercises that makes this branch of the literature a useful field to leverage the advantages of machine learning algorithms.

The remainder of this paper is organized as follows: section 2 gives a brief introduction with respect to current empirical approaches in the literature. Section 3 introduces regression trees 3
and illustrates how to use them in the context of inequality of opportunity estimations. An empirical illustration based on the EU Survey of Income and Living Conditions is contained in section 4, in which we will also highlight the particular advantages of using tree-based estimation methods. Lastly, section 5 concludes.

# **2 Empirical Approaches to Equality of Opportunity**

Consider a population of size _N_ indexed by _i ∈{_ 1 _, ..., N }_ and an associated vector of incomes _Y_ = _{y_ 1 _, ..., yi, ..., yN }_ . Individual _i_ ’s outcome of interest _yi_ is the result of two sets of factors. First, a set of _circumstances_ beyond her control of size _P_ : Ω _i_ = _{Ci_<sup>1</sup><sup>_, ..., C_</sup> _i_<sup>_p, ..., C_</sup> _i_<sup>_P}_.Second,a</sup> set of _efforts_ of size _Q_ : Θ _i_ = _{Ei_<sup>1</sup><sup>_, ..., E_</sup> _i_<sup>_q, ..., E_</sup> _i_<sup>_Q}_.Ingeneral,theoutcomegeneratingfunction</sup> _g_ : Ω _×_ Θ _→_ R+ can therefore be written as Each circumstance _C_<sup>_p_</sup> _∈_ Ωis characterized by a total of _X_<sup>_p_</sup> possible realizations, where each realization is denoted as _x_<sup>_p_</sup> . Based on the realizations _x_<sup>_p_</sup> we can partition the population into a set of non-overlapping _types T_ = _{t_ 1 _, ..., tm, ..., tM }_ . A type is a subgroup of the original population uniform in terms of circumstances, i.e. individuals _i_ and _j_ belong to the same type _tm ∈ T_ if _x_<sup>_p_</sup> _i_<sup>=</sup><sup>_xp_</sup> _j_<sup>_∀Cp∈_Ω.Theybelongtodifferenttypes</sup><sup>_tm∈T_if</sup><sup>_∃Cp∈_Ω:</sup><sup>_xp_</sup> _i_<sup>=</sup><sup>_xp_</sup> _j_<sup>.</sup> The number of types in the population is given by _M_ =<sup>�</sup><sup>_P_</sup> _p_ =1<sup>_Xp_.FollowingRoemer(1998)we</sup> assume that the joint realizations of the effort variables _E_<sup>_q_</sup> _∈_ Θ can be summarized by a scalar _π ∈_ [0 _,_ 1]. Individuals sharing the same expression of effort are called a _tranche_ . Hence, types and tranches define two particular ways of partitioning the population into subgroups, where group membership either indicates uniformity in circumstances (types) or effort (tranches).

In the literature we can distinguish two broad classes of equality of opportunity definitions.<sup>3</sup> First, the _ex-ante_ view focuses on between-type differences in the value of opportunity sets without paying attention to the specific effort realizations of individual type members. According to this perspective, equality of opportunity is satisfied if the aggregate value of opportunity sets is equalized across types. One example in case is the _ex-ante utilitarian_ perspective according to which the value of opportunity sets is indicated by the average outcome within the specific type. Thus, equality of opportunity would be realized if the mean outcome of each type was equal to the population mean. Second, the _ex-post_ view focuses on individual outcomes conditional on effort exertion. According to this perspective, equality of opportunity would be satisfied if individual outcomes were equalized within each tranche, i.e. individuals with equal levels of effort exertion realize the same outcomes. A comprehensive discussion of the ex-ante and expost principles of equality of opportunity can be found in Fleurbaey and Peragine (2013). In the context of this paper we will restrict ourselves to the ex-ante utilitarian approach only.

**Tests and Measures** The extant literature has witnessed the development of empirical _tests_ and _measures_ for ex-ante utilitarian inequality of opportunity. A prominent example for the former category is provided by Lefranc et al. (2009), who show that rejecting the null hypothesis of no first-order stochastic dominance in type-specific outcome distributions is sufficient to reject the existence of equal opportunities in the population from an ex-ante utilitarian perspective. Furthermore, in a recent contribution Kanbur and Snell (2017) develop likelihood ratio tests that can serve to test for ex-ante utilitarian equality of opportunity. A widely adopted example of the latter category, is the measure developed by Van de gaer (1993) and Chec-

> 3Measures different from the ones illustrated here, have been proposed in the literature. The interested reader is referred to Van de gaer and Ramos (2016) for a comprehensive overview.

4
chi and Peragine (2010). They propose to measure inequality in a counterfactual distribution _Y_<sup>_EA_</sup> = _{y_ 1<sup>_EA_</sup> _, ..., yi_<sup>_EA_</sup> _, ..., yN_<sup>_EA}_obtainedbyremovinginequalitywithintypesfromtheoriginal</sup> distribution. To be precise, individual outcomes are re-scaled to match their respective type mean: where _Nm_ is the size and _μm_ the average outcome of type _tm_ . Therefore, any remaining inequality in _Y_<sup>_EA_</sup> reflects inequality between types and inequality of opportunity can now be summarized by applying any standard scalar measure of inequality I( _·_ ), like the Gini index or a member of the generalized entropy class (Cowell, 2016), to the counterfactual distribution _Y_<sup>_EA_</sup> . Any such measure obtains its minimal value in the case of equality of all type means, i.e if _μm_ = _μl_ = _μ ∀tm, tl ∈ T_ .

**Estimation** In practice we do not observe the full set of circumstances Ω. Rather we observe the subset Ω<sup>ˇ</sup> _⊆_ Ωof size _P_<sup>ˇ</sup> . For example, in most datasets we do not have full information on the genetic make-up of individuals neither do we have a gapless documentation of the socioeconomic conditions in which individuals grew up. Analogously, for most _C_<sup>_p_</sup> _∈_ Ωwe<sup>ˇ</sup> only observe the subset _X_<sup>ˇ</sup><sup>_p_</sup> of the true number of realizations _X_<sup>_p_</sup> . For example, in many datasets information on parental education and occupation is coded in categorical variables of varying detail, which may mask more nuanced socioeconomic differences among households.

Depending on the strength of their distributional assumptions, estimations of inequality of opportunity are typically classified as either non-parametric or parametric. A point in case for the former approach is the abovementioned measure put forward by Van de gaer (1993) and Checchi and Peragine (2010). The researcher partitions the sample into mutually exclusive cells based on the realizations of all circumstance variables under consideration. Hence, the researcher makes no assumption on the interaction of circumstance variables in the determination of individual outcomes. This comes at a high cost, however. To avert overfitting, the partition must be constructed such that a sufficient number of observations belongs to each cell. Conditional on the dataset being rich enough in information on circumstances, this in turn forces the researcher to make a discretionary choice on the _relevant_ partition. Consider for instance a continuous circumstance variable like parental income. Employing the non-parametric estimation approach, the researcher must split the parental income distribution into quantiles for constructing the type partition. The potential granularity of this split obviously depends on the sample sizes of the ensuing cells. Additionally, the researcher must balance the informational content of a finer partition of parental income against the opportunity cost of being forced to exclude another circumstance variable from the investigation. To put it in formal terms: the researcher must select a subset Ω<sup>ˆ</sup> _⊆_ Ω<sup>ˇ</sup> _⊆_ Ωfrom the set of observed circumstances. Furthermore, within the confines of limited degrees of freedom the researcher must also decide for each _C_<sup>_p_</sup> _∈_ Ω,<sup>ˆ</sup> how to restrict the number of realizations _X_<sup>ˆ</sup><sup>_p_</sup> _⊆ X_<sup>ˇ</sup><sup>_p_</sup> _⊆ X_<sup>_p_</sup> in order to construct a statistically meaningful type partition.

To address this problem, the literature commonly resorts to parametric estimation approaches. Here, the researcher obtains the counterfactual distribution by estimating a Mincerian regression with circumstances as the sole right-hand side variables (Bourguignon et al., 2007; Ferreira and Gignoux, 2011): 5
The counterfactual distribution, _Y_<sup>_EA_</sup> can then be constructed from the predicted values Although the parametric approach solves some of the shortcomings of the non-parametric approach, it is not a panacea. The standard version of the parametric approach assumes a linear impact of all circumstances and therefore neglects the existence of interdependencies and non-linearities in the impact of circumstances. To pick up the example from the introduction, the researcher cannot allow for a differential impact of the same child-rearing arrangement on male and female children. Of course, to alleviate this shortcoming the researcher may integrate interaction terms and higher order polynomials into equation (3). At the extreme the researcher may even estimate a fully saturated model, in which case parametric and non-parametric estimation coincide. This congruence, however, reiterates the fundamental problem of current approaches towards the estimation of inequality of opportunity. In view of restrictions on the available degrees of freedom, the researcher is forced to make a discretionary choice on the model she estimates, which in itself is a strong determinant of the ensuing results when testing and measuring equality of opportunity. Furthermore, just as the non-parametric approach, the parametric estimation is at risk of overfitting the data when the set of circumstances is large.

In analogy to this paper, Li Donni et al. (2015) have discussed the issue of ad-hoc model selections in the empirical literature on equality of opportunity. To resolve this issue, they propose a data-driven type partition by estimating a latent class model. In this approach, observable circumstances are considered indicators of membership in an unobservable latent type, _tm_ . For each possible number of latent types, _M_ , the model obtains the partition into types by minimizing the within-type correlation of observable circumstances, _C_<sup>_p_</sup> _∈_ Ω.<sup>ˇ</sup> The optimal number of groups _M_<sup>_∗_</sup> is selected by minimizing an appropriate model selection criterion such as Schwarz’s Bayesian Information Criterion (BIC). The latent class approach therefore partly solves the issue of arbitrary model selection. However, it cannot solve the problem of model selection once the potential number of type characteristics exceeds the available degrees of freedom. In these cases the latent class approach replicates the limitations of other prevalent approaches towards estimating inequality of opportunity: the researcher must pre-select the relevant set of circumstances, their subpartition as well as the respective interactions. To the contrary, our approach embodies a method to select circumstances from the set of all observed variables in a non-arbitrary fashion. Furthermore, latent types are constructed in the attempt to explain circumstances’ correlation. The partition is therefore insensitive to the degree of association between circumstances and outcome. However, one may consider explaining outcome variability as function of circumstances to be precisely the purpose of inequality of opportunity measurement.<sup>4</sup> Lastly, we prefer the conditional inference approach as it provides the particular advantages of being econometrically more tractable while providing a stronger economic meaning of the identified types.

# **3 Estimating Inequality of Opportunity from Regression Trees**

Originally, tree-based methods were developed to make out-of-sample predictions of a dependent variable based on a number of observable predictors. As we will outline in the following, they can be straightforwardly applied to equality of opportunity estimations and solve many of the

> 4This issue is common to any two-stage analysis in which latent classes serve as controls for a distal outcome. The effect of latent class membership on the distal outcome is attenuated and the explained variability is reduced (Lanza et al., 2013).

6
issues associated with the prevalent estimations approaches outlined in section 2. While we put a particular emphasis on _regression_ trees, our main arguments also hold for _classification_ trees. Thus, the proposed estimation methods are not restricted to continuous variables like income, but can also be fruitfully employed with respect to non-continuous outcomes, such as health (Trannoy et al., 2010) or education (Oppedisano and Turati, 2015).

In what follows we will present two tree-based estimation procedures both of which solve the model selection problem outlined in section 2. First, we will introduce conditional inference regression trees. Their simple graphical illustration is particularly instructive for longitudinal or cross-sectional comparisons of opportunity structures. In spite of their intuitive appeal, however, they perform relatively poorly in out-of-sample predictions. Second, to address the concern of estimate reliability we will also introduce conditional inference forests, which are – loosely speaking – a collection of many conditional inference trees. Forests do not have the intuitive appeal of regression trees. However, they perform significantly better in terms of out-ofsample predictions. In fact, we will show in section 4.5 that they outperform all other considered estimation techniques along this dimension.

# **3.1 Conditional Inference Trees**

Tree-based methods obtain predictions for outcome _y_ as a function of the input variables _I_ = _{I_<sup>1</sup> _, ..., I_<sup>_p_</sup> _, ...I_<sup>_P_</sup> _}_ . Specifically, they use the set _I_ to partition the population into a set of non-overlapping groups, _G_ = _{g_ 1 _, ..., gm, ..., gM }_ , where each group _gm_ is homogeneous in the expression of each input variable. These groups are also called _terminal nodes_ or _leafs_ in a regression tree context. The predicted value for outcome _y_ of observation _i_ is calculated from the mean outcome _μm_ of the group _gm_ to which the individual is assigned. Hence, in addition toˆ theˆobserved income vector _Y_ = _{y_ 1 _, ..., yi, ..., yN }_ one obtains a vector of predicted values _Y_ = _{y_ 1 _, ...,_ ˆ _yi, ...,_ ˆ _yN }_ , where The mapping from regression trees to equality of opportunity estimation is straightforward. Conditional on the input variables being circumstances only, i.e. _I ⊆_ Ω<sup>ˇ</sup> _⊆_ Ω, it is evident that _Y_ eachˆ is analogousresulting groupto the _gm_ smoothed _∈ G_ candistributionbe interpreted _Y_<sup>_EA_</sup> as, thea circumstanceconstructiontypeof which _tm ∈_ we _T_ . haveFurthermore,outlined in section 2 to illustrate ex-ante utilitarian measures of inequality of opportunity. In view of the fact that our predictor space is confined to circumstance variables only, we use the terms “input variables” and “circumstances” as well as “groups” and “types” interchangeably in the following. Input variables will be denoted by _C_<sup>_p_</sup> and groups by _tm_ . In line with equation (5), ˆ we will refer to individual predictions _yi_ as _μm_ .

**Algorithm** Considering all possible ways in which the population can be split into groups is a daunting task when the set of input variables is large. In conventional estimation approaches the researcher is left to her own devices in (i) selecting Ωfrom<sup>ˆ</sup> Ω,<sup>ˇ</sup> (ii) to restrict the number of realizations of each _C_<sup>_p_</sup> _∈_ Ω,<sup>ˆ</sup> and (iii) to determine the relevant interactions among all _C_<sup>_p_</sup> _∈_ Ω.<sup>ˆ</sup> The magnitude of this choice set oftentimes leads to arbitrary model selection. To the contrary, with regression trees the researcher does not need to make these choices herself. The researcher only submits the full and unrestricted set of observed variables that qualify as circumstances, ˇΩ, while the algorithm chooses the relevant circumstances, their subpartition and the respective interactions. To be precise, the observations are divided into _M_ groups (or types) by what is known as _recursive binary splitting_ . Recursive binary splitting starts by dividing the full sample

7
into two distinct groups according to the value they take in one input variable _C_<sup>_p_</sup> . If _C_<sup>_p_</sup> is a continuous or ordered variable, then _i ∈ tm_ if _Ci_<sup>_p<xp_and</sup><sup>_i∈tl_if</sup><sup>_C_</sup> _i_<sup>_p≥xp_.If</sup><sup>_Cp_isa</sup> categorical variable then the categories can be split into any two arbitrary groups. The process is continued such that one of the two groups is divided into further subgroups (potentially based on another _C_<sup>_p_</sup> _∈_ Ω),<sup>ˇ</sup> and so on. Graphically, this division into groups can be presented like an upside-down tree (Figure 1).

Figure 1: Exemplary Tree Representation _Note_ : Artificial example of a regression tree. The grey boxes indicate splitting points, while the white boxes indicate terminal nodes. The values inside the white boxes show predicted values associated with each terminal node ( _μm_ ).

The exact manner in which the split is conducted depends on the type of regression tree that is used. In this paper we follow the methodology proposed by Hothorn et al. (2006), leading to what they call conditional inference trees.<sup>5</sup> Conditional inference trees are grown by a series of permutation tests according to the following 4-step procedure:

1. Test the null hypothesis of independence, _H_ 0<sup>_Cp_</sup> : _D_ ( _Y |C_<sup>_p_</sup> ) = _D_ ( _Y_ ), for each input variable _C_<sup>_p_</sup> _∈_ Ω,<sup>ˇ</sup> and obtain a _p_ -value associated with each test, _p_<sup>_Cp_</sup> .

- _⇒_ Adjust the _p_ -values for multiple hypothesis testing, such that _p_<sup>_C_</sup> _adj._<sup>_p_= 1</sup><sup>_−_(1</sup><sup>_−pCp_)</sup><sup>_P_</sup> (Bonferroni Correction).

2. Select the variable, _C_<sup>_∗_</sup> , with the lowest adjusted _p_ -value, i.e. _C_<sup>_∗_</sup> = _{C_<sup>_p_</sup> : argmin _p_<sup>_C_</sup> _adj._<sup>_p}_.</sup>

- _⇒_ If _p_<sup>_C_</sup> _adj._<sup>_∗> α_:Exitthealgorithm.</sup>

- _⇒_ If _p_<sup>_C_</sup> _ad_<sup>_∗_</sup> _<u>j.</u>_<sup>_≤α_:Continue,andselect</sup><sup>_C∗_asthesplittingvariable.</sup>

> 5An alternative would be Classification and Regression Trees (CART) as introduced by Breiman et al. (1984). CART chooses splits so as to minimize the mean squared error, MSE = _N_ <u>1</u> � _m_ � _i∈tm_<sup>(</sup><sup>_yi−μm_)2.Weprefer</sup> conditional inference trees since CART are biased towards splitting variables made of many categories (Hothorn et al., 2006). Furthermore, we avoid the intricacies of tree _pruning_ (Friedman et al., 2009) by establishing a test criterion that considers the bias-variance trade-off before making an additional split.

8
3. Test the discrepancy between the subsamples for each possible binary partition, _s_ , based on _C_<sup>_∗_</sup> , i.e. _Ys_ = _{Yi_ : _Ci_<sup>_∗< xp}_and</sup><sup>_Y−s_=</sup><sup>_{Yi_:</sup><sup>_C_</sup> _i_<sup>_∗≥xp}_,and obtaina</sup><sup>_p_-valueassociated</sup> with each test, _p_<sup>_C_</sup> _s_<sup>_∗_</sup> .

- _⇒_ Split the sample based on _Cs_<sup>_∗∗_,bychoosingthesplitpoint</sup><sup>_s_thatyieldsthelowest</sup> _p_ -value, i.e. _Cs_<sup>_∗∗_=</sup><sup>_{C_</sup> _s_<sup>_∗_: argmin</sup><sup>_pC_</sup> _s_<sup>_∗_</sup> _}_ .

4. Repeat the algorithm for each of the resulting subsamples.

Conditional inference trees offer a particularly relevant structure in the context of inequality of opportunity. Each hypothesis test is essentially a test for whether equal opportunities exist within a particular (sub)sample. If the algorithm results in no splits at all, then we cannot reject the null hypothesis of equality of opportunity. The deeper the tree is grown, the more types are necessary to fully account for the inherent inequality of opportunities in the society under consideration. Each split tells us that the resulting types have significantly different opportunities under an ex-ante utilitarian interpretation. In all of the resulting types (i.e. the terminal nodes of the tree), we cannot reject the null of equal opportunities.

**Tuning** Note that the structure and depth of the resulting opportunity tree hinges crucially on the level of _α_ . The less stringent the _α_ -requirement, the more we allow for false positives, i.e. the more splits will be detected as significant and the deeper the tree will be grown. So how should _α_ be chosen? On the one hand, _α_ can be chosen a priori in line with the disciplinary convention to require significance levels of at least 5% or even 1%. On the other hand, we can let the data speak on the optimal specification of _α_ , i.e. we can _tune_ the _α_ -parameter to find a model that performs optimally according to a pre-specified testing criterion.

If opting for the latter option, _α_ is typically chosen by _K_ -fold cross-validation (CV). To perform cross validation, one starts by splitting the sample into _K_ subsamples, also called folds. Then, one implements the conditional inference algorithm on the union of _K −_ 1 folds for varying levels of _α_ , while leaving out the _k_ th subsample. This makes it possible to compare the predictions emanating from the _K −_ 1 folds with the real data points observed in the _k_ th fold. The mean squared prediction error serves as an evaluation criterion: This exercise is repeated for all _K_ folds, so that MSE<sup>_CV_</sup> ( _α_ ) = _K_ <u>1</u> � _k_<sup>MSE</sup> _k_<sup>_CV_</sup> ( _α_ ). One then chooses the _α_<sup>_∗_</sup> that delivers the lowest MSE<sup>_CV_</sup> ( _α_ ): In our empirical application we fix _α_<sup>_∗_</sup> = 0 _._ 01, which is in line with the disciplinary convention for hypothesis tests. However, we provide a sensitivity check using cross-validated _α_ in Figure A.1 of Appendix A.3.

> 6One may argue that a criterion that evaluates models according to their capacity to predict _individual_ outcomes is misplaced for ex-ante utilitarian inequality of opportunity estimations. Afterall, we have demonstrated above that we are mainly concerned with estimating type means rather than individual outcomes. In Appendix A.1 we show that the MSE-criterion and its handling of the variance-bias trade-off straightforwardly extends to balancing upward and downward biases in inequality of opportunity estimations.

9
# **3.2 Conditional Inference Forests**

Regression trees solve the model selection problem outlined in section 2 and provide a simple and non-arbitrary way of dividing the population into types. Furthermore, trees are easily mapped and thus lay bare the opportunity structure of a given society for a larger audience. However, constructing the counterfactual distribution _Y_<sup>_EA_</sup> from conditional inference trees suffers from two shortcomings: first, they only make limited use of the information inherent in the set of observed circumstances since not all _C_<sup>_p_</sup> _∈_ Ωare<sup>ˇ</sup> used for the construction of the tree. Yet, the omitted circumstances may possess some informational content that can increase predictive power even though they are not significantly associated with _Y_ at level _α_<sup>_∗_</sup> . This is a particular issue if two important circumstances are highly correlated. Once a split is done using either of the two, the other will unlikely yield enough information to cause another split. Second, the predictions and thus the values of opportunity sets, _μm_ , emanating from trees have a high variance. The structure of trees - and therefore the ensuing distribution _Y_<sup>_EA_</sup> - is fairly sensitive to alternations in the respective data samples. This is a particular issue if there are various circumstances that are close competitors for defining the first split (Friedman et al., 2009). In what follows we will introduce conditional inference forests. Conditional inference forests build methodologically on conditional inference tress and are able to deal with both of these shortcomings (Breiman, 2001).

**Algorithm** In short, random forests create many trees and average over all of these when making predictions. Trees are constructed according to the same 4-step procedure outlined in the previous subsection. However, two tweaks are made. First, each tree is estimated on a random subsample _b_ of the original data.<sup>7</sup> In total _B_ such trees are estimated. Second, a random subset of circumstances Ω<sup> ̄</sup> _⊆_ Ωof<sup>ˇ</sup> size _P_<sup> ̄</sup> is allowed to be used at each splitting point. Together these two tweaks remedy the shortcomings of single conditional inference trees. Drawing only on subsets Ω<sup> ̄</sup> _⊆_ Ωincreases the likelihood that all circumstances with informational content at some<sup>ˇ</sup> point will be identified as the splitting variable _C_<sup>_∗_</sup> and thus addresses the limited information use of regression trees. Furthermore, averaging over the _B_ predictions cushions the variance of individual predictions _μm_ and thus addresses the second shortcoming identified in relation with single regression trees. Therefore, predictions are formed as follows:

**Tuning** From equation (8) it is evident that individual predictions are a function of _α_ – the significance level governing the implementation of splits –, _P_<sup> ̄</sup> – the number of circumstances to be considered at each splitting point –, and _B_ – the number of subsamples to be drawn from the data. Again, these parameters can be imposed a priori by the researcher or they can be determined by tuning the three-dimensional grid ( _α, P, B_<sup> ̄</sup> ) to optimize the out-of-sample fit of the model. In our empirical illustration we proceed as follows. First, to reduce computational costs we fix _B_ at a level at which the marginal gain of drawing an additional subsample in terms of out-of-sample prediction accuracy becomes negligible.<sup>8</sup>

Second, we determine _α_<sup>_∗_</sup> and _P_<sup> ̄</sup><sup>_∗_</sup> by minimizing the _out-of-bag_ error. This entails the following four steps for a grid of values of _α_ and _P_<sup> ̄</sup> :

> 7 Alternatively, one can draw bootstrapped samples, i.e. sample with replacement until a dataset with the same size as the original data is reached. We use the subsampling technique since it has been shown that using bootstrapping leads to biased variable selection (Strobl et al., 2007).

> 8Empirical tests show that this is the case with _B∗_ = 200 for most countries in our sample (see Figure A.2 of Appendix A.3).

10
1. Run a random forest with _B_ subsamples, where _P_<sup> ̄</sup> circumstances are randomly chosen to be considered at each splitting point, and _α_ is used as the value for the hypothesis tests.

2. Calculate the average predicted value of observation _i_ using each of the subsamples _b−i_ (the so called _bags_ ) in which _i_ does not enter: _y_ ˆ _i_<sup>_OOB_</sup> ( _α, P_<sup> ̄</sup> ) = _B_ <u>1</u> _−i_ � _b−i_<sup>_μ_</sup> _m_<sup>_b_(</sup><sup>_α,P_ ̄).</sup> 3. Calculate the out-of-bag mean squared error: MSE<sup>_OOB_</sup> ( _α, m_ ) = _N_<sup><u>1</u></sup> � _i_<sup>[</sup><sup>_yi −y_ˆ</sup> _i_<sup>_OOB_</sup> ( _α, P_<sup> ̄</sup> )]<sup>2</sup> .

4. Choose ( _α_<sup>_∗_</sup> _, P_<sup> ̄</sup><sup>_∗_</sup> ) = _{_ ( _{α ∈ A}, {P_<sup> ̄</sup> _∈ P_<sup>ˇ</sup> _}_ ) : argmin MSE<sup>_OOB_</sup> _}_ .

The logic behind this tuning exercise is similar to cross-validation. However, instead of leaving out the _k_ th fraction of the dataset to make out-of-sample predictions, we leverage the fact that each tree of a forest is grown on a subsample _b−i_ that excludes all observations _i_ . Hence, for each tree we can use the out-of-bag data points to evaluate the predictive accuracy of the respective model.<sup>9</sup> The improved predictive quality of random forests comes at a cost. It is no longer possible to identify a fixed set of types _T_ into which we can partition the population. For example, depending on the subset Ω<sup> ̄</sup> _⊆_ Ωused for a particular tree as well as the the particular subsample<sup>ˇ</sup> _b_ drawn from the data, it may be that _i, j ∈ t_<sup>_b_</sup> _m_<sup>but</sup><sup>_i∈tb_</sup> _m_<sup>+1</sup> while _j ∈/ t_<sup>_b_</sup> _m_<sup>+1.Asaconsequence,</sup> the individual prediction and hence the valuation of the individual opportunity set is an average over the value of opportunity sets _μ_<sup>_b_</sup> _m_<sup>associatedwitheachtreeoftheforest.Therefore,the</sup> valuation of opportunity sets is less straightforward and opportunity structures are hard to illustrate in a graphical manner. It is nevertheless possible to describe opportunity structures by calculating the relative variable importance of each _C_<sup>_p_</sup> _∈_ Ωin<sup>ˇ</sup> constructing the forest. See section 4.3 for an illustration.

# **4 Empirical Application**

In this section we provide an illustration of our methodology using harmonized survey data from 31 European countries. As outlined above, conditional inference trees and random forests solve the issue of model selection associated with the prevalent approaches to equality of opportunity estimations. Conditional inference trees are easily tractable and lend themselves to cross-sectional (and longitudinal) comparisons of opportunity structures. Conditional inference forests are less tractable but outperform the former approach in terms of predictive accuracy. In the following, we will illustrate the merits of both approaches. Furthermore, we will compare the results from both versions of our method with prevalent measurement approaches in the extant literature; namely parametric, non-parametric and latent class models. Comparisons will be made along two dimensions. First, the estimates themselves, and second, the respective out-of-sample accuracy. The latter criterion should be interpreted as an indicator of how well the respective method balances upward and downward biases in inequality of opportunity estimations. A formal argument for why this is the case, is provided in Appendix A.1.

## **4.1 Data**

The empirical illustration is based on the 2011 wave of the European Union Statistics on Income and Living Conditions (EU-SILC). EU-SILC provides harmonized survey data with respect to

> 9In principle tuning can be conducted analogously to regression trees by means of _k_ -fold cross validation. This, however, is computationally expensive. Cross-validation would require to repeat the entire estimation exercise for a total of _K_ folds. This is not necessary when using the out-of-bag error since out-of-sample points are already delivered by leaving out observations _i_ when using bag _b−i_ . Hence, in the case of forests using the out-of-bag error is _K_ times more computationally efficient than cross-validation.

11
incomes, poverty, and living conditions on an annual basis and covers a cross-section of 31 European countries in the 2011 wave.<sup>10</sup> We draw on the 2011 wave since it contains an ad-hoc module about the intergenerational transmission of (dis)advantages, which allows us to construct finely-grained type partitions. The set of observed circumstances Ωand<sup>ˇ</sup> their respective expressions _x_<sup>_p_</sup> are listed in Table 1 whereas descriptive statistics concerning circumstances are reported in Appendix A.2. As an additional advantage, EU-SILC has been extensively studied by the empirical literature on inequality of opportunity and thus provides appropriate benchmarks against which we can compare our method (Checchi et al., 2016; Marrero and Rodr ́ıguez, 2012; Palomino et al., 2016).

The unit of observation is the individual, whereas the outcome of interest is equivalent disposable household income. Aware that inequality statistics tend to be heavily influenced by outliers (Cowell and Victoria-Feser, 1996) we adopt a standard winsorization method according to which we set all non-positive incomes to 1 and scale back all incomes exceeding the 99.5th percentile of the country-specific income distribution to this lower threshold. Our analysis is focused on the working age population. Therefore, we restrict the sample to respondents aged between 30 and 59. To assure the representativeness of our country samples all results are calculated by using appropriate individual cross-sectional weights.

Table 2 shows considerable heterogeneity in the income distributions of our country sample. While households in Norway (NO) and Switzerland (CH) on average obtained incomes above e40,000 in 2010, the average households in Bulgaria (BG), Romania (RO) and Lithuania (LT) did not exceed the e5,000 mark. The lowest inequality prevails in the Nordic countries of Norway (NO), Sweden (SE) and Iceland (IS), all of which are characterized by Gini coefficients of below 0 _._ 22. At the other end of the spectrum we find the Eastern European countries of Latvia (LV), Lithuania (LT) and Romania (EL) with Gini coefficients well above the level of 0.33.

# **4.2 Benchmark Methods**

We compare our estimates from trees and forests against three benchmark methods that have been proposed in the extant literature.

First, we draw on the parametric approach as proposed by Bourguignon et al. (2007) and Ferreira and Gignoux (2011). In line with equation (3), estimates are obtained by a Mincerian regression of equivalent household income on the following controls: father occupation (10 categories), father and mother education (five categories), area of birth (three categories), and tenancy status of the household (two categories). The model specification therefore includes 20 binary variables and resembles the specification used in Palomino et al. (2016).<sup>11</sup> Second, we draw on the non-parametric approach as proposed by Checchi and Peragine (2010). Non-parametric estimates are obtained by partitioning the sample into 40 types. Individuals in type _tm_ have parents of equivalent education (five categories), share their migration status (a binary variable whether the respondent is a first or second generation immigrant), and have fathers working in the same occupation. To minimize the frequency of sparsely populated

> 10The sample consists of Austria (AT), Belgium (BE), Bulgaria (BG), Switzerland (CH), Cyprus (CY), Czech Republic (CZ), Germany (DE), Denmark (DK), Estonia (EE), Greece (EL), Spain (ES), Finland (FI), France (FR), Croatia (HR), Hungary (HU), Ireland (IE), Iceland (IS), Italy (IT), Malta (MT), Lithuania (LT), Luxembourg (LU), Latvia (LV), Netherlands (NL), Norway (NO), Poland (PL), Portugal (PT), Romania (RO), Sweden (SE), Slovenia (SI), Slovak Republic (SK), and Great Britain (UK).

> 11We have estimated the predicted outcomes both as the exponential of the predicted log outcome, _yiEA_ = _P_ ˇ exp _p_ =1<sup>_β_ˆ</sup><sup>_pC_</sup> _i_<sup>_p_</sup> , and by introducing, assuming a normally distributed error term, the correction _yi_<sup>_EA_</sup> = �� � _P_ ˇ exp _p_ =1<sup>_β_ˆ</sup><sup>_pC_</sup> _i_<sup>_p_+</sup><sup>_σ_2</sup><sup>_/_2</sup> , where _σ_<sup>2</sup> is the estimated variance of the error term. We do not find any signifi�� � cant differences in the level of estimated inequality of opportunity when introducing the correction. This may explain why the need of such correction has never been explicitly discussed in previous contributions.

12
Table 1: List of Circumstances

| Respondent’s sex:|- Medium (3-4 ISCED-97)|
|---|---|
|- Male|- High (5-6 ISCED-97)|
|- Female| Father/mother occupational status:|
| Respondent’s country of birth:|- Unknown or dead father/mother|
|- Respondent’s present country of resi-|- Employed|
|dence|- Self employed|
|- European country|- Unemployed|
|- Non-European country|- Retired|
| Presence of parents at home:|- House worker|
|- Both present|- Other inactive|
|- Only mother<br>- Only father<br>- Without parents<br>- Lived in a private household without<br>any parent| Father/mother main occupation (based on<br>International Standard Classifcation of Oc-<br>cupations, published by the International<br>Labour Ofce ISCO-08):<br>- Managers (I-01)|
| Number of adults (aged 18 or more) in re-<br>spondent’s household|- Professionals (I-02)<br>- Technicians (I-03)|
| Number of working adults (aged 18 or more)<br>in respondent’s household|- Clerical support workers (I-04)<br>- Service and sales workers (including|
| Number of children (under 18) in respon-<br>dent’s household|also armed force) (I-05 and 10)<br>- Skilled agricultural, forestry and fshery<br>|
| Father/mother country of birth and citizen-<br>ship:|workers (I-06)<br>- Craft and related trades workers (I-07)|
|- Born/citizen<br>of<br>the<br>respondent’s<br>present country of residence|- Plant and machine operators, and as-<br>semblers (I-08)|
|- Born/citizen of another EU-27 country|- Elementary occupations (I-09)|
|- Born/citizen<br>of<br>another<br>European<br>country|- Father/mother did not work, was un-<br>known or was dead (I-0)|
|- Born/citizen of a country outside Eu-<br>rope| Managerial position of the father/mother:<br>|
||- Supervisory|
| Father/mother education (based on interna-<br>ilSddClififEdi|- Non-supervisory|

| Father/mother education (based on interna-|
|---|
|tional Standard Classifcation of Education|
|1997 (ISCED-97)):|
|- Unknown father/mother|
|- Illiterate|
|- Low (0-2 ISCED-97)|

-  Tenancy status of the house in which the respondent was living: - Owned - Not owned

types we divert from the occupational list given in Table 1 by re-coding occupations into three categories: highly skilled non-manual (I-01–I-03), lower-skilled non-manual (I-04–I-05 and I-10), skilled manual and elementary occupation (I-06–I-09 and father/mother unknown or dead). This 13
Table 2: Summary Statistics

|Country|Sample size<br>Avg.|eq.|income|Std. dev.|Gini|
|---|---|---|---|---|---|
|AT|6,220||25,451|13,971|0.268|
|BE|6,011||23,291|10,948|0.249|
|BG<br>|7,154<br>||3,714<br>|2,491<br>|0.333<br>|
|CH|7,583||42,208|24,486|0.279|
|CY|4,589||21,058|11,454|0.279|
|CZ|8,711||9,006|4,320|0.250|
|DE|12,683||22,221|12,273|0.276|
|DK|5,897||32,027|13,836|0.232|
|EE|5,338||6,922|3,912|0.330|
|EL|6,184||13,184|8,651|0.334|
|ES|15,481||17,088|10,597|0.329|
|FI|9,743||27,517|13,891|0.246|
|FR|11,078||24,299|14,583|0.288|
|HR|6,969||6,627|3,819|0.306|
|HU|13,330||5,327|2,863|0.276|
|IE|4,318||24,867|14,307|0.296|
|IS|3,684||22,190|9,232|0.210|
|IT|21,070||18,786|11,730|0.309|
|LT|5,403||4,774|3,150|0.344|
|LU|6,765||37,911|19,977|0.271|
|LV|6,423||5,334|3,618|0.363|
|MT|4,701||13,006|6,747|0.277|
|NL|11,411||25,210|11,414|0.235|
|NO|5,026||43,260|16,971|0.202|
|PL|15,545||6,103|3,690|0.316|
|PT|5,899||10,781|7,296|0.334|
|RO|7,867||2,562|1,646|0.337|
|SE|6,599||26,346|10,700|0.215|
|SI|13,183||13,772|5,994|0.225|
|SK|6,779||7,304|3,416|0.257|
|UK|7,391||25,936|16,815|0.320|

_Note_ : Summary statistics for the 31 countries in the 2011 wave of EU-SILC. Income variables are measured in Euros.

partition is similar but more parsimonious than the one used in Checchi et al. (2016), who base their analysis on a total of 96 types.

Lastly, we compare our estimates against the latent class approach as proposed by Li Donni et al. (2015). The eligible set of circumstances is the full set of observable circumstances, Ω.<sup>ˇ</sup> We follow Li Donni et al. (2015) in using Schwartz’s Bayesian Information Criterion (BIC) to select the most adequate number of latent types.

# **4.3 Estimates of Inequality of Opportunity**

Table 3 shows inequality of opportunity estimates for our country sample according to five different estimation procedures. Columns 2-4 list results using the parametric, the non-parametric, and the latent class approach, all of which have been proposed in the extant literature (see section 4.2). Columns 5 and 6 list results from conditional inference trees and conditional inference forests, respectively. For all methods, inequality of opportunity estimates are obtained by calculating the Gini index in the counterfactual distribution _Y_<sup>_EA_</sup> .

Of all methods under consideration the parametric approach delivers the highest estimates. For 29 out of 31 countries the inequality of opportunity estimates are higher than the results 14
from both conditional inference trees and forests. Analogously, the unweighted average estimate over all countries equals 0.103 Gini points for the parametric approach as compared to 0.079 and 0.078 Gini points for trees and forests, respectively. Also in terms of country rankings, the parametric approach delivers markedly different results in comparison to our preferred methods. While the parametric approach identifies Romania (RO), Bulgaria (BG) and Greece (EL) as the countries in which opportunities are most unequally distributed, these countries rank 6th, 1st and 5th (6th, 2nd and 7th) in the case of trees (forests).

Non-parametric measures of inequality of opportunity take a middle-ground between the parametric approach and our preferred methods. For 16 (19) out of 31 countries the nonparametric estimate exceeds the estimate coming from trees (forests), while the unweighted cross-country average estimate amounts to 0.084 Gini points. In terms of country rankings the non-parametric approach shows much closer resemblance to our preferred methods than the parametric approach. For example, the three most unequal countries from an opportunity perspective as identified by the non-parametric approach are Bulgaria (BG), Portugal (PT) and Luxembourg (LU), which is congruent with the top three countries identified by trees and forests.

Lastly, the latent class model tends to furnish much lower estimates than all other methods, including trees and forests. This is not very surprising if one considers how latent types are constructed. Latent classes are obtained in the attempt to maximize local independence, that is to minimize the within-type correlation of circumstances. The algorithm constructs types (and selects their most appropriate number) ignoring the covariance of circumstances and outcome. Conditional inference trees instead construct types by maximizing the outcome variability that can be explained by circumstances. For 8 (9) out of 31 countries the latent class estimate falls short of the estimate coming from trees (forests), while the unweighted cross-country average estimate amounts to 0.069 Gini points. Also in terms of country rankings the latent class approach differs markedly from our preferred methods. It identifies Romania (RO), Greece (EL) and Portugal (PT) as the countries in which opportunities are most unequally distributed, whereas these countries rank 6th, 5th and 1st (6th, 7th and 3rd) in the case of trees (forests).

To gain further understanding as regards the relation of existing measurement approaches to our preferred methods, Figure 2 plots the estimates from each method against the estimates from conditional inference forests. The black diagonal indicates the 45 degree line, on which all data points should align if the different methods were perfectly congruent. The upper left panel plots the estimates from the parametric approach against the forest estimates. We can confirm the previous diagnosis that the parametric approach delivers higher estimates than forests (and trees). The difference is particularly pronounced for countries that are characterized by relatively low levels of inequality of opportunity, like the Nordic countries. The upper right panel shows the same plot for the non-parametric approach. We again find relatively high upward divergences in comparison to conditional forest estimates for countries in which inequality of opportunity is low. However, the differences are less pronounced. Interestingly, this pattern in reversed when looking at the correlation plot for the latent class approach in the lower left panel. Instead of overestimating the impact of circumstances in societies of low inequality of opportunity, it underestimates the impact of circumstances in societies that are characterized by high inequality in opportunities. Finally, as expected, trees and forests tend to produce very similar results. The correlation between estimates is high (0.98) and in contrast to all other approaches, the sign of the difference is uncorrelated with the level of the estimate.

# **4.4 Opportunity Structure**

Endowed with an estimate of inequality of opportunity, adequate policy responses must be informed by the particular opportunity structure of a society. That is, policy makers want to 15
Table 3: Inequality of Opportunity Estimates

|Country|Parametric|Non-Parametric|Latent Class|Cond. Inf. Tree|Cond. Inf. Forest|
|---|---|---|---|---|---|
|AT|0.0888|0.0751|0.0796|0.0865|0.0880|
|BE|0.1108|0.0868|0.0534|0.0868|0.0912|
|BG|0.1542|0.1356|0.1148|0.1362|0.1335|
|CH|0.0917|0.0827|0.0631|0.0796|0.0901|
|CY|0.0942|0.0831|0.0738|0.0799|0.0800|
|CZ|0.0716|0.0659|0.0600|0.0569|0.0511|
|DE|0.0704|0.0588|0.0467|0.0697|0.0793|
|DK|0.0772|0.0409|0.0289|0.0212|0.0204|
|EE|0.1108|0.1020|0.0744|0.0967|0.1005|
|EL|0.1476|0.1208|0.1165|0.1264|0.1089|
|ES|0.1421|0.1201|0.0893|0.1280|0.1200|
|FI|0.0687|0.0515|0.0475|0.0197|0.0275|
|FR|0.0858|0.0863|0.0717|0.0904|0.0980|
|HR|0.1312|0.0884|0.0758|0.0822|0.0763|
|HU|0.1098|0.1033|0.0951|0.1134|0.1079|
|IE|0.1048|0.0971|0.0484|0.0843|0.0784|
|IS|0.0669|0.0321|0.0297|0.0123|0.0157|
|IT|0.1213|0.0907|0.0799|0.1078|0.0969|
|LT|0.0947|0.0674|0.0587|0.0693|0.0672|
|LU|0.1340|0.1209|0.0904|0.1326|0.1356|
|LV|0.1335|0.1099|0.0951|0.1102|0.1110|
|MT|0.0872|0.0796|0.0566|0.0710|0.0716|
|NL|0.0661|0.0529|0.0411|0.0284|0.0194|
|NO|0.0480|0.0405|0.0296|0.0202|0.0234|
|PL|0.1111|0.0973|0.0953|0.1019|0.0991|
|PT|0.1376|0.1236|0.1156|0.1362|0.1267|
|RO|0.1698|0.1040|0.1194|0.1204|0.1107|
|SE|0.1178|0.0604|0.0251|0.0247|0.0313|
|SI|0.0772|0.0730|0.0588|0.0317|0.0361|
|SK|0.0626|0.0507|0.0420|0.0495|0.0459|
|UK|0.1012|0.0896|0.0622|0.0714|0.0791|

_Note_ : Estimates of inequality of opportunity using five different estimation methods. Inequality of opportunity is measured as the Gini coefficient in the counterfactual distribution _Y_<sup>_EA_</sup> .

learn about the particular circumstance types the differences among which cause the existence of inequality of opportunity. In this section we illustrate such analyses for both trees and forests. To keep the analysis intelligible we restrict ourselves to two interesting cases: Sweden and Germany.

**Trees** As outlined in section 3.1, the analysis of opportunity structures is particularly intuitive in the case of trees as the relevant information can be directly read off their graphical illustration. Figure 3 illustrates the opportunity structure of Sweden that can be summarized by a tree with two terminal nodes. Inequality of opportunity in Sweden is due to marked differences between first-generation immigrants born outside Europe and the collective group of native residents and European immigrants. The former type accounts for about 10% of the population and on average obtains an equivalent household income that is 35% lower than the corresponding income of the latter group. Recall that each split is based on a statistical test for the existence of equality of opportunity within the respective internal node. Thus, in Sweden we can reject the null hypothesis of equal opportunities for first-generation immigrants born outside Europe and the remainder of the population. However, within these sub-groups the null hypothesis of equality of opportunity cannot be rejected.

A different picture arises when considering Germany. Parental occupation, parental educa16
# Figure 2: Correlation of Estimates by Method

<!-- Start of picture text --> (a) Parametric Approach (b) Non-Parametric Approach<br><!-- End of picture text --> <!-- Start of picture text --> G RO<br>0.150.10 IS DK G GG SE G FI G G SI CZ G MT G LTHR G DEIE G CY GGG UK G ATBE GG CH G FRPLIT GGG EE G ELLV GG HU G ES G G PT G BG G LU 0.150.10 SI G G MT G HRLT G IE GG CY GG UKATBE G CH GG ITEE GG FR GG PLELLV GGG HU G ROES G G PT G BG G LU<br>0.05 NL G NO SK G 0.05 NL G SE G FI G CZ G SK DE G<br>DK G IS G G NO<br>0.00 0.00<br>0.00 0.05 0.10 0.15 0.00 0.05 0.10 0.15<br>CONDITIONAL FOREST CONDITIONAL FOREST<br>(c) Latent Class Analysis (d) Conditional Inference Tree<br>0.15 0.15<br>BG<br>PT G G<br>0.100.05 IS NL G DK GG FI G NO G SI G SE G CZ G SK G LTMT G HRUK G IE GGGGG DECYATCH GGG PLITFRBE GGGG HUEEEL GG RO GG LV G ESPT G LU G BG G 0.100.05 DKNO GG NL G G FI G SE G SISKCZ G G MTLT G HR G UK G IE GGGG DEATCY GG G ITPLBECH GGG HU G FRELEE GGGG ROLVES G G LU<br>IS G<br>0.00 0.00<br>0.00 0.05 0.10 0.15 0.00 0.05 0.10 0.15<br>CONDITIONAL FOREST CONDITIONAL FOREST<br>PARAMETRIC<br>NON−PARAMETRIC<br>LATENT CLASS<br>CONDITIONAL INFERENCE TREE<br><!-- End of picture text --> _Note_ : Comparison of inequality of opportunity estimates based on random forests with estimates based on four other methods. Along the solid line inequality of opportunity is the same for the two methods.

tion, migration status, the number of working adults in the household, and parental tenancy status interact in creating a complex tree made of 14 splits and 15 terminal nodes. The null hypothesis of equality of opportunity is most firmly rejected for individuals whose fathers work in different occupations. If a respondent’s father worked in one of the higher ranked occupations (I-01–I-05), the individual belongs to a more advantaged circumstance type than otherwise (Terminal nodes 5-10). These types together account for 37.4% of the population and have an average outcome of e26,380 – far above the population average of e22,221. However, the advantage of this circumstance characteristic is contingent on the educational status of the father.

17
<!-- Start of picture text --> Birth area<br>p<0.001<br>Lo- IN<br>89.6% 10.4%<br>y = 27126 y= 20155<br><!-- End of picture text -->
|occ_father<br>p < 0.001<br>≤4<br>>4<br>father_edu<br>p < 0.001<br>{Medium, High}<br>{Unknown, None, Low}<br>supervisory_mother<br>p < 0.001<br>Not working<br>{No, Yes}<br>occ_father<br>p < 0.001<br>≤2<br>>2<br>n = 725<br>y = 29574<br>5<br>n = 1097<br>y = 26350<br>6<br>father_birth<br>p = 0.007<br>Country of residence<br>EU−27<br>n = 1952<br>y = 25718<br>8<br>n = 271<br>y = 22808<br>9<br>n = 708<br>y = 21390<br>10<br>mother_edu<br>p < 0.001<br>{Low, Medium, High}<br>{Unknown, None}<br>father_edu<br>p < 0.001<br>High<br>{Unknown, None, Low, Medium}<br>birth_area<br>p = 0.009<br>Native<br>Outside EU<br>n = 593<br>y = 26744<br>14<br>n = 48<br>y = 20041<br>15<br>tenancy<br>p < 0.001<br>Owned<br>Not owned<br>n_w_adult<br>p < 0.001<br>≤3<br>>3<br>father_edu<br>p = 0.003<br>{None, Low, Medium}<br>Unknown<br>n= 2869<br>y = 23265<br>19<br>n_w_adult<br>p = 0.004<br>≤1<br>>1<br>n = 87<br>y = 22615<br>21<br>n = 92<br>y = 17379<br>22<br>n = 126<br>y = 18501<br>23<br>n_w_adult<br>p = 0.007<br>≤3<br>>3<br>n= 2739<br>y = 21590<br>25<br>n = 95<br>y = 17176<br>26<br>tenancy<br>p = 0.004<br>Owned<br>Not owned<br>n = 511<br>y = 19784<br>28<br>n = 770<br>y = 17442<br>29<br>Father occ.<br>p<0.001<br>Father edu.<br>p<0.001<br>Mother Superv.<br>p<0.001<br>Father occ.<br>p<0.001<br>Father birth<br>p<0.007<br>Father edu.<br>p<0.001<br>Birth area<br>p<0.009<br>Mother edu.<br>p<0.001<br>House own.<br>p<0.004<br>House own.<br>p<0.001<br>Working adults<br>p<0.001<br>Working adults<br>p<0.007<br>Father edu.<br>p<0.003<br>Working adults<br>p<0.004<br>{Unknown, None,Low}<br>{Medium, High}<br>{Native}<br>{Outside<br>Europe}<br>{Native}<br>{EU-27}<br>{None, Low,<br>Medium}<br>{Unknown}<br>{Owned}<br>{Not Owned}<br>5.7%<br>8.6%<br>15.4%<br>2.1%<br>5.6%<br>4.7%<br>0.4%<br>22.6%<br>0.7%<br>0.7%<br>1%<br>21.6%<br>0.7%<br>6.1%<br>4%<br>{Service, Clerical, Technician, Professional, Manager}<br>{Professional,<br>Manager}<br>{Not working, Elementary, Plant operator, Craft/Trades,Agriculture}<br>{Service, Clerical,<br>Technician}<br>_Note_: Opportunity tree for Germany. White rectangular boxes indicate terminal nodes. The frst number inside the rectangular boxes indicates the<br>share of the population belonging to this group, while the second number indicates the predicted income. Occupation refers to ISCO-08 one digit codes.<br>All variables describing household characteristics refer to the period in which the respondent was about 14 years old. See Table 1 for details.|
|---|

19
construction of opportunity forests. One measure of variable importance, as proposed by Strobl et al. (2007), is obtained by permuting input variable _C_<sup>_p_</sup> _∈_ Ωsuch<sup>ˇ</sup> that its dependence with _y_ is lost. After this, the out-of-bag error rate, MSE<sup>_OOB_</sup> , is re-computed. The increase of MSE<sup>_OOB_</sup> in comparison to the baseline out-of-bag error indicates the importance of _C_<sup>_p_</sup> _∈_ Ωfor<sup>ˇ</sup> prediction accuracy. Repeating this procedure for all _C_<sup>_p_</sup> _∈_ Ωaffords<sup>ˇ</sup> a relative comparison of all circumstances.

Figure 5 shows the results from this procedure for our example cases of Germany and Sweden. Each black dot is the importance of one of the _P_<sup>ˇ</sup> variables in the set of observed circumstances ˇΩ. We standardize the ensuing results such that the variable importance measure for the circumstance with the greatest impact in each country equals one. For the case of Sweden birth area is the only circumstance that has a meaningful predictive value. In Germany, father’s occupation and father’s education are most important, followed by the number of working adults in the household and mother’s education.

Figure 5: Variable Importance for Germany and Sweden <!-- Start of picture text -->
1.00 G Father's Occupation G Birth Area<br>0.75<br>G Father's Education<br>0.50<br>G # of Working Adults<br>G Mother's Education<br>0.25 G Tenancy<br>GGGGG<br>GG G<br>0.00 GG G G G G G G GG G GG G<br>Germany Sweden<br>Variable Importance<br><!-- End of picture text -->

_Note_ : Each dot shows the importance of a particular circumstance for the predictions from our random forest. The importance of a circumstance is measured by permuting the circumstance, calculating a new MSE<sup>_OOB_</sup> , and computing the difference in the MSE<sup>_OOB_</sup> between the original model and the model with the permuted circumstance. The importance measure is standardized such that the circumstance with the greatest importance in each country equals one. Occupation refers to ISCO-08 one digit codes. All variables describing household characteristics refer to the period in which the respondent was about 14 years old. See Table 1 for details.

It is reassuring that these findings are in line with the graphical analysis of opportunity trees. In Figure A.3 of Appendix A.4 we show variable importance plots for all countries in our sample. Broadly, we can divide our country sample into three groups according to the circumstances that determine their opportunity structure. First, there is a handful of primarily Nordic countries where the respondent’s birth area is the most important circumstance. Second, there is a large group of primarily Western and Southern European countries for which father’s occupation and father’s education are most important. Third, there is a group of Eastern European countries for which mother’s education and occupation is most important.

20
## **4.5 Out-of-Sample Performance**

Recall that current approaches towards estimating inequality of opportunity are subject to different biases. Models are downward biased to the extent that the full set of circumstances Ωis unobserved. Models are upward biased to the extent that they over-utilize the set of observed circumstances Ωleading<sup>ˇ</sup> to overfitted estimates that do not replicate out-of-sample (see Appendix A.1 for the formal argument).

In order to assess how well different estimation approaches trade off these biases, we follow the machine-learning practice of splitting our sample into a _training set_ with _itrain ∈{_ 1 _, ..., Ntrain}_ and a _test set_ with _itest ∈{_ 1 _, ..., Ntest}_ . For each country in our sample, _Ntrain_ =<sup><u>2</u></sup> 3<sup>_N_while</sup> _Ntest_ =<sup><u>1</u></sup> 3<sup>_N_.Wefitourmodelsonthetrainingsetandcomparetheirperformanceonthetest</sup> set according to the following procedure:

1. Run the chosen models on the training data (for the specific estimation procedures, see section 3.1 for trees, section 3.2 for forests, and section 4.2 for our benchmark methods).

2. Store the prediction functions _f_<sup>ˆ</sup> _train_ (Ω).<sup>ˇ</sup>

- ˆ

- 3. Predict the outcomes of observations in the test set: _yitest_ = _f_<sup>ˆ</sup> _train_ (Ω<sup>ˇ</sup> _itest_ ).

Figure 6 compares the resulting MSE<sup>_test_</sup> of the different models. For each country, the MSE<sup>_test_</sup> of random forests is standardized to equal 1, such that an MSE<sup>_test_</sup> larger than 1 represents a worse fit out-of-sample. That implies that the respective method performs worse than forests in trading off upward and downward biases, either by neglecting the use of circumstances or overfitting. We derive 95% confidence intervals based on 200 bootstrapped re-samples of the test data using the normal approximation method.

As expected, random forests outperform all other methods in nearly all cases. On average, the parametric approach gives a fit 9.4% worse than forests. With average shortfalls of around 3%, out-of-sample prediction errors are less pronounced for non-parametric models and latent class analysis. Yet both methods perform worse than conditional inference forests for the vast majority of countries in our sample. Hence, relative to random forests, our benchmark methods either underutilize or overutilize the information contained in Ωand are therefore biased in their<sup>ˇ</sup> inequality of opportunity estimates. The estimates presented in section 4.3 suggest that the parametric and the non-parametric partitions are overfitting the data, while the type partition delivered by latent class analysis is too coarse.

On average conditional inference trees are closest to the test error rate of forests. Yet they also fall short of the performance of forests due to their poorer utilization of the information given in Ω.<sup>ˇ</sup>

# **5 Conclusion**

In this paper we have proposed two novel approaches towards estimating inequality of opportunity based on regression trees. Both conditional inference trees and forests minimize arbitrary model selection by the researcher, while trading off downward and upward biases in inequality of opportunity estimates. On the one hand, conditional inference forests outperform all methods considered in this paper in terms of their out-of-sample performance. Hence, they deliver the best estimates of inequality of opportunity. On the other hand, conditional inference trees are econometrically less complex and provide a handy graphical illustration that can be used for the straightforward analysis of opportunity structures. The fact that trees are very close to forests in terms of their inequality of opportunity estimates (section 4.3), the importance they assign to 21
Figure 6: Comparison of Models’ Test Error <!-- Start of picture text --> (a) Parametric Approach (b) Non-Parametric Approach<br>G<br>1.2 1.2<br>G G<br>G<br>G<br>1.1 G G G G G G G G G G G G G 1.1<br>G G G<br>1.0 G G G G G G G G G G 1.0 G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G<br>CY CH FR AT SK NL HU DE BE UK EE PT ES LV EL AVG DK NL PL HR LV LU CY LT DE EL NO IE HU RO IS AVG<br> CZ BG LU PL SI NO IS FI IE MT RO HR LT IT DK CZ SK BG UK ES EE IT SI FR MT CH AT PT FI BE<br>Country Country<br>(c) Latent Class Analysis (d) Conditional Inference Tree<br>1.2 1.2<br>1.1 1.1<br>G<br>G G G G G G<br>1.0 G G G G G G G G G G G G G G G G G G G G G G G G G 1.0 G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G<br>CZ NL LT HR SI RO LV FI DE FR PT HU ES EL CY AVG HR IS BG ES CZ IT CY RO AT SI MT FR EL PT BE AVG<br> SK UK PL IS NO BG DK MT CH IT LU EE IE AT BE SK DK PL HU FI LT NL NO LV EE LU DE UK CH IE<br>Country Country<br>test<br>test / Random Forest MSE<br> / Random Forest MSE test<br>test<br>Parametric MSE<br>Non−Parametric MSE<br>test<br> / Random Forest MSE test<br>test / Random Forest MSE<br>test<br>Ctree MSE<br>Latent Class MSE<br><!-- End of picture text --> _Notes:_ The figure compares the test error of the different models. The test error of random forests is standardized to 1, such that a test error larger than 1 represent worse fits than random forests. 95% confidence intervals are derived based on 200 bootstrapped re-samples of the test data using the normal approximation method. Sweden is excluded from the figure since it is an outlier. The test errors for Sweden are 1 _._ 43 [1 _._ 21 _,_ 1 _._ 66] for the parametric approach, 1 _._ 11 [1 _._ 01 _,_ 1 _._ 21] for the non-parametric approach, 1 _._ 06 [1 _._ 02 _,_ 1 _._ 11] for latent class analysis, and 1 _._ 06 [1 _._ 01 _,_ 1 _._ 11] for conditional inference trees.

specific circumstances (4.4) and their out-of-sample performance (4.5) makes us confident that they are a useful tool for communicating issues related to inequality of opportunity to a larger audience.

To be sure, the development of machine learning algorithms and their integration into the analytical toolkit of economists is a highly dynamic process. We are well aware that finding the best machine learning algorithm for inequality of opportunity estimations is a methodological horse race with frequent entry of new competitors that eventually will lead to some method outperforming the ones we proposed in this work. Therefore, the main contribution of this work should be understood as paving the way for new methods that are able to handle the intricacies of model selection for inequality of opportunity estimations. While we restricted ourselves to ex-ante utilitarian measures of inequality of opportunity, the exploration of these algorithms for 22
other methods in the inequality of opportunity literature, such as ex-post measures `a la Pistolesi (2009) or ex-ante and ex-post tests `a la Lefranc et al. (2009), provides an interesting avenue for future research.

23