In basic security and/or targeted welfare systems, income support (welfare) payments are designed to provide a minimum standard of living to households who are unable to meet essential consumptions needs with income from private sources (Korpi and Palme 1998) . Long-term receipt of income support thus commonly characterises ongoing experiences of economic disadvantage and social exclusion. It can also impose significant demands on government budgets, reduce economy-wide market output, and lead to the intergenerational transmission of welfare cultures (Cobb-Clark et al. 2017; Dahl et al. 2014; Dahl and Gielen 2020) . For these reasons, long-term income support receipt is an issue many governments around the world aim to prevent (Hanna 2019; HM Government 2010; Reddel 2018; Scoppetta and Buckenleib 2018; Welfare Working Group 2011) .
It is worth noting that welfare receipt is not necessarily a problem in and of itself since it can serve to protect the health and well-being of recipients, particularly those who face unexpected shocks or major structural impediments to work (Aizer and Currie 2004; Mitrut and Tudor 2018) . In contrast, entrenched reliance on income support is considered an unintended cost of work disincentives embedded in program designs, such as that of highly targeted welfare systems (Feldstein 2005; Hoynes 1997; Hoynes and Schanzenbach 2012; Moffitt 1985 Moffitt , 1992 . To prevent such entrenched reliance, policymakers try to intervene early in the welfare careers of high-risk registrants. Yet, this is a challenge since long-term welfare receipt is the product and interaction of multiple risk factors, which often cannot be predicted with a slim set of early warning signs (Bradbury and Zhu 2018) .
Identifying high-risk individuals with greater accuracy is a key motivation for this paper. Greater predictive accuracy can deliver practical benefitsnamely, more effective allocation of resources -compared to targeting efforts based on coarser profiling models. Reflecting this goal, we develop predictive models of income support receipt intensities for any payment enrolee in the Australian social security system, using novel nation-wide social security data combined with Machine Learning (ML) tools. A key attraction of an Australian case-study is that it has one of the most targeted welfare systems in the world, which means that when we predict the risk of long-term income support receipt we closely predict the incidence of ongoing poverty and social exclusion (Whiteford 2010). Furthermore, our research is extremely policy relevant and timely since the Australian government is currently trialling innovative profiling tools in the targeting of early intervention programs, with the specific aim of preventing welfare dependency (Reddel, 2018) . If proven successful, this early warning system could thus act as a blueprint for other countries on resource allocation solutions.
The topic of our research speaks to two previous bodies of work. The first is the nascent literature on estimating ML models to inform resource allocation decisions. This literature has shown that ML algorithms can assist personnel with day-to-day decisions because it better predicts at-risk individuals compared to standard regression tools (Kleinberg et al. 2015) . For example, judges can improve bail-granting decisions (Kleinberg et al. 2017) , health inspectors can be more efficiently allocated to unhygienic restaurants (Glaeser et al. 2018; Kang et al. 2013) , program administrators can better target interventions at high-risk youth (Chandler et al. 2011) , employers can make better hiring decisions (Hoffman et al. 2018) , schools can promptly identify students at risk of dropping out (Sansone 2019), and surgeons can more ethically and effectively screen patients for hipreplacement surgery (Mullainathan and Spiess 2017) . In addition, researchers have combined ML with satellite data to predict poverty levels in countries with limited survey and administrative data (Jean et al. 2016; Yeh et al. 2020) . Although there are limitations to ML potential and applications (Kizilcec et al. 2020; McKenzie and Sansone 2019; Salganik et al. 2020) , researchers have started to analyse how (and if) these algorithms can add value also when they are used to complement the judgement and skills of human experts (Stevenson and Doleac 2019). For example, once ML models identify high-risk individuals, caseworkers can then choose which types (or intensities) of programs ought to be targeted at which individualsone way in which to address the contextual bandit problem (Athey 2019) . To our knowledge, ML models have not yet been applied to predicting the risk of long-term income support receipt.
A second literature is the one that examines the factors explaining long-term income support receipt. Several relevant issues emerge when analysing this topic and they often stem from data limitations. First, welfare churn (i.e., repeated exit and re-entry into the welfare system) and welfare scarring (i.e., the current probability of income support receipt increases as a result of income support receipt in the past) are common phenomena: point-in-time measures or yearly snapshots of welfare receipt can fail to capture these dynamics (Bäckman and Bergmark 2011; Barrett 2000; Bhuller et al. 2017; Tseng and Wilkins 2003) . Second, non-take-up of income support payments among eligible individuals is a significant issue and using data that excludes non-recipients fails to shed light on their income and employment outcomes (Bitler et al. 2003; Currie and Grogger 2001) . And last, the influence of personal characteristics (such as age, sex, ethnicity, household structure) has been shown to interact with the impact of structural factors (such as geography, policy reform, macroeconomic conditions) in explaining entrenched welfare receipt (Bradbury and Zhu 2018) . Therefore, accurately and flexibly modelling relationships between individual characteristics and long-term income support receipt requires larger data combined with a richer set of covariates.
This study bridges the innovations made in these two areas of research by developing predictive ML models of long-term income support receipt. We use novel data of the full population of Social Security System enrolees. These data have several key benefits. First, they are high frequency with daily information of income support receipt status from 2000 to 2019. These repeat observations enable an analysis of the dynamics of welfare exit and entry. Second, they include both welfare recipients and individuals receiving other government transfers, so they also include individuals who did not receive income support payments. Third, they include rich information on over 32 million persons who had contact with the social security system anytime from 2000 to 2019, therefore representing a large dataset ideal for calibrating ML algorithms. Fourth, caseworkers and front-line agency staff already have access to these data, making it easier to integrate the ML algorithms developed here in future decision-making processes.
We argue that our estimated ML models applied to these novel and rich data can be a large part of the solution to the resource allocation problem faced by welfare agencies. First, we show that our estimated ML models significantly improve predictive accuracy compared to simpler predictive models or early warning systems currently in use. Specifically, our simulations show how ML algorithms can increase the R 2 of models predicting the proportion of time individuals will be on income support by at least 22% (or 14 percentage points) compared to alternative models. Second, we argue the predictive models can improve decision-making by systemising the process of identification, helping to promptly identify at-risk individuals using only short-term variables, potentially even as soon as they register with the system. This is especially useful when there has been no prior familiarity with the registrant's situation. In this way, our predictive models can reduce workload pressures on caseworkers (because they act as automatic screening devices) and have the potential to detect and thus prevent conscious and/or unconscious biases (Kleinberg et al. 2020; Kretsedemas 2005; McBeath et al. 2014; Pentaraki 2019) . Furthermore, these improvements can be made at low-cost. In effect, when our predictive models are paired with caseworker expertise, they can enable a superior assignment policy.
The primary purpose of Australia's social security system is to provide people with a "minimum adequate standard of living" (Australian Treasury 2010). The main class of benefits provided are called "income support payments" and they are targeted at individuals with no or low levels of income and/or assets. Generally, for these recipients, income support payments serve as their primary source of income. These welfare payments are provided on a regular basis and assists with basic living costs. The maximum annual income support amount in 2018 ranged from AUD 12,400 (for Unemployment Benefits) to AUD 20,700 (for Disability Benefits), whereas the median annual personal income was AUD 48,400 (ABS 2019a). Thus, those who are receiving an income support payment are a highly disadvantaged group.
There are six main categories of income support payments: (1) student payments; (2) unemployment payments; (3) parenting payments; (4) disability payment; (5) carer payment; and (6) age pension. These main income support payments are strictly means-tested. This means that a formal process is used to determine an individual's eligibility for payments. Entitlement is based on current (not previous) levels of income and assets. Cash transfer amounts reduce when earnings and assets increase. This targeted model applies to all the income support payments; however, the income threshold and taper rates (i.e., how much welfare transfers decrease when earning and assets increase) vary depending on the type of income support payment. For example, Unemployment Benefits have one of the highest taper rates at 50-60%, which means that benefits cut out as soon as recipients earn roughly 24,400 AUD. The taper rates for other benefits are lower at 40-50%. This strictness of the income and asset test gives the Australian welfare system the title as one of the most targeted systems in the OECD. For example, in 2005, the average share of transfers received by the poorest population quintile as a ratio of the share received by the richest quintile was 2.1 across the OECD but as high as 12.4 for Australia. In other words, "the poorest 20 per cent of the [Australian] population [received] more than 12 times as much in social security benefits as the richest quintile" (Whiteford 2010).
For the most part, the welfare system aims to support people in immediate need. However, the Australian government also provides financial support through the social security system that is not considered an income support payment. Other payments available through the social security system include: payments that are designed to assist families with the cost of raising children, such as family tax benefits, maternity leave benefits, supplementary payments for the main income support benefits; and rent assistance. As some of the non-income support payments are relatively untargeted, the social security system includes cross-sections of the population that are relatively financially well-off. Consequently, nearly a fourth of the Australian population receive social security payments. Indeed, in any one fortnight in 2018, about 5 million Australians received an income support payment while approximately 855,000 families received family tax benefits in addition (Whiteford 2018). In total, social security payments equalled AUD 112.4 billion in the 2017-2018 fiscal year (AIHW 2019a).
While the Australian social security system provides for a minimum standard of living with payments targeted to people who do not have the means to support themselves, this goal is balanced with the aim to encourage self-reliance and reciprocity by including activity tests, monitoring and sanctions (Klapdor 2013) . For example, to remain qualified for unemployment payments, recipients are required to actively look, plan and prepare for work in the future. Failure to comply with these mutual obligation requirements can result in loss of payments.
The government also has additional education and training programs and mentoring services specifically aimed at reducing long-term receipt. For example, in Australia in 2016, as part of a 92-million-dollar initiative called "Try, Test and Learn", the government targeted early intervention programs at three high-risk groups of individuals: young carers age 24 or under, young parents age 18 or under, and students receiving study-related income support payments. These groups were selected based on an actuarial valuation of their estimated lifetime welfare costs (Department of Social Services, 2015 , Price Waterhouse Coopers, 2016 . The initiative funded one-on-one tailored support, including short-term education, workshops, mentoring, work placements, and self-employment/business start-up initiatives. An issue with this current profiling approach is that individuals within the broad groups identified as high-risk can have drastically different risk profiles. Using ML-models can allow us to identify more high-risk individuals.
Australian federal social security records from 2000 to 2019 form the basis of our dataset. All social security payments are administered by the national welfare agency called Centrelink. There are over 32 million persons in these data who had any contact with the Centrelink system between 2000 and 2019. All registrants are 15 years or above as this is the minimum age of eligibility. The financial circumstances of these registrants vary greatly: some have high levels of financial needs, such as those who are in receipt of highly-targeted income support payments; others have higher incomes and register with the social security system because they receive one of the non-income support payments described in the previous section, such as one-off government bonuses or costof-children payments.
Each individual is tracked over time on a highly frequent basis. For our main variable of welfare receipt status, we know the precise start and end date associated with payment receipt. A key advantage of this data structure for our study is that we can construct a precise picture of the duration and dynamics of welfare receipt over a long period of time.
The reason we observe such high-(daily)-frequency data is because income support (or welfare) payments are highly targeted. This means that recipients are required to report their financial circumstances and living arrangements to Centrelink on a regular (bi-weekly) basis. For example, income support recipients are required to fill in a 14-34-page form that elicits information about the recipients' (and if applicable, partners') basic information (name, address, contact details, gender, date of birth, ethnicity, language, citizenship, arrival information), marital status and relationship event history, demographic information about their dependent children and, accommodation details, employment and study details.
Our first step is to identify the universe of individuals who received any type of payment from Centrelink in 2014, and who were aged 15 to 66 on 1 January 2014. On average, each person in this sample incurred approximately AUD 160,950 in welfare expenditures over the 2000-2018 period.
We have chosen to focus on the working-age population since this is the target group for programs aimed at preventing long-term welfare dependency. Our next step is, for computational reasons driven by the power of the server where the data are stored, to draw a random 1% sample (50, 615 individuals) from this population. 3
It is worth emphasizing that the resulting sample is not representative of the whole Australian population (Table 1) . It disproportionately captures individuals with low-to middle-incomes. The highly targeted nature of Australian welfare programs means low-income individuals are overrepresented. Furthermore, the non-income support payments, such as the cost-of-children payments, are provided to primary carers of children. This means there is a higher percentage of females in our sample since they continue to be over-represented among primary carers of children (as well as among individuals eligible for other carer-related payments).
We have chosen to focus on individuals in the social security system in 2014 (base year) and then calculate their welfare receipt intensity from 2015 to 2018, inclusive. Note that we ignore the data from 2000 to 2013. This allows us to estimate the machine learning algorithms in a period after the Global Financial Crisis. We have also chosen to exclude the year of 2019 because the end of our observation window is 14 October 2019. Partial inclusion of 2019 data points could bias our results if there are strong seasonal patterns associated with welfare receipt.
We calculate the intensity of welfare receipt as the proportion of time an individual received an income support payment during the four-year period spanning 2015 to 2018, i.e. the number of days they received an income support payment from 1 January 2015 to 31 December 2018 divided by the total number of days over this period. We have chosen this four-year period over which to calculate the duration of welfare receipt because it is long enough to identify long-term welfare dependence but also short enough so that we don't obscure the individual's needs at different stages of the life cycle.
Our measure of welfare receipt intensity is consistent with how welfare reliance has been conceptualised by previous studies and in government reviews such as the most recent McClure review (Reference Group on Welfare Reform 2015). It also captures a more severe form of economic disadvantage than measures that only consider the receipt of income support at one point in time. This is important since receipt of welfare payments is not the same thing as being dependent on welfarea point that has been well emphasised in the literature (Penman 2006).
Long-term welfare receipt is a prevalent issue in Australia. For example, in 2018, almost 3 in 4 income support recipients aged 18-64 had been on a payment for two years or more and among those receiving income support payments in 2009, more than half (56%) were receiving payments 9 years later in 2018 (AIHW 2019b). This issue is also increasing in importance. In 2018, 24.5% of unemployed people aged 15 and over had been looking for work for more than a year (annual average), increasing from 14.8% in 2009 (ABS 2019b). Figure 1 plots the distribution of income support intensity. In Panel A of Figure 1 , focusing on everyone in our sample, we see spikes in the proportion of people who never received income support (32.3% of the sample), and those who received income support consistently over the period 2015 to 2018 (36.7% of the sample). 4 In Panel B of Figure 1 , we display the distribution of income support intensity once we exclude the two extreme cases. Here, we see a more uniform distribution across the spectrum of intensities with some concentration at the higher end.
The DOMINO data capture a wide range of information on individuals. Based on data observed in 2014 (the base year), we include information on: demographics (sex, age, country of birth, and indigenous status); household structure (parent status, number of children, and ages of children), government benefit receipt history (by benefit type); personal relationships (partnership status and marital status, as well as the duration and instability of relationship status); employment and underemployment (zero-hour contracts); work instability; location and residential mobility; housing; education; income and wealth.
A key benefit of the high frequency DOMINO data is the ability to model the dynamics of economic behaviour. For example, we include variables that capture the instability, variability, and intensity (number or duration) over time of income support history and employment. Examples include: exits and entries onto the income support system, annual fluctuations in the amount and duration of benefits, wages and earnings, hours of employment, number of jobs (including simultaneously held jobs), and changes in the working arrangements (day-of-week and hours per week) both within the same employer and across employers.
One limitation of the DOMINO data is a dearth of variables on the individual's non-cognitive ability or behavioural tendencies, such as risk preferences or forward-looking behaviour. To proxy for these unobservable factors, we include measures such as whether the individual was ever sanctioned or had their benefit cancelled because of non-compliance reasons. Indeed, one interpretation of being sanctioned may be disorganisation (Banerjee and Duflo 2014) on the part of the registrant (to fulfil mutual obligations, for example). However, we recognise that these measures are likely to reflect other factors and constraints as well (Klapdor 2013) .
We begin by providing a benchmark for the ML performance and to proxy current methods used by practitioners when predicting individuals at-risk of welfare dependency. To do this, we estimate simple OLS models that predict the proportion of time an individual spends on income support between 2015 and 2018, as reported in Table 2 . The most basic benchmark is a regression containing only a constant term (Model 1). Building on this, we evaluate the predictive power of demographic characteristics: sex (Model 2), education (Model 3), and age (Model 4). We then consider the predictive power of income support historyi.e., a series of binary variables indicating whether an individual had ever received an income support payment at any time in 2014, separately for different types of income support payments -in Model 5.
Another natural benchmark is to estimate a heuristic model, which includes variables that the past literature has identified as key drivers of welfare dependency such as sex, age, education level, parent-status, migration status, ethnicity, marital status, state of residence, and unemployment status (Model 6). We further enhance this model by adding the income support history from Model 5 (Model 7). In addition, given the emphasis in the past literature on local neighbourhood effects (Chetty et al. 2018) , we include detailed geographical information in Model 8.
Finally, we test the potential predictive gain of ML models over the profiling indicators currently used by the Australian government to prevent welfare dependency. As outlined in Section 2, these are indicators for the three groups with the largest estimated lifetime welfare costs: young carers age 24 or under, students receiving a particular income support payment, and young parents age 18 or under.
The heuristic models rely on a small set of variables and on methods like simple aggregation for data reduction. Yet, in practice, simple models are unlikely to represent the complex processes underlying welfare dependency. Fortunately, our baseline data enable greater flexibility because it contains a rich set of possible predictors. We include a wide array of these predictors and different functional forms for each predictor. For example, we include a full set of indicator variables for a recipient's country of birth, their marital status, employment and education status, along with other non-linear expressions (and complex interactions) between these variables (as described in detail in the Online Appendix). Moreover, for continuous time-varying variables such as income, hours worked, and government benefits received, we can incorporate as inputs not only their mean value for each individual over the baseline year, but also measures of their variability over time. (2017), we do not drop redundant or aggregated variables since they could be useful to obtain better predictions with less complexity. For example, we include categorical variables recording the number of children in the household while also including an extensive set of binary indicators indicating family size and fertility progression. The net result of this is that we have approximately 1,800 possible predictors. Moreover, if we start considering interaction terms between some of these predictors, the number of variables can reach or even exceed the number of individuals in the data. In a standard OLS regression, this would not be possible.
By contrast, ML efficiently handles high-dimensional data. A common issue with adding complexity to a model (in the form of more variables, more interactions, or more flexible functional forms), however, is overfitting. This issue is evident when an estimated model tightly fits a training sample but poorly fits new samples. The problem of overfitting cannot be easily solved by estimating simple models (e.g., with fewer variables subjectively selected) because this may result in models that underfit the data and exclude powerful predictors. For instance, if the true relation between y and x is quadratic, a linear model would be an under-fit (high bias), while estimating a 4 th degree polynomial would lead to over-fitting (high variance). ML addresses the bias-variance trade-off by carefully reducing the number of variables, and through a process called regularization. The latter keeps all the variables but reduces the magnitude of each coefficient. This works well when there are numerous variables that contribute to predicting the outcome in a statistically significant, albeit economically modest, fashion. We have used three different ML algorithms which employ one or both methods for dimension reduction.
Estimating a range of ML models (as we do) is advisable because each have their benefits and drawbacks. As noted by , there are no formal results that shows that one ML approach is better than another. Therefore, the choice of which algorithms to use can often be rather arbitrary. We choose to estimate different classes of ML models, including LASSO, Support Vector Regression (SVR) and Boosting (reported in Table 2 Models 10-12). This allows us to evaluate algorithms which have different levels of flexibility and interpretability. LASSO is straightforward to explain given its similarity with OLS and can be easily interpreted. SVR offers extremely flexible functional forms. Similarly, Boosting can account for a large number of possible interactions among the inputs. We have briefly summarised each of these algorithms in the Online Appendix. A more comprehensive review of the tools available to practitioners is provided by, among others, Hastie et al. (2009) , Mullainathan and Spiess (2017) , Athey and Imbens (2019).
We follow the recommendation of Mullainathan and Spiess (2017) and split the data into two subsamples. A training sample (80% of the data) is used to calibrate and estimate the algorithm under each of the ML methods. Out-of-sample performance is reported using the hold-out sample (the remaining 20% of the data). A detailed step-by-step description of our calibration procedure is available in the Online Appendix.
In many cases, a single algorithm does not perform as well as a combination of methods. Different machine learning algorithms may capture different features of the data. Therefore, combining these algorithms may potentially lead to a superior performance . We have aggregated in Table 2 Model 13 the predictions from the three main ML algorithms (LASSO, SVR, and Boosting) using weights obtained through running a linear regression of the outcome on these predicted values, as described in Mullainathan and Spiess (2017) . Table 2 shows the out-of-sample performance of the aforementioned models when predicting the proportion of time an individual receives income support between 2015 and 2018 using their available information from 2014. For each model, we report the Mean Squared Error (MSE) to compare the predictive accuracy of different models. The MSE is a widely accepted criterion to measure the performance of models that predict continuous variables. Given its intuitive appeal, we also report the square of the Pearson correlation coefficient, i.e. the correlation between the actual and fitted dependent variable. This is equivalent to the R 2 in linear least squares regressions. All these goodness-of-fit measures are estimated out-of-sample using a holdout sample. We use a separate training sample to estimate and calibrate the models. In-sample performances are reported in Table B4 in the Online Appendix.
In Models 2-4, we report that demographic characteristics have limited predictive power: OLS models including only information on sex or education have an R 2 of 3% or lower (Models 2-3). The OLS model using age as the predictor explains a higher percentage of the variability in the outcome variable (17.4%, Model 4). This is to be expected given the age requirements in several income support programs. Despite the improvement in predictive performance over Models 2-3, the R 2 remains low in absolute terms.
In Model 5, we show that income support history is strongly correlated with future welfare dependency. Specifically, a simple OLS model that includes binary indicators for any income support payment (separately for different types of payments) received in 2014 can explain almost 60% of the variability in the proportion of time on income support in the four subsequent years.
In Model 6, we find that the heuristic model does not improve performance in a substantive way. This is surprising as the set of predictors of demographic characteristics, location, employment and education status is commonly used by economists to explain economic behaviour. Further adding this set of predictors to Model 5 (i.e., with the income support history) only leads to a small reduction in the MSE. This suggests that our indicators of income support history are stronger predictors of future welfare receipt intensity. We obtain similar conclusions when we expand the set of controls to include detailed geographic information (Model 8). In addition, we show that the current profiling indicators used by the Australian government to prevent welfare dependency perform poorly when modelled in an OLS specification (Model 9). Importantly, the results in Table 2 indicates that using ML to exploit a larger set of information already available in administrative data (that are accessible by caseworkers) lead to performance gains (Models 10-12). These improvements are consistently achieved irrespective of the particular ML algorithm implemented. The stability of the performance across algorithms is both reassuring and surprising given the contrast in their data-fitting mechanics (in terms of the flexibility allowed for functional forms and interaction terms). Combining all three algorithms (LASSO, SVR, and Boosting) in Model 13 further improves prediction performance. It is important to note that we obtain this out-of-sample predictive accuracy without overfitting the data (as evident in Table B4 ) and by using a 1% random sample of the full set of observations. In our context, it suggests that further sample size augmentations (and the attendant computational costs) are unnecessary to properly calibrate these ML algorithms.
The Ensemble method, compared to the benchmark models (Models 1-9), performs significantly better. Specifically, its out-of-sample MSE is less than one-fourth of the corresponding MSE of the basic benchmark model using only a constant (Model 1). Furthermore, its MSE represents a 42% reduction from the MSE of Model 5 with only the income support history. Bootstrapped confidence intervals are also tight around the mean MSE, suggesting that all comparisons in predictive performance between OLS and ML models are significantly different from zero. Similarly, the R 2 jumps to more than 76%, almost a 14-percentage point (or 22%) increaseagain, at no extra costcompared to the OLS model with the highest R 2 (Model 8). According to our calculations of the total annual accrued welfare costs, the individuals identified in the ML model accrued an additional welfare cost of AUD 0.99 billion compared with the comparably sized (three) groups identified in the actuarial model. To give a sense of the magnitude, back-of-theenvelope calculations suggest that this represents roughly 10% of the total annual unemployment benefit expenditure (AIHW 2019a).
Given the results in Model 5 Table 2 , agencies lacking the time or resources to develop and apply more sophisticated models could identify -with a medium level of accuracy -individuals already in the system who are at risk of relying on income support for an extended period of time using only past income support payment information. Alternatively, one could focus on the top predictors from LASSO (Model 10 Table 2 ) and Boosting (Model 12 Table 2 ): Table 3 lists the most powerful predictors selected by these two algorithms. While most of these variables are not surprising, a subset of themsuch as income fluctuations, number of residential moves, and failure to complete means tests -is not routinely used in simple early warning systems. However, as shown in Models 1-3 in Table 4 , 5 using these selected inputs in an OLS model would yield high out-ofsample performance. Therefore, while ML algorithms reach the lowest out-of-sample MSE, insights from these algorithms can be easily incorporated into simpler modelling frameworks such as an OLS model. This is especially useful if technical, political, or administrative barriers prevent ML algorithms from being implemented at a larger scale. Table 4 reports additional extensions and robustness checks. Despite documented large economic inequalities between native and non-native populations (Markham and Biddle 2018), indigenous status has limited predicted power in this context (Model 4). Similar conclusions are reached when focusing on country of birth (Model 5). In Model 6, to investigate how much income dependency is affected by local factors, we further extend our set of detailed geographical inputs to include almost 2,500 postal codes, but we do not improve substantially upon the model that includes only information on income support history (Model 5 in Table 2 ). Including indicators of parental welfare receipt -a factor that has been shown to increase the likelihood of one's own welfare receipt (Cobb-Clark et al. 2017; Dahl et al. 2014; Dahl and Gielen 2020) -to the heuristic models has only a marginal effect on out-of-sample performance (Model 7-8). Finally, Model 9 uses the same variables used in the heuristic Model 6 in Table 2 , but as inputs of a Boosting algorithm rather than an OLS model. The out-of-sample R 2 increases from 30.6% to 37.3%, thus suggesting that interaction terms and non-linearities do play a role in this context, but they are not enough to reach the same performance as the OLS model with income support history or the ML algorithms with the full set of inputs.
In this paper, we show that ML algorithms applied to data already available to caseworkers can significantly improve predictive accuracy compared to OLS models that use a smaller set of variables or compared to currently used early warning systems. The ML predictions can be used to potentially reduce workloads for caseworkers and to supplement their expertise in the process of identifying high-risk individuals. We find evidence that using the better-performing ML models, as opposed to the currently used early warning indicators, could lead to substantial savings in public spending because individuals identified by the former accrue on average higher welfare expenditures than those identified by the latter.
Several caveats are worth noting. First, although we have used some of the most popular ML algorithms, it is possible that more advanced algorithms or extensive grid-searches could further improve performance. However, such algorithms may still be computationally infeasible, even for big companies (Johnston 2012 ), or extremely difficult to code. In addition, these algorithms have been designed to be applied to large data sets with millions of observations and thousands of potential inputs. Even if administrative data have extremely large numbers of observations, the number of potential inputs is often limited. For instance, we did not have any information on where the individual went to college, the individual's risk preferences, their non-cognitive skills, or the full record of their parents' education and employment history. We should also emphasise that our simulations are based on data collected during a period of stable or improving macroeconomic conditions; ML algorithms would need to be retrained using data from economic downturns in order to improve external validity claims, especially when predicting welfare dependency among individuals affected by the Covid-19 pandemic. Despite these limitations, the findings in this paper suggest that researchers and practitioners could obtain more precise predictionsat no extra costs -by exploiting the available datasets.
Second, the predictions in this paper are different from treatment effect estimates that a policymaker may use to optimally allocate resources (Athey 2017) : such agent would be more interested in knowing which individuals would benefit the most from a welfare transfer. But even in such cases, accurately identifying a pool of high-risk individuals may be a useful first step. Furthermore, by systemising the screening process, it can potentially limit the impact of explicit and implicit biases, as well as allow caseworkers to focus on the most critical welfare recipients. In addition, from a social welfare perspective, policymakers may prioritise allocating resources to the most vulnerable individuals even if it does not necessarily maximise the marginal gains from a treatment program.
Future research could explore how to use the predictions from these algorithms as a preliminary step in randomised control trials (Duflo 2018) . For instance, these algorithms could help researchers to identify their population of interest, e.g. the subset of individuals most at-risk, and then design an RCT to find the appropriate treatment to support these welfare recipients. Chandler et al. (2011) provide an early example of incorporating predictive models in public policy programs targeting specific sub-populations. In this context, ML techniques known as multi-armed or contextual bandits can regularly update algorithms with newly harvested information (on registrants) and recommend personalised treatments (Athey, 2019) . Finally, future research could elucidate how ML algorithms ought to be effectively combined with human expertise, including an understanding of the contexts in which one should rely heavily on ML algorithms and those in which we ought to solely rely on humans ( Model 9 identifies three groups with the largest estimated lifetime welfare costs: young carers age 24 or under ("Carer"); students receiving a particular Income Support payment ("student"); and young parents age 18 or under ("parent"). All variables are described in Appendix A. LASSO selected 323 variables and Boosting selected 72 variables. The missing indicator selected by Boosting -"Number of jobs (missing)" -is perfectly collinear with the variable "Ever on Income Support" (p_evis14). In LASSO, the reported coefficients are computed by taking the selected variables and estimating an OLS linear regression with them. In Boosting, the influence of an input depends on the number of times a variable is chosen across all iterations (trees) and its overall contribution to the loglikelihood function; such values are then standardised to sum up to 100. All variables are described in Appendix A. 
Proportion of time spent on income support from 2015 to 2018 (isprop) . This is the total number of days an individual received income support payments from 1 January 2015 to 31 December 2018 divided by the total number of days spanning this period. Income support payments include the following:
• 
For each variable, missing values (if any) have been set to zero and a new binary variable has been generated to indicate the observations that are missing.
Female (p_female) records whether an individual is recorded as a woman.
Age records an individual's age on 1 January 2014. We have deleted anyone aged less than 15 and older than 66 as on 1 January 2014. We have created a series of binary indicators for ages 15 to 66 (p_aged*) as well as for the following age categories:
• Below 25 (p_agecatd1)
• 25-29 and then in increments of 5 years up to age 64 (p_agecatd2-p_agecatd9) • 65 and above ((p_age65ab)
• Receipt of Remote Area Allowance in 2014 (p_evraa2014) records whether an individual had ever received Remote Area Allowance in 2014. Remote Area Allowance is not an income support payment in its own right, but rather an additional payment that income support recipients can receive if they live in a remote area.
Receipt of one-off Crisis Payment in 2014 (p_cripay2014) records whether an individual had ever received a one-off Crisis Payment in 2014. Crisis Payment is a one-off payment that income support recipients can access if they are in severe financial difficulty due to extreme circumstances.
. This is the total number of days an individual received income support payments from 1 January 2014 to 31 December 2014 divided by the total number of days spanning this period.
Times transferred between different income support payments (p_numistran) records the number of times an individual had transferred between different income support payments in 2014.
Seasonality in Centrelink administered payment receipt. A set of indicators recording whether an individual had ever received a Centrelink administered payment for each month (p_Jan2014-p_Dec2014), as well as quarter of 2014 (p_qr12014-p_qr42014).
Duration on benefits in 2014 (p_bendur2014) measures the total number of days an individual was in the Centrelink system in 2014 including the days they were receiving a government payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) and the days when they were suspended from a payment.
Government benefit durations in 2014 record the total number of days an individual was receiving income support payments (p_isdur14), Family Tax Benefits (p_ftbdur2014) and the following (types of) income support payments in 2014:
• Disability Support Pension ( Ever received advanced payment in 2014 (p_evadv2014) records whether an individual had ever received advanced payments in 2014. This is a companion variable to Ever received a one-off Crisis Payment in 2014 (p_cripay2014). The difference between p_evadv2014 and p_cripay2014 is that the former shifts forward a scheduled payment whereas the latter is an additional payment.
Maximum proportion of income support recipients in neighbourhood in 2014 (p_maxisnei2014) measures the maximum percentage of income support recipients in any of the postcodes or SA1 regions an individual has lived in 2014.
Sanctioned in 2014 (p_evsus2014) records whether an individual had ever had their Centrelink administered payment suspended during 2014.
Government Benefit terminated for non-fulfillment/breach in 2014 (p_evsannb2014) record whether an individual had ever had a government benefit terminated for non-fulfillment or breach of activity agreements to income support receipt in 2014.
Ever single in 2014 (p_evsing2014) records whether an individual had ever been single (having a marital status that is either single, separated, divorced or widowed) in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Ever married in 2014 (p_evmar2014) records whether an individual had ever been married in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Ever in a de facto partnership in 2014 (p_evdef2014) records whether an individual had ever been in a de facto partnership in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Ever separated in 2014 (p_evsep2014) records whether an individual had ever been separated in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Ever divorced in 2014 (p_evdiv2014) records whether an individual had ever been divorced in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Ever widowed in 2014 (p_evwid2014) records whether an individual had ever been widowed in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Ever changed relationship status in 2014 (p_evchms2014) records whether an individual had ever changed relationship status between being single (single, separated, divorced or widowed) and partnered (de facto or married) in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment. (p_numchms2014) records the number of times an individual had changed relationship status between being single (single, separated, divorced or widowed) and partnered (de facto or married) in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Relationship duration in 2014 (p_reldur2014) records the number of days from 1 January 2014 to 31 December 2014 for which an individual had known relationship status in the DOMINO data, i.e. either single, separated, divorced, widowed, de facto partnered or married.
Single parent in 2014 (p_evlp2014) records whether an individual had ever been a single parent in 2014. We have constructed this variable from first identifying whether an individual had ever been single and whether an individual had even been a parent within each half-month period of 2014, separately. If an individual had been both single at a point of time during any of these 24-period and the same individual had also been a parent within the same period, we have coded the individual as having been a single parent in 2014.
With child in 2014 (p_parent2014) records whether an individual had ever had a child in their care in 2014 when they in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Children below 16 in 2014 (p_kidb16in2014) record whether an individual had ever had a child that was under the age of 16 in their care in 2014 when they in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Children below 10 in 2014 (p_kidb10in2014) record whether an individual had ever had a child that was under the age of 10 in their care in 2014 when they in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Children below 5 in 2014 (p_kidb5in2014) record whether an individual had ever had a child that was under the age of 5 in their care in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Age of youngest in 2014 (p_chageyng2014) records the age of the youngest child in an individual's care in 2014.
Age of eldest in 2014 (p_chageeld2014) records the age of the eldest child in an individual's care in 2014.
Average age of children in 2014 (p_chagemean2014) records the average age of the children in an individual's care in 2014.
Number of children by age group records the number of children an individual had under their care in 2014 that fell into the following age categories:
• Below 1 (p_numchb1) • At least 1, below 3 (p_numch1t3) • At least 3, below 5 (p_numch3t5) • At least 5, below 8 (p_numch5t8) • At least 8, below 10 (p_numch8t10) • At least 10, below 15 (p_numch10t15) • 15 and above (p_numch15ab)
Presence of children by age (p_havekidaged*) is a set of binary indicators for whether an individual had a child under their care in 2014 that was aged for each year from 0 to 16, respectively.
Presence of children by age group is a set of binary indicators for whether an individual had a child under their care in 2014 that fell into the following age categories:
• Below 1 (p_havekidb1) • At least 1, below 3 (p_havekid1t3)
• At least 3, below 5 (p_havekid3t5) • At least 5, below 8 (p_havekid5t8) • At least 8, below 10 (p_havekid8t10) • At least 10, below 15 (p_havekid10t15) • 15 and above (p_havekid15ab)
Annual employment income in 2014 (p_totinc2014) records the total dollar amount an individual earned through employment in 2014 when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment. We have not top-coded this variable, but we have created a new variable to indicate values at the 99 th percentile or above of the entire income distribution for the sample (p_inctc).
Annual hours in 2014 (p_tothr2014) records the total hours worked by an individual in 2014 when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment. We have top-coded this variable so that its maximum is 5200 hours and we have created a new variable to indicate values that have been top-coded (p_thrtc).
Wage rate in 2014 (p_wage2014) records the wage rate calculated by annual employment income in 2014 over annual hours in 2014.
Average daily hours worked in 2014 (p_avhr2014) records the average daily hours worked by an individual in 2014 across income spells when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment.
Minimum daily employment income in 2014 (p_mininc2014) records the minimum daily income an individual earned through employment in 2014 when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment. We have not top-coded this variable, but we have created a new variable to indicate values at the 99 th percentile or above of the entire income distribution for the sample (p_dinctc).
Minimum daily hours in 2014 (p_minhr2014) records the minimum daily hours worked by an individual in 2014 when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment.
Zero hour contracts in 2014 (p_evzh) records whether an individual had ever worked in a zerohour contract (where they reported zero income and zero hours, which is distinguished from the case of positive income and zero hours which reflect bonuses received for a given job) for continuous income in 2014 when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment. If the individual was never in the income support system in 2014 but had worked in a zero-hour contract while receiving some non-income support government benefits in 2014, then we have also set this variable to 1 for this person but otherwise this variable is missing for those who never received any income support in 2014. This is a proxy for underemployment.
Number of jobs in 2014 (p_numjob2014) records the total number of jobs, as measured by a change in employers, an individual had worked in when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment.
Maximum number of simultaneous jobs worked in 2014 (p_maxsimjob2014) records the maximum number of simultaneous jobs, as measured by working with different employers at the same time, an individual had worked in when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment.
Number of substantial employment spells (p_lempnum2014) records the number of substantial employment spells an individual had in 2014. To measure substantial employment spells, we ignore any breaks in employment of 28 days or less.
Minimum wage worker in 2014 (p_minwage2014) records whether an individual had worked in a job whose hourly wage was at or below the federal minimum wage in July 2014 when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment.
Income fluctuation in 2014 (p_sdinc) records the fluctuation in the amount of an individual's employment income in 2014 when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment, as measured by the standard deviation in the amount of employment income earned on a bi-weekly basis.
Hours fluctuation in 2014 (p_sdhr2014) records the fluctuation in the amount of an individual's employment hours in 2014 when the individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment, as measured by the standard deviation in hours worked on a bi-weekly basis. Note that we have first top-coded fortnightly total working hours to 200 hours.
Average length of employment tenure (p_maxjobten2014) records the average length of tenure with a given employer in 2014 when an individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment.
Maximum length of employment tenure (p_avjobten2014) records the maximum length of tenure with a given employer in 2014 when an individual was in the income support system, i.e. they were receiving income support payments or were suspended from a payment.
Number of times that working hours changed in 2014 (p_numhrch) records the number of times an individual changed their working hours by more than 20% on a bi-weekly basis when they were in the income support system, i.e. they were receiving income support payments or were suspended from a payment.
Intensity of working hours change in 2014 records whether an individual had changed working hours on a bi-weekly basis when they were in the income support system, i.e. they were receiving income support payments or were suspended from a payment. We distinguish magnitudes of change by the following categories:
• Decreased by 51% or more (p_evhrd50m)
• Decreased by 31-50% (p_evhrd31t50)
• Decreased by 11-30% (p_evhrd11t30)
• Remained relatively unchanged: hours fluctuate by up to 10% (p_evsamehr)
• Increased by at least 11-30% (p_evhri11t30)
• Increased by at least 31-50% (p_evhri31t50) • Increased by 51% or more (p_evhri50m)
Ever had increased work instability in 2014 (p_evacqvar2014) records whether an individual had ever had increased work instability in 2014 when they were in the income support system, i.e. they were receiving income support payments or were suspended from a payment, as measured by whether they had only continuous income spells in 2014 and then 'acquired' new spells of variable income. If the individual was never in the income support system in 2014 but had 'acquired' new spells of variable income while receiving some non-income support government benefits in 2014, then we also set this variable to 1 for this person but otherwise this variable is set to missing for those who never received any income support in 2014
Ever had increased work instability within employer in 2014 (p_morewithinwi2014) records whether an individual had ever stayed with same employer but went from having continuous to variable income with the employer in 2014 when they were in the income support system, i.e. they were receiving income support payments or were suspended from a payment. If an individual was never in the income support system in 2014 but had increased work instability within the same employer in 2014, then we also set this variable to 1 for this person but otherwise this variable is set to missing for those who never received any income support in 2014.
Ever had changed days worked in the week in 2014 (p_evchwday2014) records whether an individual had ever changed days worked in the week in 2014 when they were in the income support system, i.e. they were receiving income support payments or were suspended from a payment. This is derived from examining the day of the week on which individuals' income spell start and end on. For example, if their reporting changed from being from Sunday to Saturday to being from Monday to Sunday instead, we assume they had changed the days in the week that they worked on in 2014. If the individual was never in the income support system in 2014 but had changed days worked in the week in 2014, then we also set this variable to 1 for this person but otherwise this variable is set to missing for those who never received any income support in 2014. payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment.
Rental Assistance in 2014 including the following:
Rental burden in 2014 (p_rentstr2014) records the rental burden an individual experienced in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment. It is calculated as the total rent paid in 2014 divided by the sum of the total amount of government benefits received in 2014 and the annual employment income in 2014. We have not top-coded this variable, but we have created an indicator for rental burden being greater than 1 (p_rstc).
In the DOMINO data, we have postcode (ZIP code) information. Also, where geocoding can be applied we use geographic information pertaining to the Australian Statistical Geography Standard (ASGS), a framework used by the Australian Bureau of Statistics that covers the all of Australia and allow a variety of different geographical aggregation for which data analysis can be undertaken at. The ASGS has 4 levels.
• Statistical Areas Level 1 (SA1s) are the smallest geographic aggregation, with a population between 200 and 800 people • Statistical Areas Level 2 (SA2s) are aggregations of SA1s and are meant to reflect functional areas that represent a community that interacts together socially and economically. Its population size ranges from 3000 to 25,000 persons • Statistical Areas Level 3 (SA3s) are aggregations of SA2s. They are broad geographic regions that are about the size of a town or a cluster of suburbs around a major business and transport centre. Population size ranges from 30,000 to 130,000
For the DOMINO, we have SA1 and SA2 codes, from which SA3 codes can be easily derived. For the main set of geographic variables, we use the SA3 level.
Moved in 2014 (p_evmove) records whether an individual had ever moved during 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from said payment. We proxy residential moves with movement between SA1 regions or postcodes. Temporary housing in 2014 (p_evtempres2014) records whether an individual had ever lived in temporary housing in 2014 when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from a payment. This is measured by an individual having a residence spell that does not coincide in timing with a permanent residence spell in 2014 because this indicates that the individual was using a Postal Office Box for correspondences rather than living in temporary housing in 2014.
Ever studied in 2014 (p_evsd2014) records whether an individual had ever studied when they were in the Centrelink system, i.e. they were receiving a Centrelink administered payment (including income support payments, Family Tax Benefits and other payments such as rental assistance) or were suspended from said payment in 2014.
Highest education attainment in 2014 records an individual's education attainment up to the last point they were observed in 2014 for the following levels:
• Children Carer in 2014 (p_chcare2014) records whether an individual had ever cared for a child when they were receiving Carer Payment in 2014.
Note: these inputs have been used in extension analyses (Table 4) and not in the main ML algorithms (Table 2) Location Postcode in 2014 (p_evlpc*) includes a set of indicators for whether an individual had ever lived in each of the postcodes in Australia in 2014.
Father's income support history in 2014 records whether any of the fathers who had ever cared for an individual before 2014 had ever received any type of income support payments during 2014 (p_dais14), as well as receipt for the following (types of) income support payments:
• Disability Support Pension ( 
The first method we have used in our analysis is LASSO (Least Absolute Shrinkage and Selection Operator). This method adds a penalisation term to the OLS objective function:
Where k is the number of potential predictors (potentially larger than the number of observations, n), and λ is the regularization (or penalization) term. In practice, this algorithm selects variables with high predictive power, and shrinks their coefficients, while constraining all other variables to have coefficients of zero. Therefore, the key underlying assumption of LASSO is sparsity, i.e. that only a limited set of inputs in ′ play an important role in predicting the outcome variable . LASSO is increasingly being used by economists (Mullainathan and Spiess 2017) , with several applications also in studies estimating causal effects (Belloni et al. 2014; Knaus et al. 2020) .
We have estimated the algorithm in Stata 16 using the built-in command lasso linear. We have chosen the penalisation term λ via 5-fold cross-validation in the 80% training sample using a grid search over 100 values. As shown in Table B1 , the selected λ is in the interior of the interval considered for the grid search. In addition, Figure B1 plots the relation between λ and the 5-fold average cross-validation MSE (plotted using the command cvplot in Stata 16). This figure shows that the choice of λ substantially affects the LASSO performance, and that the selected value is indeed the one that minimises the MSE. Furthermore, as shown in Tables 2 and B4 , the in-sample and out-of-sample MSE are close to each other, thus confirming that the algorithm is not overfitting or underfitting the training data. Figure B1 : LASSO cross-validation
The second method we have used in our analysis is Support Vector Machines (SVM). SVM aims to classify the data into two groups by means of a hyperplane in a high-dimensional space. It can be written as a modified penalized logistic regression that solves the following objective function:
There are four terms here that differ from that in the standard logistic regression, and which allow SVM greater flexibility in covering a wide range of functional forms and dealing with highdimensional data. In addition to the penalization factors, and ‖ ‖ 2 ; there is a kernel K, and the max(0,.) component, which allows for some observations to be misclassified close to the separating margin. SVM has been found to achieve better performance than a number of other machine learning algorithms in some applications (Maroco et al., 2011) . Nevertheless, given the similarity in terms of objective function, in other cases its performance has been found to be similar to that obtained with logistic regressions (Verplancke et al., 2008) .
In practice, we have used an extension of SVM for continuous outcomes called Support Vector Regression (SVR). As SVM deviates for the logit model, SVR likewise deviates from OLS by using kernels to flexibly map the predictors into a hyperplane in a high-dimensional space, by allowing a margin in which it is insensitive to errors, and by adding a penalisation term (Guenther and Schonlau, 2016) . One downside of SVM and SVR is that the parameters are difficult to interpret, and so it is not possible to say which variables matter most for determining the predictions made by this method.
We have implemented Support Vector Regression in Stata 16 using the command svmachines (Guenther and Schonlau 2016) . A grid-search has been used to find the optimal parameters, as described in Appendix B.1. We have considered 5 possible values for each parameter (the penalization term C, the kernel smoothing parameter γ, and the bandwidth ε), as well as one kernel form (Guassian) for a total of 125 combinations (1x5 3 ). As shown in Table B2 , the selected values are in the interior of the provided intervals. Furthermore, as shown in Tables 2 and B4 , the insample and out-of-sample MSE are close to each other, thus reducing any concern about overfitting, and confirming that the algorithms have been properly calibrated. 
The third method we have used in our analysis is Boosting (also called Boosted Regression), using the gradient boosting algorithm of Friedman et al. (2000) . Gradient boosting is an ensemble method, which builds a classifier out of a large number of smaller classifiers. It works by combining many regression trees together. The basic idea behind regression trees is to split a sample sequentially based on a single covariate at a time exceeding a given threshold. The predicted outcome for each observation is then the average outcome in the final split. Regression trees are particularly effective when there is a large set of potential inputs, and only a few of them are actually strong predictors (Athey and Imbens, 2019).
As explained in Athey and Imbens (2019), boosting algorithms based on regression trees with one split can approximate any function that is additive in the features. Similarly, using regression trees with two splits allows for general second order effects, and so on. We consider trees with up to 6 splits, allowing up to 6-way interactions between variables. Thus, we model the function in a very flexible way. Boosting has been found to have superior performances than a number of other machine learning algorithms in many simulations (Bauer et al. 1999; Friedman et al. 2000) and has been used by Chalfin et al. (2016) in their work on predicting police hiring.
Boosting has been implemented in Stata using the command boost (Schonlau 2005) . A grid-search has been used to find the optimal parameters, as described in Appendix B.1. In addition, we have used only a random subset of the train set (80% of it) to build the tree at each iteration. This technique -bagging -is used to reduce the variance of the final prediction without influencing the bias. We have also imposed a normal distribution for the error term and a shrinkage term (determining the contribution to each tree to the final selection) of 1. As shown in Table B3 , we have considered trees with up to 6 tree splits, and up to 100 iterations (i.e., number of trees), for a total of 600 combinations (6*100). The selected values are in the interior of the provided intervals. Furthermore, as shown in Tables 2 and B4 , the in-sample and out-of-sample MSE are close to each other, thus reducing any concern about overfitting, and confirming that the algorithms have been properly calibrated. 