metadata: authors: 0: author: Mckenzie,David J.
1: author: Sansone,Dario
count: Nigeria
docna: 0: docna: Man vs. machine in predicting successful
            entrepreneurs : evidence from a business plan competition in Nigeria
docty: Policy Research Working Paper
subtopic: Private Sector Economics,Private Sector Development Law,Marketing,Labor Markets,Gender and Development,Educational Sciences,Educational Populations,Educational Policy and Planning - Textbook,Education for Development (superceded),Education For All
teratopic: Law and Development,Private Sector Development,Social Protections and Labor,Education,Gender
abstracts: cdata!: This paper compares the relative
            performance of man and machine in being able to predict
            outcomes for entrants in a business plan competition in
            Nigeria. The first human predictions are business plan
            scores from judges, and the second are simple ad hoc
            prediction models used by researchers. The paper compares
            these (out-of-sample) performances with those of three
            machine learning approaches. The results show that (i)
            business plan scores from judges are uncorrelated with
            business survival, employment, sales, or profits three years
            later; (ii) a few key characteristics of entrepreneurs such
            as gender, age, ability, and business sector do have some
            predictive power for future outcomes; (iii) modern machine
            learning methods do not offer noticeable improvements; (iv)
            the overall predictive power of all approaches is very low,
            highlighting the fundamental difficulty of picking winners;
            and (v) the models do twice as well as random selection in
            identifying firms in the top tail of performance.
geo_reg_key: 517191,1927320
repnme: repnme: Man vs. machine in predicting successful
            entrepreneurs : evidence from a business plan competition in Nigeria
display_title: Man vs. machine in predicting
            successful entrepreneurs : evidence from a business plan
            competition in Nigeria
listing_relative_url: /research/2017/12/29265869/man-vs-machine-predicting-successful-entrepreneurs-evidence-business-plan-competition-nigeria
topicv3: Private Sector Development m1326060 1921,Textbooks m1326802 2129,Education m1327048 470,Gender m1327621 615,Entrepreneurship m1582903 4424
publishtoextweb_dt: 2017-12-12T00:00:00Z
historic_topic: Law and Development,Private Sector Development,Social Protections and Labor,Education,Gender
no_of_pages: 66
author_keys: 0: author_key: 437864
1: author_key: 1245648
model_extractions: WPS8271 Policy Research Working Paper 8271 Man vs. Machine in Predicting Successful Entrepreneurs Evidence from a Business Plan Competition in Nigeria _David McKenzie Dario Sansone_ Development Research Group Finance and Private Sector Development Team December 2017
Policy Research Working Paper 8271

# **Abstract**

This paper compares the relative performance of man and machine in being able to predict outcomes for entrants in a business plan competition in Nigeria. The first human predictions are business plan scores from judges, and the second are simple ad hoc prediction models used by researchers. The paper compares these (out-of-sample) performances with those of three machine learning approaches. The results show that (i) business plan scores from judges are uncorrelated with business survival, employment, sales, or profits three years later; (ii) a few key characteristics of entrepreneurs such as gender, age, ability, and business sector do have some predictive power for future outcomes; (iii) modern machine learning methods do not offer noticeable improvements; (iv) the overall predictive power of all approaches is very low, highlighting the fundamental difficulty of picking winners; and (v) the models do twice as well as random selection in identifying firms in the top tail of performance.

This paper is a product of the Finance and Private Sector Development Team, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at dmckenzie@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
**Man vs. Machine in Predicting Successful Entrepreneurs: Evidence from a Business Plan Competition in Nigeria**<sup>#</sup>

David McKenzie<sup>1</sup> Dario Sansone<sup>2</sup>

**Keywords** : entrepreneurship, machine learning, business plans, Nigeria

**JEL** : C53, L26, M13

> # We are grateful to Eric Auerbach, Keisuke Hirano, William Maloney, Givi Melkadze, Allison Stashko and participants in seminars at Georgetown University, The World Bank and UIUC for their helpful comments. Funding from the Strategic Research Program (SRP) of the World Bank is gratefully acknowledged. The usual caveats apply.

> 1 Development Research Group, The World Bank Group. E-mail: dmckenzie@worldbank.org

> 2 Georgetown University. E-mail: ds1289@georgetown.edu
# **1. Introduction**

Millions of small businesses are started every year in developing countries. However, more than a quarter of these die within their first year (McKenzie and Paffhausen 2017), while only a small subset of firms grows rapidly, creating disproportionate value in terms of employment and incomes (Olafsen and Cook 2016). The ability to identify ex ante which firms will succeed is of key interest to investors seeking to maximize returns, determining the extent to which capital can be allocated to the highest return projects. Being able to identify these high growth potential firms is also important for governments seeking to target programs to these firms (OECD 2010), and is of interest to researchers seeking to characterize what makes a successful entrepreneur. Moreover, if the characteristics that are predictive of high growth are malleable, this can also spur policy efforts to attempt to change these in individuals without these attributes.

Business plan competitions are increasingly used in both developed and developing countries to attempt to spur high growth entrepreneurship. These competitions typically attract contestants with growth prospects that are much higher than the average firm in the economy. But then the key question is whether one can predict which firms from among these applicants have better growth prospects. We use data from applicants to Nigeria’s YouWiN! program, the world’s largest business plan competition (Kopf 2015), to answer this question. We compare the relative performance of three different methods for identifying which firms will be most successful: the standard approach of relying on expert judges, simple ad hoc prediction models used by researchers that employ a small number of variables collected in a baseline survey, and modern machine learning (ML) methods that consider 393 possible predictors along with potential interactions among these.

Business plan competitions typically rely on expert judges to score proposals, with higher scores given to those businesses judges view as having higher likelihoods of success. This approach has the advantage of using context-specific knowledge and enabling holistic judgement of business plans. However, using judges to score proposals can be costly and time-consuming, and there are also concerns that human evaluations can sometimes lead to discriminatory decisions (Blanchflower et al. 2003), or be driven by overconfidence (Zacharakis and Shepherd 2001). One alternative may then be to attempt to predict which businesses will succeed based on simple regression models that include survey variables like gender, age, and ability suggested by theory and researcher judgement (Fafchamps and Woodruff 2017; McKenzie 2015). However, the selection of which variables to include in these models is ad hoc and, since it reflects researcher opinion, may omit important determinants of future success.

Machine learning techniques offer an alternative approach that is less reliant on human judgement, and are designed specifically for out-of-sample prediction (Kleinberg et al. 2015). They offer the potential for better 2
prediction performance by using high-dimensional data reduction methods to identify patterns that might be too subtle to be detected by human observations (Luca et al. 2016). This could then offer a cheaper and more effective alternative to identifying which businesses will succeed.

Using data on more than 1,000 winners and more than 1,000 non-winners, we find that the business plan scores from judges do not significantly predict survival, or employment, sales and profits outcomes three years after entry. This is not due to the scores all being similar to one another, nor to the outcomes being similar for all firms. For example, among the non-winners, a firm at the 90<sup>th</sup> percentile has 13 times the employment and 2.4 times the profits of a firm at the 10<sup>th</sup> percentile. Rather, conditional on getting through the first phase of the competition, the scores of judges do not predict which firms will be more successful.

We then use several ad hoc prediction models. One set of these models are simple univariate prediction models using variables like gender, age, education, and ability that the literature has suggested might be related to firm performance. A second set of models comes from a simple logit based on this literature, used by McKenzie (2015), and from a similar model used by Fafchamps and Woodruff (2017). We do find that some observable characteristics of entrepreneurs do help predict future success, with older males with high ability doing better. Nevertheless, the overall accuracy of these models is still low.

These results are then contrasted with out-of-sample predictions from three modern ML approaches - PostLASSO, Support Vector Machines, and Boosted Regression – that use data reduction techniques and highly nonlinear functions to make predictions starting with 393 possible predictors or more. These methods may end up choosing interactions of variables and survey responses that would be unlikely to be used by human experts. However, we find that the overall accuracy of these models is similar to that of the simpler ad hoc models, and thus also low. In fact, the score on a 12 question Raven test is found to predict performance just as well as human experts or machines.

Although we find that machine learning is unable to beat simple predictors or simple models from economists when it comes to predicting average performance, we do find some role for machine learning, and sometimes judges, in identifying the top tail of performance. This suggests some potential for investors or governments attempting to identify high-growth firms. Nevertheless, even our best models are only able to ex ante identify 22 percent of the firms that will end up in the top decile of the outcome distribution.

Taken together, these results point to the fundamental difficulty of identifying in advance which entrepreneurs will be more successful. This is a feature of the fundamental riskiness inherent in entrepreneurship (Hall and Woodward 2010), and it is consistent with the inability of professional investors to know in advance whether a technology or venture will succeed, conditional on it passing some initial screening (Kerr et al. 2014; Hall and Woodward 2010).

3
This paper builds on two existing literatures. The first is a literature on predicting which firms will succeed. Most of the existing literature looks at start-ups and venture-backed firms in developed countries, and while some of these studies find that judges’ scores have some predictive power (e.g. Astebro and Elhedhli 2006; Scott et al. 2015), they also point to the immense difficulty of identifying who will be more successful (Kerr et al. 2014; Nanda 2016). Data analysts have also started to predict which new tech firms will be successful by looking at which sub-sectors have attracted the highest investments by venture capitalists and which companies have raised funds from the top-performing investment funds in the world (Ricadela 2009; Reddy 2017). There has been much less study of this issue in developing countries (Nikolova et al. 2011). An important exception is Fafchamps and Woodruff (2017), who compare the performance of judges’ evaluations to survey-based measures, and find that both have some (independent) predictive power among business plan contestants in Ghana. We build on their work with a much larger sample, and through the incorporation of machine learning.

Secondly, we contribute to a growing literature that uses machine learning in economics (Mullainathan and Spiess 2017). Kleinberg et al. (2015) note that this method may be particularly useful for a number of policy-relevant issues that are prediction problems, rather than causal inference questions. Economists have started applying these methods to a number of different settings such as predicting denial in home mortgage applications (Varian 2014), quality of police and teachers (Chalfin et al. 2016), poverty from satellite data (Jean et al. 2016), conflicts in developing countries (Celiku and Kraay 2017), if defendants will commit crimes when released on bail (Kleinberg et al. 2017), and college dropout (Aulck et al. 2016). Our work shows the limits of this approach when it comes to predicting entrepreneurial success, and that machine learning need not necessarily improve on the performance of simpler models that have been more commonly used by economists.

# **2. Context and Data**

Business plan competitions are a common approach for attempting to attract and select entrepreneurs with promising business ideas, and have been increasingly used in developing countries as part of government efforts to support entrepreneurship. The typical competition attracts applicants with the promise of funding, training or mentoring, and/or publicity for the winners. Applications are then usually subjected to an initial screening, with the most promising plans then being scored and prizes awarded to the top-scoring entries.

# **2.1 The YouWiN! Competition**

Our context is the largest of these competitions, the _Youth Enterprise with Innovation in Nigeria_ (YouWiN!) competition. This competition was open to all Nigerians aged 40 or younger, who could apply to create a new firm or expand an existing one. We work with the first year of the program, which had a closing date 4
for initial applications of November 25, 2011. The competition attracted 23,844 applications. A first stage basic scoring selected the top 6,000 applicants, who were offered a 4-day business plan training course. Those who attended the course were then asked to submit a detailed business plan, with 4,517 proposals received. These proposals were then scored by judges - in a process described below - with the highest scores used to select 2,400 semi-finalists. 1,200 among those were then chosen as winners and received grants averaging almost US$50,000 each. 480 of these winners were chosen on the basis of having the top scores nationwide or within their region, while 720 winners were randomly chosen from the remaining semi-finalists.

McKenzie (2017) uses this random allocation to measure the impact of winning this competition, and provides more details on this competition and selection process. His analysis found that being randomly selected as a competition winner resulted in greater firm entry from those with new ideas, higher survival of those with existing businesses, and higher sales, profits, and employment in their firms. It shows that the competition was able to attract firms that, on average, had high growth prospects. Our goal in this paper is to use this setting to see whether the judge scores, simple heuristic models, or machine learning approaches can identify ex ante which firms from among these applicants will be most likely to survive and grow.

# **2.2 Data**

Our sample consists of 2,506 firms that were tracked over time by McKenzie (2017). These comprise 475 non-experimental winners (the national and regional winners), 729 experimental winners, 1,103 semifinalists that are in the control group, and 199 firms that submitted business plans but did not score high enough for the semi-finals.<sup>3</sup> For each of these firms we have data from their initial online application, from the submission of their business plans, the score provided by judges on their business plans, short-term follow-up survey data on some ability and personality measures that are assumed to be stable over time, and then data from a three-year follow-up survey on key outcomes. We discuss each in turn, and provide a full description of the different variables in Appendix A.1.

The application form required individuals to provide personal background such as gender, age, education, and location, along with written answers to questions like “why do you want to be an entrepreneur?” and “how did you get your business idea?” We construct several simple measures of the quality of these written responses, such as the length of answer required, whether the application is written in all capital letters, and

> 3  McKenzie (2017) dropped 5 non-experimental winners who were disqualified by the competition, and includes 9 experimental winners who were disqualified. We drop 9 control group firms that were non-randomly chosen as winners. In addition, McKenzie (2017) attempted to survey 624 firms that were not selected for the business plan training. Since these firms did not submit business plans, they do not have business plan scores and are thus excluded from our analysis.

5
whether they claim to face no competition. We also use administrative data on the timing of when applications were submitted relative to the deadline.

The business plan submission contained more detailed information, including a short baseline survey instrument. This collected additional information about the background of the entrepreneur, including marital status, family composition, languages spoken, whether they had received specific types of business training, employment history, time spent abroad, and ownership of different household assets. It elicited risk preferences, asked about motivations for wanting to operate a business, and measured self-confidence in the ability to carry out different entrepreneurial tasks. More detailed information about the business they wanted to operate included the name of the business, business sector, formal status, access to finance, taxes paid, challenges facing the business, and projected outcomes for turnover and employment.

The initial application and business plan submission were all done online. While they contain a rich set of data on these applicants, there were some measures which needed to be collected face-to-face with explanations, and so were left for follow-up surveys. Two ability measures were collected in the follow-up surveys: digit-span recall, and a Raven progressive matrices test of abstract reasoning. In addition, the grit measure of Duckworth et al. (2007) was also collected during these short-term follow-ups. We assume that these measures are stable over one to two years, and therefore include them as potential characteristics that could be used to predict future outcomes had they been collected at the time of application.

# **2.3 Measuring Success**

McKenzie (2017) tracked these firms in three annual follow-up surveys, and then in a five-year follow-up during a period of recession in Nigeria. We use data from the three-year follow-up survey, fielded between September 2014 and February 2015, to measure business success in this paper. This survey had a high response rate, with data on operating status and employment available for 2,140 of the 2,506 firms in our sample of interest (85.4 percent). We believe this is a more useful test than the five-year follow-up, which, in addition to a lower response rate, occurs in an atypical period in which Nigeria suffered its worst economic performance in thirty years, resulting in many firms struggling. Moreover, three years is the time horizon over which business plans were prepared (see next section).

We focus on four metrics of success: business operation and survival, total employment, sales, and profits. Appendix A.1 describes how these are measured. 93.5 percent of the competition winners were operating businesses three years after applying, compared to 58.4 percent of the control group and other non-winners. Figure 1 then plots the distributions of employment, sales, and profits outcomes for the winners and nonwinners. We see substantial dispersion in the outcomes of these firms three years after they applied for the program. For example, among winning firms in operation, a firm at the 90<sup>th</sup> percentile has 20 employees, 6
five times the level of a firm at the 10<sup>th</sup> percentile of employment; and a firm at the 90<sup>th</sup> percentile of profits earns 500,000 naira per month (USD $2,730), more than four times the level of a firm at the median, and 100 times that of a firm at the 10<sup>th</sup> percentile. Likewise, while average levels of these outcomes are lower for non-winners, there is also a long right tail for this group, with some firms doing substantially better than others. Given how much variability there is in these outcomes, it is therefore of interest to examine whether the more successful of these firms could have been identified in advance.

# **3. Human Approaches to Predicting Business Success**

The standard approach to choosing businesses to support is to rely on the human judgement of experts. We consider prediction by two types of experts. The first is the judges in the business plan competition, while the second is the predictions of economists based on their expert judgements of what variables should predict business success.

# **3.1 Using Judges to Evaluate Business Plans**

The business plans prepared by applicants were lengthy documents, averaging 13,000 words of narrative coupled with spreadsheets of financial projections. They contained a detailed description of the proposed business, market and industry analysis, production plans, operations and logistic details, customer analysis, financial analysis, competitive environment, and other details, along with financing plans, discussion of what the grant would be used for, discussion of the risks, threats, and opportunities, and sales and growth projections for the next three years.

Scoring for the competition was overseen by an independent project unit supported by DFID and the World Bank, with significant effort taken to ensure that the scoring was impartial and rigorous. Scoring was done by the Enterprise Development Center (EDC), a sister unit of the Lagos Business School within the PanAtlantic University in Lagos; and by the Nigerian office of PriceWaterhouseCoopers (PwC). Each selected 10 judges, who were experts in business and entrepreneurship and knowledgeable about local conditions. These judges included business owners and alumni of Lagos Business School, as well as experts involved in working directly with successful businesses in Nigeria. These judges were given training, which included marking sample plans and feedback. Markers also had access to a forum where they could seek advice from their peers and resolve issues.

In order to ensure impartiality, and to be able to score such a large number of applications, applicants did not make in-person pitches to judges, but were scored only on the basis of their business plan. These plans were anonymized. Judges were given a set template for scoring, which assigned marks on 10 different criteria covering the management skills and background of the owner, the business idea and market, 7
financial sustainability and viability, job creation potential, and a capstone score to allow for their overall assessment of the chance of business success (Appendix A.1). Importantly these judges were asked to assess which firms were most likely to succeed, not which firms would have the largest treatment effects from receiving a grant from the competition. A typical plan took 30 to 45 minutes to mark. A random sample of 204 of the business plans were then re-scored by an independent team from Plymouth Business School and any significant disparities reviewed. The mean score in this sample was within 0.4 points out of 100 of the mean from the original judges, with this difference not statistically significant.

Figure 2 plots the distribution of business plan scores according to their competition outcome. We see that that distributions of scores are wide for both the winners and the non-winners. The winning scores range from 30 to 91 out of 100, with a mean of 56 and standard deviation of 12. The scores of the non-winners in our sample range from 2 to 73 out of 100, with a mean of 51 and a standard deviation of 11. This shows that the judges did view firms as substantially different from one another in terms of their potential for business success. Moreover, we have seen that the firms also had wide dispersion in subsequent outcomes. This then raises the question of whether the judges were able to identify in advance which firms were more likely to succeed.

# **3.2 Do Judges’ Scores Predict Business Success?**

Since the winning firms received substantial business grants, which were shown in McKenzie (2017) to have a causal impact on firm outcomes, we separate all our analysis by whether or not the firm was a competition winner. We are then interested in seeing whether the winners scored more highly by the judges did better than winners with lower scores, and likewise whether the non-winners with higher business plan scores did better than those with lower business plan scores.

Figure 3 then plots the relationship between our four key outcomes (business operation and survival, sales, profits, employment) and the judges’ scores for the non-winners. We see that the likelihood of operating a firm three years later is actually slightly lower for those in the upper quintiles of the business plan score. We then fit both unconditional lowess (which codes employment, profits, and sales, as zero for firms not operating) and conditional lowess (which only considers firms in operation) and plot these lines along with the scatterplots for sales, profits and employment. It is apparent that the business plan scores explain very little of the variation in these outcomes, with the fitted lines showing, if anything, poorer results for those with higher scores.

The first column of Table 1 (survival), Table 2 (employment), Table 3 (profits), and Table A1 (sales) then tests whether the relationship between business plan scores and firm outcomes is statistically significant for these non-winners by estimating a logit (survival) or least squares regression (other outcomes) of the 8
outcome on the business plan score and an indicator for whether the application was for an existing firm compared to a new firm. The regressions are unconditional, coding outcomes as zero for non-operating firms. For all four outcomes, we see the business score has a small, negative, and not statistically significant association. The R<sup>2</sup> are below 0.04, with even this small amount of variation explained coming largely from the dummy for existing firm status. Judges’ scores therefore do not predict which of these non-winners are more likely to succeed.

Figure 4 then plots the associations between business outcomes and the judges’ scores for the winners. We again see the business plan scores explain very little of the variation in these outcomes, with the fitted lines showing a fairly flat relationship for profits and sales, and a slightly upward sloping relationship for employment. Column 4 of Tables 1-3 and A1 then shows the fitted logit and regression relationship, after controlling for whether the application was for an existing firm. The amount of grants the winners received varied among firms, with an average of US$48,991 and standard deviation of $17,546. There is a significant positive relationship between business score and amount received, and more capital has a direct impact on firm outcomes. Therefore, column 5 of Tables 1-3 and A1 also controls for the logarithm of the grant awarded. We see the judges’ score has a small and statistically insignificant impact on survival, profits, and sales, with or without this control for grant amount. In contrast, we see in Table 2 that the judges’ scores are significantly correlated with employment three years later in these firms (p=0.004), although this relationship weakens after controlling for the grant amount (p=0.110). Moreover, the share of variation in employment accounted for by these judges’ scores is still very low, with an R<sup>2</sup> of 0.02 in column 4 of Table 2.

This analysis shows that the overall scores of judges are uncorrelated with business success for nonwinners, and at most weakly correlated with future employment outcomes, but not other outcomes, for winners. As an additional robustness check, Table A2 shows that this result continues to hold when we condition on judge fixed effects.

The overall score aggregates multiple scores on 10 subcomponents, and so it is of interest to examine whether this aggregation masks predictive ability of some subcomponent of the score. In Appendix Table A3 we test whether the ten subcomponents of the business plan score are jointly associated with future outcomes, as well as examining individual subcomponents. For the non-winners, we cannot reject that the ten sub-scores are jointly unrelated to survival, sales, or profits, but we can do so for employment (p=0.040). We also find that the score out of 10 for “ability to manage” is positively and significantly related to all four outcomes - with p-values of 0.001, 0.002, 0.008 and 0.017 respectively - and continues to be so even after applying a Bonferroni or step-down correction for multiple testing. This effect mainly appears to be 9
coming through the extensive margin of predicting whether or not the individual will have a firm in operation. The employment sub-score is also weakly predictive of employment (p=0.056).

For the winners, this “ability to manage” score is not significant for any outcome, and we cannot reject that the ten sub-scores are jointly unrelated to survival, sales, or profits. The employment score does significantly predict employment among the winners (p=0.002), and we can weakly reject that the ten subscores are jointly unrelated to the employment outcome (p=0.056). As a reference, a one standard deviation (6.0) increase in this employment sub-score is associated with the winning firm having 1.1 more workers three years later. Notably, the capstone score - which is intended to reflect the judge’s overall assessment of promise not picked up in the other scores - has no significant association with future business success for either winners or non-winners.

# **3.3. Using Human Expert Judgements to Predict Business Success**

An alternative human approach to predicting business success is for economists to use their expert judgement to choose characteristics measured in the baseline survey (or that are measured later but assumed to be time-invariant) that they think may predict outcomes, and then to fit a simple logit or OLS regression model as a function of these characteristics.

This approach was carried out in Appendix 18 in the working paper of McKenzie (2015), but dropped for space reasons in the published version (McKenzie 2017). It uses principal components and aggregate index measures for data reduction: a household wealth measure is constructed as the first principal component of 20 durable assets; an ability measure is constructed as the first principal component of the Raven test (itself an aggregate of responses on 12 questions) and digit-span recall scores; and a grit measure is an average of responses on 12 questions proposed by Duckworth et al. (2007). In addition to this, gender, age, attitude towards risk, a dummy for having graduate education, whether the applicant has worked abroad, indicators for the six geographic regions, and indicators for the three most common industries (agricultural business, manufacturing business, IT business) were selected as variables that the literature and an understanding of the business setting suggested might help predict future performance. No model selection criteria were used to select these variables: they were just proposed as an ad hoc judgement of what might matter for predicting business outcomes.

A similar approach is used by Fafchamps and Woodruff (2017) for a business plan competition they ran in Ghana. They propose a set of core measures that they think are likely to be viewed as potentially important: the ability of the owner, past borrowing activity, and management practices, along with controls for gender, age, sector, and location. They then add attitudinal questions based on the reason for going into business, and on optimism and control over life. Again, no model selection criteria were used to select these variables.

10
We examine the extent to which these proposed variables coming from baseline survey measures are associated with future business outcomes in the remaining columns of Tables 1-3 and A1. Columns 2 and 6 use the variables selected in McKenzie (2015), while Columns 3 and 7 attempt to match as closely as possible the variables used in Fafchamps and Woodruff (2017). Appendix A.1 describes how these variables were constructed in detail. These regressions allow us to examine whether the proposed models are useful for predicting business outcomes within sample – in the next section we will examine out-ofsample performance to allow comparison with machine learning methods. We also include the business plan score in these regressions so that we can test whether these survey measures add value beyond that contained in the judges’ scores, and conversely, whether the judges’ scores now contain information once we have conditioned on other features of the applicants.

Consider predicting outcomes for the non-winners. We can strongly reject that the McKenzie (2015) set of regressors, and the Fafchamps and Woodruff (2017) set of regressors, are not associated with each of our four business outcomes (p<0.001). This greater performance relative to the judges comes largely from a few characteristics of the owner. First, consistent with a broad body of literature (e.g. Boden and Nucci 2000; Bruhn 2009; Robb and Watson 2012; McKenzie and Paffhausen 2017), female applicants who do not win are less likely to start or continue businesses than men, and operate firms that are smaller in size and less profitable.

Second, while the business plan competition was designed for youth, it maintained an expansive definition of youth which ranged from age 18 to 40, with a median age of 30 among our non-winners. We see that older applicants were more likely to be running firms three years later among the non-winners, and to run firms of larger size, sales, and profitability. This is consistent with a return to work and life experience, and with evidence from other developing countries that businesses have the lowest failure rates when the owners are middle-aged (McKenzie and Paffhausen 2017). In contrast to pitching situations where judges see the applicants, the anonymized business plans used for scoring meant that judges did not observe gender or age, and therefore did not consider it in their assessments (and indeed, concerns about sex- or agediscrimination would occur if they did directly consider it).

Third, higher ability of the owner is associated with better performance in all four outcomes for the nonwinners. Both the McKenzie (2015) measure which is the first principal component of Raven test and digit span, and the Fafchamps and Woodruff (2017) measure which is the first principal component of Raven, digit span, and education, are significant predictors. Examining the sub-components, the Raven test score is the most important contributor here. We then also see some role for business sector in predicting some outcomes: IT firms tend to hire fewer workers, while retail firms have higher sales.

11
These models based on few key variables selected by economists do better than the judges in predicting outcomes for the non-winners. The business plan scores remain insignificant, even conditioning on the variables in these models. However, these models still only explain a small fraction of the variation in our key outcomes: the R<sup>2</sup> are 0.06-0.07 for employment, 0.12-0.15 for sales, and 0.10-0.14 for profits.<sup>4</sup> Moreover, much of the predictive ability comes from the extensive margin of whether firms are operating or not. Appendix Table A4 shows that these R<sup>2</sup> are even lower when we consider only the ability of these predictors to explain the variation in outcomes among firms in operation.

Since almost all of the winners are operating firms after three years (93.5%), there is not much variation in this outcome. We cannot reject that the McKenzie (2015) set of variables are jointly orthogonal to this outcome, while none of the variables in Fafchamps and Woodruff (2017) are individually significant. We do find some predictive power for employment, sales, and profits, but the R<sup>2</sup> are only 0.10, 0.04 and 0.050.06 respectively. Several of the attributes that helped predict outcomes for non-winners are much less predictive of outcomes for winners: gender, while negative, is only significant for profitability; ability is never significant, and has negative coefficients in all but one specification; and being older predicts higher employment, but not higher sales or profits. Sector continues to matter, with IT firms hiring fewer workers, and retail firms having higher profits and sales. We do see that the business plan score of judges continues to be a significant predictor of employment at the 10 percent level, even after controlling for the variables in these models.

It is also notable from Tables 1-3 and A1 that some characteristics that were expected to help distinguish among business owners did not have significant predictive power. A first example is education, which is negatively associated with employment among winners, and otherwise not significant. One possible reason for not seeing the expected positive return to education (e.g. Michelacci and Schivardi 2016; Queiro 2016) is that the earlier stages of the competition (including the need to apply online) appear to have selected heavily on education. Only 5.5 percent of Nigerian youth have university education, but 52 percent of the applicants, and 64 percent of those invited to business plan training had this level. Likewise, applicants who got through to the business plan submission stage are wealthier, and more likely to have overseas work experience, than the average Nigerian youth, yet conditional on getting to the business plan submission stage, we generally do not find these variables to be significant predictors of business outcomes.<sup>5</sup> The

> 4  By way of comparison, Fafchamps and Woodruff (2017) report an R2 of 0.14 for profits, 0.23 for sales, and 0.46 for employment. However, their firms had been in operation for an average of 9 years, and do not include start-ups. Moreover, they are able to include baseline sales and profits among their predictors, which exhibit some persistence and which we do not have for start-up firms.

> 5  Wealthier individuals have higher profits as winners (at the 10 percent significance level), while we find overseas work experience has a negative association with profits for winners (at the 10 percent level), while these variables are not significant for other outcomes for winners, or for any outcome for non-winners.

12
literature has also suggested risk preferences determine selection into entrepreneurship and the success of these businesses (e.g. van Praag and Cramer 2001; Hvide and Panos 2014; Skriabikova et al. 2014), yet we see no significant impact of our risk preference measure. Nor do we see any predictive power for the grit measure of Duckworth et al. (2007).

In addition to the models discussed in this and the previous section, we examine in Appendix A.2 whether there are differences in outcomes according to three other characteristics suggested by recent literature: necessity entrepreneurship, whether the firm is named after the owner, and registration time.

# **4. Machine Learning Approaches to Predicting Business Success**

The human prediction approaches rely on a small set of variables, and on methods like principal components and simple aggregation for data reduction. Yet, in practice, the application and baseline data contain a much richer set of possible predictors. Many of these come from just considering a lot more variables, but the number of predictors becomes even larger once we consider how responses to certain questions should be coded. We discuss a few specific examples, and then the total number of potential predictors we consider.

A first example comes from questions with Likert scales. For example, applicants were asked a set of selfefficacy questions about their confidence in carrying out nine different business tasks, such as “find and hire good employees to expand your business”. They had six possible responses to their confidence on this task: not at all confident, somewhat confident, quite confident, very confident, don’t know, or not applicable. Rather than trying to code these as binary variables and then aggregate, an alternative is to code each task and response as a separate dummy variable (e.g. being not at all confident that they can estimate accurately the costs of a new project). This yields 54 potential predictors.

A second example comes from the Raven test. Applicants were given 12 different puzzles to solve, and had a choice from among 8 options each time, with one of these options correct. The standard approach, used in the models estimated in the previous section, is to aggregate these into a single score out of 12. Yet some questions may be potentially more useful in predicting business success than others, and different types of wrong answers may reflect different abstract reasoning deficiencies. For example, question 6 had 39 percent choose the correct answer, three other answers were chosen by 10 to 11 percent of individuals each, and the remaining four answers were all chosen by between 5 and 8 percent each. Coding each question and answer as a separate dummy variable then yields 96 potential predictors.

A third example comes from the business sector that the applicant proposes for their new business venture, or has for their existing business. They were asked to select from 32 different business sector categories. In the ad hoc model of McKenzie (2015), five of these were aggregated together to form a manufacturing 13
dummy, and two other sectors were also chosen, while the rest were lumped together as the base comparison group. In contrast, we consider all 32 categories as possible predictors here.

The net result of this, once we drop variables which are perfectly collinear, is that there are 393 possible predictors that we can consider. We include here the initial application score and the ten different subcomponent scores from the judges, in addition to the survey data. Moreover, once we also start considering interaction terms between some of these predictors, the number of variables to consider can easily exceed the number of firms for which we have data. It is then not possible to include all of these variables in a standard OLS regression, or logit model.

Machine learning provides tools for using these high-dimensional data. Adding too many variables to a model may lead to over-fitting, i.e. to estimate a model which fits a training sample very well, but may fail to properly work with new samples. For instance, if the true relation between y and x is quadratic, a linear model would be an under-fit (high bias), while estimating a 4<sup>th</sup> degree polynomial would lead to over-fitting (high variance). There are two ways to address over-fitting: one is through reducing the number of variables, the other is regularization. The latter keeps all the variables, but reduces the magnitude of the coefficients on each. This works well when there are several variables which contribute slightly to predicting y. We use three different machine learning approaches which use one or both of these methods for data reduction, and describe each below.

# **4.1 The Machine Learning Algorithms**

We use three different machine learning approaches. We briefly summarize each here, and provide more details in Appendix A.3.

The first method is **LASSO** (Least Absolute Shrinkage and Selection Operator). This method adds a penalization term to the OLS objective function: Where _k_ is the number of potential predictors (potentially larger than the number of observations, _n_ ), and λ is the regularization (or penalization) term. In practice, this selects variables with high predictive power, and shrinks their coefficients, while constraining all other variables to have coefficients of zero. Since this introduces a bias in the coefficients, in a second step, an OLS regression is run using only the predictors selected by LASSO, with this known as Post-LASSO. LASSO is one of the first tools taught in machine learning courses (Hastie et al. 2009). A key assumption is that the data generating process is sparse: only a 14
small subset of variables is assumed to have high predictive power, which may not be the case in some economic applications (Giannone et al. 2017).

In practice, we start with 393 possible prediction variables, and run LASSO on this to select predictors. In our setting, this typically chooses around 50 variables. We then generate all two-way interactions among these variables, and add in this set of interactions, along with quadratic and cubic terms for all non-binary variables to generate a list of around 1,200 possible predictors. We then use this larger set of variables for the LASSO estimation. We choose the penalization term λ via cross-validation from a grid search.

The second method is **Support Vector Machines (SVM)** for our business operation outcome, and the extension of **Support Vector Regression (SVR)** for our continuous outcomes of employment, sales, and profits. SVM aims to classify the data into two groups (in our case, into firms that will be operating and those that will not) by means of a hyperplane in high-dimensional space. It can be written as a modified penalized logistic regression that solves the objective function: There are four terms here that differ from that in the standard logistic regression, and which allow SVM greater flexibility in covering a wide range of functional forms and dealing with high-dimensional data. In addition to the penalization factors, C1 and ‖�‖�; there is a kernel _K_ which controls the smoothness of the fitted curve, and a “soft margin” that comes from the _max_ (0,.) component, which allows for some observations to be misclassified close to the separating margin. We consider a Gaussian kernel, and use cross-validation to choose the penalization term and the kernel smoothing parameter. SVM has been found to achieve better performance than a number of other machine learning algorithms in some applications (Maroco et al. 2011). Nevertheless, given the similarity in terms of objective function, in other cases performances have been found to be similar to those obtained with logistic regressions (Verplancke et al. 2008).

Support Vector Regression likewise deviates from OLS by using kernels to flexibly map the predictors into a hyperplane in high-dimensions, allowing a soft margin in which it is insensitive to errors, and a penalization term (see Appendix A.3). We consider the Gaussian kernel, and then have three terms (a penalization term, a kernel smoothing parameter, and a bandwidth) to select via cross-validation from a total of 2,197 possible values. We use all possible predictors. Since the kernels allow for flexibility, we do not consider interaction terms in these variables. One downside of SVM and SVR are that the parameters are difficult to interpret, and so we will not be able to say which variables matter most for determining the predictions made by this method.

15
The final method we consider is **Boosting** (also called Boosted Regression), using the gradient boosting algorithm of Friedman et al. (2000). Gradient boosting is an ensemble method, which builds a classifier out of a large number of smaller classifiers. Like random forests, it works by combining many regression trees together. Key differences are that boosting does this sequentially, where in each iteration, observations that were misclassified by the previous classifier are given larger weights; and that boosting typically uses shorter trees, including tree stumps (a single split of the data), aggregating over more of them than random forests.<sup>6</sup> We consider trees with up to 8 splits, and up to 150 iterations (i.e. trees) – thus allowing up to 8- way interactions between variables in a very flexible way. Performance is improved by using bagging, which uses only a random subset of 50% of the training data set to build the tree in each iteration, and we use cross-validation to choose the shrinkage parameter. Boosting have been found to have superior performances than a number of other machine learning algorithms in many simulations (Bauer et al. 1999; Friedman et al. 2000) and was used by Chalfin et al. (2016) in their work on predicting police hiring.

# **4.2 Estimation by Machine Learning**

We follow the recommendation of Ng (2016) and split the data into three sub-samples (separately for winners and non-winners). First, a training sample (60% of the data) is used to estimate the algorithm under each of the three methods. Then the tuning parameters (such as the shrinkage term in LASSO, or the penalization term, kernel smoothing parameter and bandwidth in SVR) are selected using a grid-search in the cross-validation sample (20% of the data). This reduces the risk of overfitting by estimating the model with the training data and measuring performance with the cross-validation sample. Finally, out-of-sample performance is reported using the test sample (the remaining 20% of the data). This last – less common - step is required since an extensive grid-search may still lead to overfitting the cross-validation sample (Ng 2016).

The main issue with this approach is that not all data are exploited to calibrate the model, and that, with relatively small samples, outliers may be overrepresented in one of the three samples. We therefore use a 5-fold approach. The data were divided into five data sets, and combined in all possible ways to create five different splits into training, cross-validation, and test samples. The out-of-sample performance results are then averaged over the five folds. This has the advantage of reducing variability and maximizing the sample by using each observation once for validation.

This process is time-consuming and computationally intensive. A separate model has to be fitted for each outcome (since different variables may predict survival than predict employment or sales or profits), and

> 6  Gradient boosting appears to perform slightly better than random forests in dimensions of 4,000 or fewer predictors, but requires more tuning (Zygmunt 2016).

16
for winners and non-winners. Given that almost all winners were operating firms, we only predict operation and survival for non-winners, giving seven outcomes to predict. Of our three machine learning approaches, LASSO was usually the fastest to implement, and only the penalization term λ needed to be chosen through cross-validation. It usually took 15-30 minutes to estimate for one outcome, although in some cases took up to 10 hours because of numerical issues. SVM and SVR took 1-2 hours for each fold (per outcome), since it was necessary to choose the penalization term _C1_ , the type of kernel, the degree of smoothness in the kernel, as well as the width of the band when looking at continuous outcome variables. Boosted regression was the most time-consuming. It required using cross-validation to select the possible length of the trees, the possible number of trees, the shrinking parameter to avoid overfitting, and the distribution of the error term. The running time was typically 5-6 hours for each fold (per outcome), but in some cases took up to 13 hours. In conclusion, given that we have 5 folds for each outcome variable and 2 samples (winners and non-winners), producing one of the final tables described in the next section took on average more than 70 hours (almost 3 days) of computing time. The actual time was much longer since working memory limitations forced us to launch one algorithm-fold at a time (around 2 weeks per table).

# **4.3 Assessing Accuracy and Comparing to Human Approaches**

We follow standard practice in using the mean squared error (MSE) as the main criteria to compare the accuracy of different models for our continuous outcomes (employment, sales, and profits). For the sake of completeness, we have also reported the square of the Pearson correlation coefficient, i.e. the correlation between the actual and fitted dependent variable. This is equivalent to the R<sup>2</sup> in linear least squares regressions.

In contrast, there are several different measures used in the literature when considering the goodness of fit for binary outcomes, such as business operation and survival in our case. The starting point is usually the matrix comparing predicted outcomes with actual ones: An overall measure of the goodness of fit is the accuracy rate, which is the proportion of predictions that are correct out of all observations: 17
Two additional criteria are also commonly used in the case of imbalanced data, i.e. when the number of positive values is much smaller or larger than the number of negative values: Both the LASSO and Boosted regression predict probabilities for binary outcomes.<sup>7</sup> We follow convention of predicting an outcome of one (survival) when the predicted probability is greater than or equal to 0.5, and zero otherwise. This is in line with the Bayes classifier (Hastie et al. 2009): the accuracy rate is maximized by assigning each observation to the most likely class, given its predicted probabilities.

Subrahmanian and Kumar (2017) note that the choice of criteria should depend on context and whether there are strong reasons to prefer the ability to detect true positives versus to avoid false negatives. In the context of predicting firm operation, we do not believe there are strong reasons to prefer one versus the other, and so use accuracy as our measure of performance. In section 5, we consider the ability of these methods to predict the upper tail of performance on profitability, in which case we also report the recall rate.

To compare the prediction success of our machine learning approaches to those of the judges and of the models of economists shown in Section 3, we also use 5-fold estimation to get out-of-sample predictive performance for these approaches. We use the same 60% training samples to fit the models, and then the 20% test sample to get predictions, MSE, and accuracy rates (and the cross-validation samples are not used). We fit the models shown in Tables 1-4, which predict outcomes as a function of the judges score, the McKenzie (2015) predictors, or the Fafchamps and Woodruff (2017) predictors. For robustness, we consider two other models based on the judges scores: using just the first-round application score, or using all ten sub-scores (as in Appendix Table A.2). We show the performance of the Fafchamps and Woodruff (2017) predictors with and without the business plan score included. Finally, we also compare to the performance of seven models each based on just a single predictor (gender, age, necessity firm, grit, digitspan, Raven test, registration time for the competition).

# **4.4 Results from Machine Learning and Human Predictions**

Table 4 examines the 5-fold out-of-sample performance of the different models in predicting business operation and survival for the non-winners three years after they applied to the competition. As a

> 7  In contrast, SVM classifies observations as 1s or 0s, and does not produce probabilities.

18
benchmark, the first row reports the accuracy rate (58.4%) of a mode which contains only a constant. The next three models consider the performance of the scores from judges – the initial application score (model 2), the business plan score (model 3), or the ten sub-scores from the business plan (model 4). All three are less accurate than just using the constant model, re-iterating the poor performance we saw in Table 1. Furthermore, the initial application scores provide higher accuracy than the business plan scores. Models 5 through 11 just use a single predictor, and most of these models perform better than the judges. The best performance here comes from the Raven test, which is able to correctly predict whether or not a business will be operating in 63.8% of the cases. Models 12 through 16 consider the ad hoc models of the human experts described in Section 3.3. Both the McKenzie (2015) and the Fafchamps and Woodruff (2017) models have relatively high performances, with 61.5% and 65.6% accuracy respectively. We see that the accuracy of the Fafchamps and Woodruff (2017) does not change when we also add the business plan score, and is similar whether we use probit or OLS instead of logit. The Fafchamps and Woodruff (2017) model provides a 12 percent improvement in accuracy compared to only using the business plan score of the judges, but it barely do 3% better than just using the Raven test score alone. Finally, models 17 through 19 are the machine learning models. Their accuracy rates range from 59.8% for boosting to 63.6% for SVM. These sophisticated modeling approaches based on 393 potential predictors, along with potential interactions and non-linearities, therefore do no better than the ad hoc models of human experts, or than just using the Raven test score alone, in terms of predicting business survival among the non-winners.

Tables 5 then shows the 5-fold out-of-sample MSE and square of the Pearson correlation coefficient for employment, sales, and profits, for both the non-winners and winners. Consider the non-winners. For employment, the Fafchamps and Woodruff (2017) model, the McKenzie (2015) model and age have the lowest MSE. Post-LASSO and SVR also reach similar performances. For sales, Boosting, SVR and the Fafchamps and Woodruff (2017) models perform best. Post-LASSO and the Raven test alone do almost as well, and beat the McKenzie (2015) model. The business plan scores of judges are among the worse. Nevertheless, the differences are small across models, with only an 8 percent reduction in MSE for sales obtained when moving from using the judge’s business plan scores to the lowest MSE model (that of Boosting). SVR and the Fafchamps and Woodruff (2017) models have also the lowest MSE for profits. The squared correlations are particularly low for employment, all below 5%, while they are 12% or below for sales and profits.

Among the winners, we do not see large differences in performance across the different methods, especially for sales and profits. The MSE of the judges, human expert models, and machine learning models are within 5 percent of one another for these two outcomes for winners, and the squared correlations are always below 3 percent. We see more difference in performance for the employment outcome. The best performance 19
comes from adding baseline employment (for existing firms) to the Fafchamps and Woodruff (2017) model. Otherwise, the post-LASSO model performs best, but the Boosting and SVR models worst. The judge scores have slightly superior performances than the other parsimonious models.

Comparing all of these sets of results leads to several conclusions. First, machine learning does not consistently lead to improved performance compared to human predictions in this context. In fact, simple models from human experts slightly outperform at least one out of three machine methods for every outcome considered. However, some of the machine learning predictions do yield more accurate predictions than using the judges scores for non-winners. Second, a simple measure of cognitive skills, the Raven test, performs at least as well on average as using judges, human expert models, or machine learning. Third, the scores from the initial applications almost always provide a lower MSE than the business plan scores, raising doubts about the cost-effectiveness of this added time and cost. Fourth, predicting outcomes for these businesses is difficult, with the out-of-sample performance of all models quite low, especially for the profits and losses of winners.

# **4.5 Does machine learning use different predictors from human experts?**

The machine learning models potentially select different predictors in each fold, and for each outcome. Nevertheless, it is of interest to see whether the machine learning algorithm ends up putting high weight on variables that would not typically be considered by experts when predicting business success. SVM and SVR use kernels over many variables and so are not easily interpretable in this regard.

Friedman (2001) and Schonlau (2005) note that for boosting regression, one can compute the influence of a variable. This depends on the number of times a variable is chosen across all iterations (trees) and its overall contribution to the log-likelihood function. Such values are then standardized to sum up to 100. We therefore look at the variables which have been selected at least once in the 30 fold-status-outcome estimations (5 folds, non-winners/winners, employment/sales/profits). It is interesting to note that, among the 393 predictors considered, 322 have been picked by the algorithm to construct a tree. However, around 90 of them have been selected only once, 60% of them have been selected 3 times or less, and only 25 variables have been used at least 10 times, i.e. in one-third of the algorithms.

Table A6 lists these 25 predictors along with the number of boosted regression they have been used in, and the 5-fold average influence for each status-outcome. The most striking result is that most of these regressors are similar to the ones selected by heuristic models. Indeed, the list includes demographic characteristics, wealth and income indicators, number of employees at baseline, and Raven tests components. The initial application and business plan scores are also among these top predictors. The most frequently selected variable is the number of workers with tertiary education, followed by registration time.

20
Given these results, it is not surprising that the machine learning algorithms do not produce substantial improvements over the other models considered: the inputs are very similar, and most of the boosted regressions select only 1 or 2 splits, implying low levels of interactions among these variables. In other words, the simple models of human experts already contain many of the most powerful predictors, and there seems not to be important nonlinearities in this context.

Nevertheless, the list also includes some additional variables which may be useful to improve predictions. Future expectations about employment and turnover are often selected. Experience abroad seems to be relevant as well. Family links and household composition can also play a role: the list includes the number of siblings, as well as the number of children. The length of the first answer on why they wanted to start a business in their application form is often chosen. Finally, two specific answers on the Raven test are chosen.

The same exercises can be replicated for LASSO. However, in this case the generated interaction terms were allowed to be different in each outcome-fold. As a result, LASSO considered around 800-1100 potentially different input variables at each iteration (see Appendix A.3.1 for additional details). Among these predictors, 320 variables were selected at least once, but only 7 of them were selected more than twice. Table A7 lists these predictors. All of them are interaction terms and higher order polynomials, making it harder to interpret them. Nevertheless, there are some overlaps with the variables selected in Table A6, e.g. the amount awarded at the YouWIN! competition. Quite interestingly, two incorrect answers from the Raven test are selected, thus suggesting that there might be important information in not just whether individuals get the right answer, but also in which wrong answer they choose.

# **5. Predicting High Performance**

Figure 1 showed substantial variation in outcomes for both winners and non-winners, with the performance of the very best firms much better than those of the typical firm. For example, the average number of workers among the non-winning firms in the top 10% of employment is 16 (median 13), while the average in the bottom 90% is 3 (median 3, i.e. one employees) and the average profits in the top decile is more than twice that in the rest of the firms. Even if the human experts and machine predictions have difficulty distinguishing between firms across the whole distribution, the question remains as to whether they fare better when trying to identify in advance these very top performers.

To investigate this, we generated a dummy variable equal to one if the firm is in the top 10 percent of thirdyear employment (or sales, or profits, respectively) and zero otherwise. We then applied the same models as in Tables 4 and 5 to modeling being in the top decile, and use this to predict the probability of being in the top decile for each firm in the test sample, again using 5-fold estimation. The accuracy rate is then 21
obtained by computing the proportion of firms correctly predicted to be in the top decile or the bottom 90%.<sup>8</sup> Since we are particularly interested in being able to detect which firms will excel, we also compute the recall rate – i.e. the proportion of the top tail of firms that were correctly predicted.<sup>9</sup> Tables 6 reports the results. As with the survival outcome, we provide a model that just uses a constant as a comparison of just selecting firms randomly. For both non-winners and winners, we see that it is possible to do about twice as well as a random draw in terms of the recall rate, which would potentially make a big difference to expected returns from an investment portfolio. We do find boosting to have the highest accuracy for employment for both winners and non-winners. Post-LASSO has also high accuracy and recall rates for employment and sales among winners, showing the potential for improvements from machine learning. However, these algorithms have lower performances for sales among non-winners, as well as profits among winners The judge scores perform better than in Table 5. In particular, they correctly identify 13.8% of firms in the top decile of winners for employment, 17.6% of firms in the top decile for sales, and 21.5% of firms in the top decile for profits. The McKenzie (2015) and Fafchamps and Woodruff (2017) models always do better than random selection, but only have the best performance when it comes to predicting sales for the nonwinners. Simple single predictor models continue to do almost as well as these more complicated approaches: using cognitive and non-cognitive measures such as the Raven test, grit and digit-span, and demographic variables such as gender and age can help in targeting the top tail.

To see how much difference this would make for an investor, and give a measure of whether these differences are economically significant, we consider the case of an investor who is picking the top 100 firms (approximately the top 10%). We assume this investor makes an investment in each of these firms, and then would get a percentage of profits, so that their goal is to maximize total profits of the firms they choose. We then use the predicted profits from Table 5 to sort firms in the test sample, and choose the top 20 firms in each fold according to each method. We then aggregate these up to report the total monthly profits that the investor would achieve if choosing firms randomly, or according to each selection method. The results are reported in Table A8. The overall profits of the top winning firms selected using the business plan scores are twice those of 20 firms randomly selected. Using the predictions of the ad-hoc researcher

> 8  We have followed this approach of setting a fixed number of businesses to predict as successful instead of the alternative of predicting one (success) if the predicted probability is above 0.5 (or the mean of the outcome variable, 0.1 in our case) because we deem this more realistic and similar to how a venture capitalist would act when having to decide on which firms to invest. 9 An advantage of our machine learning approach is the flexibility to consider different objective functions. For instance, since each start-up may have different requests in terms of initial capital necessary to found or expand the business, some investors may actually be interested in maximizing the recall rate subject to a fixed amount of money invested in the businesses predicted to be successful. The calibration procedure can be easily adapted to take this objective into account.

22
models provides even better performances, up to 4 times those from random selection, and bests the machine learning portfolios. However, business plan scores do worse than randomly choosing among the non-winners, whereas the ad hoc economist models would still offer 50 percent higher returns.

# **6. Conclusions**

Our results from a large business plan competition show that human experts in the form of judges have difficulty in identifying which firms will succeed. In fact, business plan scores have no significant correlation with the survival, employment, sales, and profits of these firms three years later. These judges do better when it comes to predicting the very top tail of performance among competition winners, identifying around 20 percent of firms that will end up in the top decile of sales and profits.

This poor performance of the judges’ scores does not mean business performance is completely unpredictable. Indeed, we find that several basic demographic characteristics of the entrepreneurs (age and gender) and measures of their ability (Raven test score) do increase prediction accuracy: males in their 30s who score highly on an ability test are more likely to succeed. We then compare simple ad-hoc models developed by economists, and machine learning approaches. Strikingly, we find that despite the large number of potential predictors available for machine learning, it neither outperforms the simple models offered by economists, nor even simpler single predictor models when it comes to predicting average performance.<sup>10</sup> We only find machine learning to do somewhat better when it comes to identifying the top tail of employment and profits, but an investor would do best just using the combination of man and machine in the ad hoc models of economists rather than relying on human judges or machine learningchosen portfolios.

Several caveats are worth noting. First, our analysis is all conditional on individuals applying for a business plan competition and getting through the first round to the point of submitting a business plan. This process likely selects out many individuals with very low growth prospects, and it may be easier to predict who definitely will not succeed than to predict who might. Nevertheless, there is still substantial variation in business performance among the sample we work with, and substantial variation in how these firms were viewed by judges.

Second, although we have used three of the most popular machine learning algorithms, it is possible that more advanced algorithms or extensive grid-searchers could further improve performance. However, such algorithms may still be computationally infeasible, even for big companies (Johnston 2012), or extremely

> 10 This lack of improvement from machine learning is in line with results in several other domains. For example, Beattie et al. (2016) found a simple average of seven non-academic variables performed as well as sophisticated machine learning techniques when analyzing the importance of personality traits in college student performance; while a simple autoregressive model has similar performances to Google’s Flu trends algorithm (Goel et al. 2010).

23
difficult to code, which is the reason behind the extremely high prizes – often reaching $1 million (Netflix 2009) - offered in machine learning competitions.

Third, these algorithms have been designed to be applied to large data sets with millions of observations. Although we use data from the world’s largest business plan competition, and policy makers and investors would typically have to make investment decisions based on the past performance of a more limited number of firms, our sample sizes may still make it hard to train these algorithms.

Finally, our focus has been on predicting which firms will have the best outcomes, which is an object of most interest to investors and lenders. In contrast, such predictions are usually inappropriate for a policy maker aiming to optimally allocate resources (Athey 2017), who would instead be most interested in allocating funds to firms that would benefit most from the grants, not to those firms which would be successful anyway.<sup>11</sup> But even in such cases, identifying a pool of firms that are likely to survive and grow may be a useful first step in screening applications, and policy makers may also care about offering support to the next flagship company.

> 11  See Hussam et al. (2016) for a recent application of using machine learning to predict which small entrepreneurs will have the highest returns to grants.

24
# **References**

Astebro T, Elhedhli S (2006) The effectiveness of simple decision heuristics: Forecasting commercial success for early-stage ventures. _Manage. Sci._ 52(3):395–409.

Athey S (2017) Beyond prediction: Using big data for policy problems. _Sci. Mag._ 355(6324):483–485. Aulck L, Velagapudi N, Blumenstock J, West J (2016) _Predicting Student Dropout in Higher Education_ Bauer E, Kohavi R, Chan P, Stolfo S, Wolpert D (1999) An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants. _Mach. Learn._ 36(August):105–139.

Beattie G, Laliberte JWP, Oreopoulos P (2016) Thrivers and Divers: Using Non-Academic Measures To Predict College Success and Failure. _NBER Work. Pap. Ser._ 1.

Blanchflower DG, Levine PB, Zimmerman DJ (2003) Discrimination in the Small-Business Credit Market. _Rev. Econ. Stat._ 85(4):930–943.

Boden RJ, Nucci AR (2000) On the survival prospects of men’s and women’s new business ventures. _J. Bus. Ventur._ 15(4):347–362.

Bruhn M (2009) Female-Owned Firms in Latin America: Characteristics, Performance, and Obstacles to Growth. _World Bank Policy Res. Work. Pap._ 5122.

Celiku B, Kraay A (2017) Predicting Conflict. _Policy Res. Work. Ppaer_ 8075.

Chalfin A, Danieli O, Hillis A, Jelveh Z, Luca M, Ludwig J, Mullainathan S (2016) Productivity and Selection of Human Capital with Machine Learning. _Am. Econ. Rev. Pap. Proc._ 106(5):124–127.

Duckworth AL, Peterson C, Matthews MD, Kelly DR (2007) Grit: Perseverance and passion for long-term goals. _J. Pers. Soc. Psychol._ 92(6):1087–1101.

Fafchamps M, Woodruff C (2017) Identifying Gazelles: Expert Panels vs. Surveys as a Means to Identify Firms with Rapid Growth Potential. _World Bank Econ. Rev._ 31(3):670–686.

Friedman J (2001) Greedy Function Approximation : A Gradient Boosting Machine. _Ann. Stat._ 29(5):1189– 1232.

Friedman J, Hastie T, Tibshirani R (2000) Additive Logistic Regression: A Statistical View of Boosting. _Ann. Stat._ 28(2):337–407.

Giannone D, Lenza M, Primiceri GE (2017) _Economic Predictions with Big Data: The Illusion of Sparsity_ Goel S, Hofman JM, Lahaie S, Pennock DM, Watts DJ (2010) Predicting consumer behavior with Web search. _Proc. Natl. Acad. Sci._ 107(41):17486–17490.

Hall RE, Woodward SE (2010) The burden of the nondiversifiable risk of entrepreneurship. _Am. Econ. Rev._ 100(3):1163–1194.

Hastie T, Tibshirani R, Friedman J (2009) _The Elements of Statistical Learning: Data Mining, Inference, and Prediction_ Second Edi. (Springer).

Hussam R, Rigol N, Roth B (2016) Targeting High Ability Entrepreneurs Using Community Information: Mechanism Design In The Field. _Work. Pap._ Hvide HK, Panos GA (2014) Risk tolerance and entrepreneurship. _J. financ. econ._ 111(1):200–223.

Jean N, Burke M, Xie M, Davis WM, Lobell DB, Ermon S (2016) Combining satellite imagery and machine learning to predict poverty. _Sci. Mag._ 353(6301):790–4.

Johnston C (2012) Netflix Never Used Its $1 Million Algorithm Due To Engineering Costs. _Wired_ (April).

Kerr WR, Lerner J, Schoar A (2014) The consequences of entrepreneurial finance: Evidence from angel financings. _Rev. Financ. Stud._ 27(1):20–55.

Kleinberg J, Lakkaraj H, Leskovec J, Ludwig J, Mullainathan S (2017) Human Decisions and Machine Predictions. _NBER Work. Pap. Ser._ 23180:76.

Kleinberg J, Ludwig J, Mullainathan S, Obermeyer Z (2015) Prediction Policy Problems. _Am. Econ. Rev. Pap. Proc._ 105(5):491–495.

Kopf D (2015) What happened when Nigeria created the World’s Largest Business Plan Competition. _Priceonomics_ .

Luca M, Kleinberg J, Mullainathan S (2016) Algorithms Need Managers, Too. _Harv. Bus. Rev._ 104(1/2):96–101.

Maroco J, Silva D, Rodrigues A, Guerreiro M, Santana I, de Mendonça A (2011) Data mining methods in 25
the prediction of Dementia: A real-data comparison of the accuracy, sensitivity and specificity of linear discriminant analysis, logistic regression, neural networks, support vector machines, classification trees and random forests. _BMC Res. Notes_ 4(1):299.

- McKenzie D (2015) Identifying and Spurring High-Growth Entrepreneurship: Experimental Evidence from a Business Plan Competition. _World Bank Policy Res. Work. Pap._ 7391.

- McKenzie D (2017) Identifying and Spurring High-Growth Entrepreneurship: Experimental Evidence from a Business Plan Competition. _Am. Econ. Rev._ 107(8):2278–2307.

- McKenzie D, Paffhausen AL (2017) _Small firm death in developing countries_ , World Bank Policy Research Working Paper no. 8236.

- Michelacci C, Schivardi F (2016) Are They All Like Bill, Mark, and Steve? The Education Premium for Entrepreneurs. _EIEF Work. Pap._ 16(12).

- Mullainathan S, Spiess J (2017) Machine Learning: An Applied Econometric Approach. _J. Econ. Perspect._ 31(2):87–106.

- Nanda R (2016) Financing high-potential entrepreneurship. _IZA World Labor_ (April):1–10. Netflix (2009) Netflix Prize. Retrieved (January 1, 2017), http://www.netflixprize.com/.

Ng A (2016) Machine Learning. _Coursera_ . Retrieved (January 1, 2016), https://www.coursera.org/learn/machine-learning.

- Nikolova E, Ricka F, Simroth D (2011) Entrepreneurship in the transition region: an analysis based on the Life in Transition Syrvey. _Cris. Transit. People’s Perspect._ (EBRD), 76–96.

- OECD (2010) High-Growth Enterprises : What Governments Can Do To Make A Difference. _OECD Stud. SMEs Entrep._

- Olafsen E, Cook PA (2016) _Growth Entrepreneurship in Developing Countries: A Preliminary Literature Review_ (The World Bank Group, Washington, D.C.).

- van Praag CM, Cramer JS (2001) The roots of entrepreneurship and labour demand: Individual ability and low risk aversion. _Economica_ 68(269):45–62.

- Queiro F (2016) The Effect of Manager Education on Firm Growth. _Harvard Work. Pap._

- Reddy S (2017) A computer was asked to predict which start-ups would be successful. The results were astonishing. _World Economic Forum_ (July).

- Ricadela A (2009) Fifty Best Tech Startups. _Bloomberg Businessweek_ (June).

- Robb AM, Watson J (2012) Gender differences in firm performance: Evidence from new ventures in the United States. _J. Bus. Ventur._ 27(5):544–558.

- Schonlau M (2005) Boosted regression (boosting): An introductory tutorial and a Stata plugin. _Stata J._ 5(3):330–354.

- Scott E, Shu P, Lubynsky R (2015) Are “better” ideas more likely to succeed? An empirical analysis of startup evaluation. _Harvard Bus. Sch. Work. Pap._ 16(13).

- Skriabikova OJ, Dohmen T, Kriechel B (2014) New evidence on the relationship between risk attitudes and self-employment. _Labour Econ._ 30:176–184.

- Subrahmanian VS, Kumar S (2017) Predicting human behavior: The next frontiers. _Sci. Mag._ 355(6324):489.

- Varian HR (2014) Big Data : New Tricks for Econometrics. _J. Econ. Perspect._ 28(2):3–28.

- Verplancke T, Van Looy S, Benoit D, Vansteelandt S, Depuydt P, De Turck F, Decruyenaere J (2008)

- Support vector machine versus logistic regression modeling for prediction of hospital mortality in critically ill patients with haematological malignancies. _BMC Med. Inform. Decis. Mak._ 8(1):56.

- Zacharakis AL, Shepherd DA (2001) The nature of information and overconfidence on venture capitalists’ decision making. _J. Bus. Ventur._ 16(4):311–332.

- Zygmunt Z (2016) What is better: gradient-boosted trees, or a random forest? _FastML_ . Retrieved (January 1, 2017), http://fastml.com/what-is-better-gradient-boosted-trees-or-random-forest/.

26
**Figure 1: Substantial Dispersion in Firm Outcomes Three Years After Applying**

<!-- Start of picture text --> Total Employment Sales<br>0 20 40 60<br>Total Employment in Operating Firms 0 5 10 15 20<br>Sales (IHS transformation)<br>Winners Non-winners<br>Winners Non-winners<br>Total employment truncated at 60 (11 outliers)<br>Profits<br>5 10 15 20<br>Profits (IHS transformation)<br>Winners Non-winners<br>.4<br>.1<br>.3<br>.05 .2<br>Density<br>Density<br>.1<br>0<br>0<br>.4<br>.3<br>.2<br>Density<br>.1<br>0<br><!-- End of picture text --> Notes: Figure shows kernel densities of firm outcomes for firms surveyed in third follow-up, and are conditional on the firm operating at that time. Winners include both non-experimental and experimental winners, and non-winners include control group semi-finalists as well as a sample of firms submitting business plans who had first round application scores just above the cut-off for getting selected for business plan training. Profits and Sales are measured in monthly Naira, and have been transformed using the inverse hyperbolic sine transformation.

27
**Figure 2: Distribution of Business Plan Scores by Competition Status**

<!-- Start of picture text --> 0 20 40 60 80 100<br>Business plan score<br>Non-Experimental Winner Experimental Winner<br>Control Non-Winner Other Non-Winners<br>.05<br>.04<br>.03<br>Density<br>.02<br>.01<br>0<br><!-- End of picture text --> Notes: kernel densities of business plan scores shown. Non-experimental winners were chosen on the basis of having the top scores nationwide, or as being the most promising within their geographic region, with fixed quotas by whether the application was for a new or existing business. Experimental winners and control non-winners were chosen randomly from within the remaining semi-finalists, with this randomization done within strata defined by region and existing or new business status. Other non-winners are the scores for our remaining sample of non-winners, selected on the basis of only just making the first round cut to receive business plan training.

28
**Figure 3: The Relationship between Business Plan Scores of Judges and Business Outcomes Three Years after applying for Non-Winners in the Business Plan Competition**

<!-- Start of picture text --> 1 2 3 4 5 0 20 40 60 80<br>Business Plan Score Quintile Business plan score<br>0 20 40 60 80 0 20 40 60 80<br>Business plan score Business plan score<br>20<br>.6<br>15<br>.4<br>10<br>.2 5<br>Proportion operating a firm IHS transformation of sales 0<br>0<br>20<br>40<br>15<br>30<br>10 20<br>5 Total Employment 10<br>IHS transformation of profits 0 0<br><!-- End of picture text --> Notes: Top left panel shows business operation three years after applying by quintile of business plan score. Remaining panels show scatterplots of the sales, profits, and employment three years after applying against the business plan score. The dashed line plotted on each figure is a lowess line of the unconditional relationship, which codes outcomes as zero for applicants who are not operating firms. The solid line on each figure is a lowess line of the relationship conditional on the business being in operation. Employment truncated at 60 workers for readability 29
**Figure 4: The Relationship between Business Plan Scores of Judges and Business Outcomes Three Years after applying for Winners in the Business Plan Competition**

<!-- Start of picture text --> 1 2 3 4 5 20 40 60 80 100<br>Business Plan Score Quintile Business plan score<br>20 40 60 80 100 20 40 60 80 100<br>Business plan score Business plan score<br>1 20<br>.8 15<br>.6<br>10<br>.4<br>5<br>.2<br>Proportion operating a firm IHS transformation of sales 0<br>0<br>20 60<br>15<br>40<br>10<br>20<br>5<br>Total Employment<br>IHS transformation of profits 0 0<br><!-- End of picture text --> Notes: Top left panel shows business operation three years after applying by quintile of business plan score. Remaining panels show scatterplots of the sales, profits, and employment three years after applying against the business plan score. The dashed line plotted on each figure is a lowess line of the unconditional relationship, which codes outcomes as zero for applicants who are not operating firms. The solid line on each figure is a lowess line of the relationship conditional on the business being in operation. Employment truncated at 60 workers for readability, profits truncated from below at zero.

30
**Table 1: Can Human Experts Predict Business Survival After 3 Years?**

|||Non-Winners|||Wi|nners||
|---|---|---|---|---|---|---|---|
||(1)|(2)|(3)|(4)|(5)|(6)|(7)|
||Judges|McKenzie|FaWo|Judges|Judges|McKenzie|FaWo|
|Business Plan score|-0.0002|-0.0009|-0.0009|0.0000|-0.0003|-0.0006|-0.0007|
||(0.0014)|(0.0015)<br>|(0.0015)<br>|(0.0006)|(0.0006)|(0.0008)|(0.0008)|
|Female||-0.1519<sup>***</sup>|-0.1250<sup>**</sup>|||-0.0009|0.0082|
|||(0.0384)<br>|(0.0385)<br>|||(0.0214)|(0.0220)|
|Age||0.0109<sup>***</sup>|0.0087<sup>**</sup>|||0.0034|0.0037|
|||(0.0032)|(0.0031)|||(0.0021)|(0.0019)|
|Graduate Education||-0.0208||||-0.0028||
|||(0.0631)||||(0.0288)||
|Worked abroad||0.0249||||-0.0397||
|||(0.0589)||||(0.0248)||
|Lottery choice||-0.0201||||-0.0145||
|||(0.0291)||||(0.0161)||
|Wealth||-0.0031||||0.0020||
|||(0.0057)||||(0.0027)||
|Grit||-0.0367||||-0.0049||
|||(0.0285)||||(0.0154)||
|Ability||0.0507<sup>**</sup>||||-0.0080||
|||(0.0169)||||(0.0088)||
|Agriculture||-0.0481||||-0.0157||
|||(0.0351)||||(0.0194)||
|IT||-0.0563||||-0.0295||
|||(0.0510)||||(0.0243)||
|Manufacture||<br>0.0028|0.0326|||<br>-0.0303|-0.0218|
|||(0.0394)|(0.0352)|||(0.0199)|(0.0177)|
|Retail|||0.0635||||0.0185|
||||(0.0800)||||(0.0457)|
|Self-confidence|||0.0229<sup>*</sup>||||0.0032|
||||(0.0093)||||(0.0040)|
|Motivation|||0.0211<sup>*</sup>||||-0.0060|
||||(0.0107)||||(0.0047)|
|Happiness|||-0.0009||||-0.0083|
||||(0.0119)||||(0.0070)|
|Optimism|||0.0336<sup>*</sup>||||0.0095|
||||(0.0135)||||(0.0075)|
|Ability|||0.0334<sup>*</sup>||||-0.0090|
||||(0.0162)||||(0.0083)|
|Credit|||<br>0.0150||||<br>-0.0140|
||||(0.0173)||||(0.0074)|
|Existing|0.2140<sup>***</sup>|0.1692<sup>***</sup>|<br>0.0721|0.0386<sup>*</sup>|0.0364<sup>*</sup>|0.0199|<br>0.0108|
||(0.0386)|(0.0409)|(0.0465)|(0.0159)|(0.0157)|(0.0195)|(0.0202)|
|Log(Total award paid)|||||<br>0.0250|<br>0.0223|<br>0.0203|
||||||(0.0147)|(0.0142)|(0.0152)|
|Regional Indicators|No|Yes|Yes|No|No|Yes|Yes|
|Sample Size|1,078|1,078|1,078|1,062|1,062|1,062|1,062|
|P-Value||0.000|0.000|||0.671|0.091|

Robust Standard Errors in parentheses. *, **, *** denote significance at the 10, 5, and 1 percent levels respectively. See Appendix A.1 for variable definitions. Dependent variable is having a business in operation 3 years after applying. Marginal effects from logit shown. P-value is for testing that set of controls added by MacKenzie (2015) or Fafchamps and Woodruff (2017) (abbreviated as FaWo) are jointly zero.

31
**Table 2: Can Human Experts Predict Employment After 3 Years?**

|||Non-Winners<br>|||Wi<br>|nners<br>||
|---|---|---|---|---|---|---|---|
||(1)|(2)|(3)|(4)|(5)|(6)|(7)|
||Judges|McKenzie|FaWo|Judges|Judges|McKenzie|FaWo|
|Business Plan score|-0.0021|0.0009|0.0047|0.0897<sup>**</sup>|0.0490|0.0809<sup>*</sup>|0.0676<sup>*</sup>|
||(0.0168)|(0.0181)|(0.0181)|(0.0310)|(0.0306)|(0.0355)|(0.0335)|
|Female||-1.0909<sup>*</sup>|-0.7578|||-0.5887|-0.0360|
|||(0.4781)<br><sup>**</sup>|(0.4858)<br><sup>*</sup>|||(0.8149)<br><sup>*</sup>|(0.7675)<br><sup>*</sup>|
|Age||0.1060<br>|0.0790<br>|||0.1688<br>|0.1546<br>|
|||(0.0410)|(0.0395)|||(0.0801)<br>|(0.0761)|
|Graduate Education||0.9657||||-2.5224<sup>**</sup>||
|||(0.9788)||||(0.8966)||
|Worked abroad||0.1697||||-0.9984||
|||(0.8817)||||(1.0346)||
|Lottery choice||0.5653||||-0.4831||
|||(0.3496)||||(0.6551)||
|Wealth||0.0975||||0.1306||
|||(0.0681)||||(0.1316)||
|Grit||-0.4671<br>||||-0.4558<br>||
|||(0.3609)||||(0.5989)||
|Ability||0.4455<sup>*</sup>||||0.0892||
|||(0.2175)||||(0.3137)||
|Agriculture||-0.5736<br>||||-1.0380<br>||
|||(0.4540)||||(0.7790)||
|IT||-1.3354<sup>**</sup>||||-2.7006<sup>**</sup>||
|||(0.4927)||||(0.8364)||
|Manufacture||-0.4641|-0.0862|||-0.3856|0.0105|
|||(0.4545)|(0.4160)|||(0.9036)|(0.7894)|
|Retail|||0.6146||||-0.7114|
||||(0.9287)||||(0.8817)|
|Self-confidence|||0.1553||||0.1095|
||||(0.0940)||||(0.2036)|
|Motivation|||0.0909||||-0.2148|
||||(0.1141)||||(0.2026)|
|Happiness|||0.2122||||0.4886|
||||(0.1395)||||(0.2640)|
|Optimism|||0.1723||||-0.1442|
||||(0.1588)||||(0.2853)|
|Ability|||0.2808||||-0.0598|
||||(0.1860)||||(0.3199)|
|Credit|||-0.0502||||1.0291|
||||(0.1548)||||(0.5703)|
|Existing|1.6360<sup>***</sup>|1.4909<sup>**</sup>|0.4277|2.5845<sup>***</sup>|2.3326<sup>***</sup>|2.9470<sup>***</sup>|1.8239<sup>*</sup>|
||(0.4491)|(0.4674)|(0.5682)|(0.6311)|(0.6157)|(0.7506)|(0.8387)|
|Log(Total award paid)|||||3.8675<sup>***</sup>|3.8336<sup>***</sup>|3.8410<sup>***</sup>|
||||||(0.4327)|(0.4440)|(0.4459)|
|Regional Indicators|No|Yes|Yes|No|No|Yes|Yes|
|Sample Size|1,078|1,078|1,078|1,062|1,062|1,062|1,062|
|P-value||0.000|0.000|||0.042|0.018|
|AdjustedR<sup>2</sup>|0.013|0.039|0.049|0.021|0.057|0.078|0.084|

Robust Standard Errors in parentheses. *, **, *** denote significance at the 10, 5, and 1 percent levels respectively. See Appendix A.1 for variable definitions. Dependent variable is number of employees 3 years after applying, coded as zero for applicants without an operating firm. P-value is for testing that set of controls added by MacKenzie (2015) or Fafchamps and Woodruff (2017) (abbreviated as FaWo) are jointly zero.

32
**Table 3: Can Human Experts Predict Profits After 3 Years?**

|||Non-Winner|s||Wi|nners||
|---|---|---|---|---|---|---|---|
||(1)|(2)|(3)|(4)|(5)|(6)|(7)|
||Judges|McKenzie|Fafchamps|Judges|Judges|McKenzie|Fafchamps|
|Business Plan score|-0.0041|-0.0059|-0.0068|0.0009|-0.0055|-0.0077|-0.0070|
||(0.0180)|(0.0187)|(0.0183)|(0.0127)|(0.0130)|(0.0143)|(0.0139)|
|Female||-1.9060<sup>***</sup><br>|-1.5232<sup>**</sup><br>|||-0.9084<sup>*</sup><br>|-0.7493<br>|
|||(0.4940)|(0.4889)|||(0.4427)|(0.4437)|
|Age||0.1241<sup>**</sup>|0.0856<sup>*</sup>|||0.0488|0.0454|
|||(0.0413)|(0.0408)|||(0.0357)|(0.0350)|
|Graduate Education||-0.9355||||-0.2196||
|||(0.8028)||||(0.6096)||
|Worked abroad||<br>0.8023||||<br>-1.2460<sup>*</sup>||
|||(0.7827)||||(0.6162)||
|Lottery choice||-0.0858<br>||||-0.3335<br>||
|||(0.3742)||||(0.2986)||
|Wealth||<br>-0.0250||||<br>0.0983<sup>*</sup>||
|||(0.0741)||||(0.0467)||
|Grit||-0.4430<br>||||-0.1569<br>||
|||(0.3611)||||(0.2745)||
|Ability||0.6697<sup>**</sup>||||-0.1657||
|||(0.2152)||||(0.1714)||
|Agriculture||-0.4077<br>||||-0.1922<br>||
|||(0.4580)||||(0.3703)||
|IT||-0.8747||||-0.4450||
|||(0.6561)||||(0.4687)||
|Manufacture||0.2694|0.5862|||0.1297|0.3557|
|||(0.4920)|(0.4386)|||(0.3850)|(0.3560)|
|Retail|||0.8505||||1.4675<sup>**</sup>|
||||(0.9435)||||(0.5256)|
|Self-confidence|||0.2508<sup>*</sup>||||0.1727|
||||(0.1164)||||(0.0959)|
|Motivation|||0.0803||||-0.1730|
||||(0.1394)||||(0.1035)|
|Happiness|||0.1224||||-0.0216|
||||(0.1550)||||(0.1238)|
|Optimism|||0.4247<sup>*</sup>||||-0.0293|
||||(0.1744)||||(0.1322)|
|Ability|||0.4466<sup>*</sup>||||-0.2251|
||||(0.2021)||||(0.1561)|
|Credit|||<br>0.2527||||<br>-0.1028|
||||(0.1453)||||(0.1971)|
|Existing|2.7561<sup>***</sup>|2.4015<sup>***</sup>|0.9045|1.0744<sup>***</sup>|1.0263<sup>***</sup>|0.8252<sup>*</sup>|0.6637|
||(0.4717)|(0.5047)|(0.5662)|(0.2971)|(0.2964)|(0.3534)|(0.4177)|
|Log(Total award paid)|||||<br>0.4332|<br>0.3865|<br>0.3941|
||||||(0.2969)|(0.3008)|(0.3027)|
|Regional Indicators|No|Yes|Yes|No|No|Yes|Yes|
|Sample Size|1,059|1,059|1,059|1,047|1,047|1,047|1,047|
|P-value||0.000|0.000|||0.058|0.001|
|AdjustedR2|0.035|0.083|0.125|0.011|0.020|0.030|0.035|

Robust Standard Errors in parentheses. *, **, *** denote significance at the 10, 5, and 1 percent levels respectively. See Appendix A.1 for variable definitions. Dependent variable is inverse hyperbolic sine of profits 3 years after applying, coded as zero for applicants without an operating firm. P-value is for testing that set of controls added by MacKenzie (2015) or Fafchamps and Woodruff (2017) (abbreviated as FaWo) are jointly zero.

33
**Table 4: Prediction Accuracy for Business Survival Among Non-winners**

|||Model|Predictors|Accuracy|
|---|---|---|---|---|
||1|Logit|Constant|58.4%|
||2|Logit|Application Score|58.1%|
|Judges|3|Logit|Business Score|57.1%|
||4|Logit|SubScore|57.5%|
||5|Logit|Gender|60.8%|
||6|Logit|Age|60.0%|
|Sinle|7|Logit|Necessity Firm|58.0%|
|g<br>Predictor|8|Logit|Grit|59.3%|
||9|Logit|Digit-span Recall|58.7%|
||10|Logit|Raven Test|63.8%|
||11|Logit|Registration time|57.0%|
||12|Logit|McKenzie|61.5%|
|Economist|13|Logit|FaWo|65.6%|
|Models|14|Logit|FaWo + BusScores|65.5%|
||15|OLS|FaWo + BusScores|65.5%|
||16|Probit|FaWo+ BusScores|65.3%|
|Mhi|17|Post-LASSO|All|61.9%|
|acne-<br>Larnin|18|SVM|All|63.6%|
|eg|19|Boosting|All|59.8%|

Notes: outcome is whether an applicant who did not win the business plan competition is found to be operating a firm three years later. Accuracy is the ratio of true positives and true negatives to all observations, and is the out-of-sample average over 5-fold estimation. Judges use scores from competition judges to predict the outcome. Single predictor models use a single survey measure. The economist models are the models of McKenzie (2015) and Fafchamps and Woodruff (2017) (abbreviated as FaWo) used in Tables 1-3. The three machine learning models are based on 393 predictors, along with interaction terms in the case of post-LASSO. All models apart from the constant model also include a dummy for whether the applicant was for an existing firm compared to a new firm.

34
# **Table 5: Prediction Accuracy for Total Employment, Monthly Sales and Monthly Profits**

||||||Total E|mploymen|t||Monthl|y Sales|||Monthl|yProfits||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|||||Non-w|inners|Win|ners|Non-w|inners|Win|ners|Non-|winners|Win|ners|
|||Model|Predictors|MSE|R<sup>2</sup>|MSE|R<sup>2</sup>|MSE|R<sup>2</sup>|MSE|R<sup>2</sup>|MSE|R<sup>2</sup>|MSE|R<sup>2</sup>|
||1|OLS|Application Score|34.26|2.4%|99.11|5.6%|43.66|4.0%|20.74|2.4%|37.54|3.9%|22.02|2.9%|
|Judges|2|OLS|Business Score|34.32|1.8%|98.77|5.9%|43.89|3.5%|20.94|2.0%|37.79|3.2%|22.21|2.1%|
||3|OLS|SubScore|34.29|2.0%|98.62|6.4%|44.00|3.4%|21.13|1.5%|37.98|2.9%|22.55|1.2%|
||4|OLS|Gender|34.16|2.4%|99.12|5.6%|43.14|5.3%|20.71|2.3%|37.17|4.9%|22.03|2.5%|
||5|OLS|Age|33.97|2.6%|98.93|6.1%|43.10|5.3%|20.74|2.4%|37.26|4.8%|22.02|2.9%|
|Sil|6|OLS|Necessity Firm|34.53|1.0%|99.79|5.0%|43.97|3.4%|20.69|2.7%|37.91|2.9%|22.32|1.5%|
|nge<br>Pdit|7|OLS|Grit|34.46|1.8%|99.09|5.6%|43.35|4.6%|20.83|2.1%|37.35|4.2%|22.10|2.6%|
|recor|8|OLS|Digit-span Recall|34.47|1.5%|99.35|5.4%|43.36|5.0%|21.04|1.8%|37.39|4.6%|22.30|1.8%|
||9|OLS|Raven Test|34.19|1.7%|99.42|5.3%|41.96|8.0%|20.73|2.2%|36.47|6.6%|21.94|2.8%|
||10|OLS|Registration time|34.33|1.9%|98.81|6.0%|43.62|4.0%|20.69|2.1%|37.54|3.9%|22.08|2.1%|
||11|OLS|McKenzie|33.98|2.9%|99.87|5.8%|42.02|8.0%|21.01|1.9%|36.62|6.5%|22.54|1.6%|
|Economist|<br>12|OLS|FaWo|33.40|4.8%|101.05|5.6%|40.18|12.0%|21.05|1.9%|35.02|10.7%|<br>22.53|1.7%|
|Models|13|OLS|FaWo + BusScores|33.42|4.8%|100.61|5.9%|40.49|11.4%|21.26|1.6%|35.26|10.1%|<br>22.74|1.6%|
||14|OLS|FaWo+ Baseline|39.96|4.9%|91.20|17.2%|NA|NA|NA|NA|NA|NA|NA|NA|
|Mhi|15|Post-LASSO|All|34.03|3.0%|94.77|13.0%|41.96|8.5%|21.46|0.1%|37.20|6.2%|23.01|1.0%|
|acne-<br>Learnin|16|SVR|All|34.07|3.0%|105.80|3.3%|40.57|11.1%|21.13|0.8%|35.25|10.0%|<br>22.34|1.0%|
|g|17|Boosting|All|35.31|1.3%|104.20|12.4%|40.17|11.9%|21.17|1.3%|36.14|8.2%|22.17|1.2%|

Notes: the first outcome is total employment in the firm three years after applying, coded as zero for applicants not operating firms. The second outcome is the inverse hyperbolic sine of monthly sales in the firm three years after applying, coded as zero for applicants not operating firms. The third outcome is the inverse hyperbolic sine of monthly profits in the firm three years after applying, coded as zero for applicants not operating firms. Models are estimating separately for competition non-winners and winners. MSE is the outof-sample average mean-squared error over 5-fold estimation, and R<sup>2</sup> is the squared correlation between the fitted outcome and actual outcome. Judges use scores from competition judges to predict the outcome. Single predictor models use a single survey measure. The economist models are the models of McKenzie (2015) and Fafchamps and Woodruff (2017) (abbreviated as FaWo) used in Tables 1-3. The three machine learning models are based on 393 predictors, along with interaction terms in the case of post-LASSO. All models include a dummy for whether the applicant was for an existing firm compared to a new firm, and models for winners also control for the total award received (in logs).

35
**Table 6: Prediction Accuracy for Tail Total Employment, Monthly Sales and Monthly Profits**

||||||Total Em|ploymen|t||Month|ly Sales|||Monthl|yProfits||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|||||Non-w|inners|Wi|nners|Non-w|inners|Wi|nners|Non-w|inners|Wi|nners|
|||Model|Predictors|Accu|Recall|Accu|Recall|Accu|Recall|Accu|Recall|Accu|Recall|Accu|Recall|
||1|Logit|Constant|80.7%|11.4%|80.8%|8.5%|81.8%|9.0%|81.3%|6.3%|80.8%|10.1%|80.5%|8.9%|
||2|Logit|Application Score|81.8%|15.7%|80.8%|8.8%|83.9%|19.1%|83.2%|15.7%|81.6%|13.2%|81.7%|<br>14.0%|
|Judges|3|Logit|Business Score|81.6%|14.3%|81.9%|13.8%|81.8%|9.1%|83.6%|17.6%|81.4%|11.8%|83.6%|<br>21.5%|
||4|Logit|SubScore|81.3%|12.6%|82.5%|17.0%|82.0%|10.2%|82.8%|14.0%|81.8%|14.1%|82.6%|17.9%|
||5|Logit|Gender|81.3%|12.9%|81.5%|12.5%|83.5%|17.5%|83.0%|14.8%|82.0%|14.4%|82.4%|<br>17.4%|
||6|Logit|Age|81.8%|15.3%|81.0%|9.4%|82.9%|14.8%|82.0%|10.1%|82.3%|16.0%|83.6%|<br>21.4%|
|Sil|7|Logit|Necessity Firm|81.6%|14.5%|81.2%|10.2%|81.8%|9.6%|82.2%|11.4%|81.6%|13.1%|81.1%|<br>11.7%|
|nge<br>Pdit|8|Logit|Grit|81.1%|11.7%|82.3%|15.2%|82.0%|10.1%|83.2%|15.9%|81.2%|11.3%|82.4%|<br>16.7%|
|recor|9|Logit|Digit-span Recall|81.1%|12.5%|81.2%|10.6%|83.9%|19.4%|82.4%|12.3%|82.7%|18.2%|82.4%|<br>17.4%|
||10|Logit|Raven Test|81.6%|14.6%|81.7%|12.5%|82.7%|13.7%|83.6%|18.2%|82.2%|15.2%|82.0%|<br>15.3%|
||11|Logit|Registration time|82.2%|15.7%|82.7%|17.1%|82.0%|10.2%|81.9%|9.2%|82.2%|15.4%|82.4%|17.4%|
||12|Logit|McKenzie|81.3%|13.4%|81.0%|9.9%|82.4%|21.3%|83.4%|17.4%|82.7%|17.7%|81.9%|<br>14.2%|
|Eit|13|Logit|FaWo|81.6%|14.6%|82.1%|15.1%|83.9%|19.2%|83.2%|16.3%|82.0%|14.7%|82.4%|<br>17.2%|
|conoms<br>Mdl|14|Logit|FaWo + BusScores|81.8%|15.6%|81.5%|12.2%|83.1%|15.9%|83.0%|15.2%|81.8%|13.8%|82.6%|<br>17.7%|
|oes|15|OLS|FaWo + BusScores|81.6%|14.9%|82.5%|16.0%|83.7%|19.0%|84.0%|20.1%|81.8%|13.7%|82.8%|<br>18.7%|
||16|Probit|FaWo+ BusScores|81.8%|15.6%|81.9%|14.1%|83.5%|17.7%|83.4%|17.2%|81.8%|13.9%|82.6%|17.9%|
|Mhi|17|Post-LASSO|All|83.3%|21.7%|84.0%|22.5%|83.1%|16.1%|84.1%|21.0%|82.0%|15.0%|81.9%|<br>14.5%|
|acne-<br>Li|18|SVM|All|80.5%|9.2%|81.0%|9.0%|83.3%|17.0%|84.1%|21.1%|81.8%|14.3%|80.5%|<br>9.4%|
|earnng|19|Boosting|All|84.7%|18.6%|85.3%|27.6%|83.1%|15.8%|83.8%|19.3%|82.9%|18.4%|81.5%|<br>12.6%|

Note: Accuracy (abbreviated as Accu) is the ratio of true positives and true negatives to all observations, and is the out-of-sample average over 5-fold estimation. Judges use scores from competition judges to predict the outcome. Single predictor models use a single survey measure. The economist models are the models of McKenzie (2015) and Fafchamps and Woodruff (2017) (abbreviated as FaWo) used in Tables 1-3. The three machine learning models are based on 393 predictors, along with interaction terms in the case of post-LASSO. All models apart from the constant model also include a dummy for whether the applicant was for an existing firm compared to a new firm. The recall rate is the percentage of businesses correctly predicted to be in the top 10% over the total number of businesses actually in the top 10%.

36
# **Online Appendix**

# **A.1 Variable Description**

# **A.1.1 Outcome Variables**

_Three-year Survival_ is an indicator variable equal to one if respondent operates a business three years after the baseline interview.

_Three-year Employment_ is the total number of wage or salaried workers, as well as casual or daily workers, in the firm. Operating firms have at least one worker (the firm owner). Partners and unpaid workers are not included in this figure. If the respondent was not operating a business, this variable is zero. If the respondent did not know the number of wage or casual employees, it is assumed that there are no such workers in the firm. The same imputation has been done if the respondent could not say or refused to say the number of these workers. This variable is set to missing if the third-year survival indicator was missing _Three-year Sales_ are the total monthly sales (in Naira) of the respondent’s business from all sources – including manufacturing, trade and services - in the month preceding the interview. Sales have been set to zero if the respondent was not operating a business at that time, or if he or she did not know the figure.

_Three-year Profits_ are the total income the business earned during the month preceding the interview after paying all expenses including wages of employees, but not including any income paid to the owner. Profits have been set to zero if the respondent was not operating a business at that time, or if he or she did not know the figure.

# **A.1.2 Baseline Predictors**

For each variable, we have assigned zero to missing values and we have generated a new dummy variable to indicate the observations that are missing.

_Female_ is an indicator variable equal to one if the respondent is a woman, zero if he is a man.

_Age_ is the respondent’s age in years. Participants had to be 18 or older.

_Education_ is measured using four indicator variables to specify if the respondent’s highest educational level is:

- High school or lower

- Post-secondary school vocational training/OND/Technical colleges

- University/HND degree

- Post graduate degree
When such information was missing, we have looked at the same variable from the first follow-up (if available).

_Regional Indicators_ have been constructed for each of the following geographical areas:

- North-Central

- North-Eastern

- North-Western

- South-Eastern

- South-South

- South-Western

_Marital status_ is measured using an indicator variable for each of the following options:

- Married or Living Together with a Partner

- Divorced/Separated

- Widowed

- Never Married and Never Lived Together

_Family Composition_ . Three variables measure the number of respondents’ children, brothers and sisters.

_Language_ measures the main language spoken at home using five different indicator variables, one for each of the following options:

- Hausa

- Yoruba

- Igbo

- English

- Other

_Internal migrant_ indicates whether the participant’s state of origin is different from the state of residence.

_Business background_ . Participants were asked in the baseline interview whether they had ever attended before the start of the competition a course covering:

- Business accounting

- Marketing

- Computer skills

- Vocational training
_Outside Employment._ Participants were asked in the baseline interview whether they were employed at that time outside their proposed business. If yes, they were also asked whether they were working part-time or full-time.

_Experience abroad._ Participants were asked in the baseline interview whether they had ever worked or studied abroad. If yes, they were asked how many years they had spent abroad.

_Risk aversion._ Participants were asked in the baseline interview to choose between a lottery which pays a prize of 5 million if they win, and receiving 1 million with certainty. The possible options were:

1. Prefer lottery ticket

2. Prefer 1 million certain amount 3. Indifferent 4. Don’t know One of these option had to be selected for each of the following scenarios:

- A lottery ticket which has a 10% chance of paying 5 million, vs receiving 1 million with certainty

- A lottery ticket which has a 20% chance of paying 5 million, vs receiving 1 million with certainty

- A lottery ticket which has a 30% chance of paying 5 million vs receiving 1 million with certainty

- A lottery ticket which has a 50% chance of paying 5 million vs receiving 1 million with certainty  A lottery ticket which has a 70% chance of paying 5 million vs receiving 1 million with certainty  A lottery ticket which has a 90% chance of paying 5 million vs receiving 1 million with certainty

- A lottery ticket which has a 100% chance of paying 5 million vs receiving 1 million with certainty

We have created an indicator variable for each scenario-choice combination.

_Motivation_ . Participants were asked in the baseline interview specify how important it was for them wanting to run their own business rather than being employed for a salary. The possible options were:

1. Very Important

2. Important 3. Somewhat important 4. Not important 5. Don’t know 6. Not applicable The possible motivations were:
- The possibility to care for children or other family members while I work

- To be my own boss and not have to rely on others

- Flexibility in hours of work

- The possibility to earn more money

- Difficulty in finding a wage job

- The possibility that my business will grow in the future

- Running my own business is less boring/more exciting than salary/wage work

We have created an indicator variable for each motivation-evaluation combination.

_Self-confidence._ Participants were asked in the baseline interview to rate their confidence in carrying on certain tasks. The possible options were:

1. Not at all confident

2. Somewhat confident 3. Quite confident 4. Very confident 5. Don’t know 6. Not applicable The tasks listed were:

- Sell a new product or service to a new client

- Find and hire good employees to expand your business

- Obtain a good price for buying business inputs

- Persuade a bank to lend you money to start your business

- Estimate accurately the costs of a new project

- Manage an employee who is not a family member

- Correctly value your business if you wanted to sell it

- Resolve a difficult dispute with a business client in another city

- Identify when it is time to stop making or selling a product that is not selling well

We have created an indicator variable for each task-rating combination.

_Registration type_ indicates whether the participant already owned a business before applying to YouWIN!

_Registration time_ records the time when participants submitted their initial application. This variable is recorded in Unix epoch time.
_Assets._ Participants were asked in the baseline interview to specify the number of items owned by their households for each of the followings:

- Sewing machine

- Gas cooker

- Stove (electric)

- Stove gas (table)

- Stove (kerosene)

- Fridge

- Freezer

- Air conditioner

- Washing Machine

- Electric Clothes Dryer

- Bicycle

- Motorbike

- Cars and other vehicles

- Generator

- Hi-Fi (Sound System)

- Microwave

- TV Set

- Computer

- DVD Player

- Satellite Dish

We have kept these variables as categorical. In addition, we have also generated two indicator variables equal to one if the participant’s household had an Internet connection in the house and owned any land, respectively.

_Business sector_ . Participants were asked in the baseline interview to select among 32 categories the sector that their current business (for existing owners) or planned business (for start-ups) was going to be in. We have generated an indicator variable for each of the possible sectors.

_Employee education_ . Participants who already owned a business at the time of the initial application were asked to specify how many individuals they employed by educational level. The possible categories were the following:
- No Education

- Primary Education

- Secondary Education

- Tertiary Education

_Business type._ We have created a series of binary variables to indicate whether the participants who already owned a business had registered this business with the Corporate Affair Commission and if yes, which kind of status they had chosen among:

- Limited liability company

- Business name

- Other

_Tax paid_ records the amount paid in taxed in the year previous to the initial application by the individuals who already owned a business at that time.

_Loans_ are a series of binary variables indicating - for the participants who already owned a business at the time of the initial application – whether this business had a loan at that time from any of the following:

- Bank

- Microfinance Organization

- Cooperative society

_Business Challenges_ are a series of binary variables indicating - for the participants who already owned a business at the time of the initial application – whether they identified the following as major challenges for their business:

- Lack of funds to start a business

- Lack of the required skills

- Lack of business experience

- Lack of interest in business

_Projected outcomes_ record the expectations of participants who already had a business at the time of the initial application. Information was recorded regarding:

- Money needed to expand business

- Number of new people the business will employ
- Projected annual turnover

# **A.1.3 Text Analysis**

_Business name_ indicates whether the business name includes the owner’s surname in it. We have constructed three related measures:

- An indicator variable equal to one in case of perfect match between the participant’s surname and the firm name.

- An indicator variable equal to one in case the participant’s name, surname, initials or subsets of the name or surname are in the firm name.

- A measure of the Levenshtein distance between participant’s surname and the firm name using the Stata command _strdist_ .

_Length_ records the length of the letter of intent in the initial applications (truncated at 244 characters). Individuals were asked to answer the following questions:

- Why do you want to be an Entrepreneur?

- How did you get your business Idea?

- Where do you Intend to locate your Business and Why?

- Why will you succeed?

_Length First Answer_ records the length of first answer “Why do you want to be an Entrepreneur?” in the initial applicants (truncated at 244 characters).

_Capital Letter_ records whether the letter of intent in the initial application was written all in capital letters.

_Employer of labor_ indicates whether the letter of intent includes the words “employer of labor” or employer of labour”.

_No competition_ indicates whether the participant’s answer to the question “What is the competition? How will it change and why are you better?” contains “no competition”.

# **A.1.4 Business Plan**

_First Round Score_ records the total marks (out of 100) given to the initial application. The evaluation criteria concerned the quality and viability of the business idea, the potential for survival and job creation, the applicant’s ability, business skills, passion and commitment. Applications were evaluated by the Enterprise Development Center (EDC). To ensure an impartial scoring process, all names and identifying information
were removed from the applications. Each application was evaluated in around 10 minutes. There was a conscious decision to favor existing businesses.

_Business Plan Total Score_ is the total score (out of 100 points) assigned by the markers to the applicant’s business plan. We also have the numbers of scores assigned based on each of the following criteria:

- Ability to manage (10 points)

- Articulation (10 points)

- Capstone (10 points)

- Financial sources (5 points)

- Financial sustainability (10 points)

- Financial viability (5 points)

- Impact job creation (25 points)

- Risk assessment (10 points)

- Time to market (5 points)

- Understanding (10 points)

_Financial details_ . A group of variables has been created to record the amount sought by the applicants in their business plans, as well as their stated financial contribution to the business.

_Final status_ is a series of indicator variables recording whether the final status of the participants is national merit winners, zonal merit winners, ordinary winners, disqualified.

_Final award_ is the total award actually paid to the YouWin winners (in logarithms).

# **A.1.5 Second Follow-up Predictors**

_Digit-span Recall._ Participants were showed a card with four numbers marked on it, for ten seconds. The interviewer then waited for 10 seconds before asking them to repeat the number in reverse order. If they got the numbers correct at the first attempt, the procedure was repeated with 5 numbers, otherwise the interviewer moved to the next section, and so on until 11 numbers. Participants were given an example with 3 numbers. If the interview was conducted over the phone, the interviewer read the numbers slowly over the phone, and then asked the respondent to repeat back in reverse order. We have created an indicator variable to indicate whether the respondent succeed in each of the recall exercises, as well as an indicator for whether the interview was conducted over the phone (91 cases).

_Raven Test._ Participants were showed a series of pictures. Each picture had figures in two rows and three columns. One figure was missing. They had to find the missing part required to complete a pattern (among
8 options). They were given 5 minutes. During that time, they were asked to do many as possible (out of 12). They were told that they could skip a picture and come back later. Two examples were given in order to clarify the task. This test was conducted only for individuals interviewed in person, not by phone. We have created an indicator variable for each option-picture combination.

_Grit_ . Participants were asked how much they identified with certain descriptions of their personality. The possible answers were:

1. Very much like me

2. Mostly like me 3. Somewhat like me 4. Not much like me 5. Not like me at all The statements were:

- I have overcome setbacks to conquer an important challenge

- New ideas and projects sometimes distract me from previous ones.

- My interests change from year to year.

- Setbacks don't discourage me.

- I have been obsessed with a certain idea or project for a short time but later lost interest.

- I am a hard worker.

- I often set a goal but later choose to pursue a different one.

- I have difficulty maintaining my focus on projects that take more than a few months to complete.

- I finish whatever I begin.

- I have achieved a goal that took years of work.

- I become interested in new pursuits every few months.

- I am diligent.

We have created an indicator variable for each option-description combination.

# **A.1.6 McKenzie Predictors**

In addition to gender and age, the following variables have been generated by combining some of the predictors described in the previous sections in order to match the variables used in McKenzie (2015).

_Graduate Education_ is an indicator variable equal to one if the respondent’s highest educational level is higher than college. When such information was missing, we have looked at the same variable from the first follow-up (if available).
_Lottery choice_ is an indicator variable equal to one if the respondent selected the lottery ticket when there was a 90% chance of winning 5 million instead of receiving 1 million with certainty. Around 55% of the respondents selected this option.

_Wealth_ has been constructed using principal component analysis on the asset variables described in Section A.1.2.

_Grit_ is an average indicator, combining the twelve grit measures described in Section A.1.5.

_Ability_ has been constructed by applying the principal component analysis to the maximum correct numbers in the digit-span recall exercises described in Section A.1.5, as well as the number of correct answers in the Raven test described in Section A.1.5.

_Business Sectors_ are three indicator variables reporting whether in the applicants reported that the sector of their current business (for existing owners) of their planned business (for start-ups) was:

- Agricultural Production

- Manufacturing (tailoring, dressmaking or shoe making; furniture; craft; other manufacturing)

- IT and Computer Services

# **A.1.7 Fafchamps-Woodruff Predictors**

In addition to gender and age, the following variables have been generated by combining some of the predictors described in the previous sections in order to match the variables used in Fafchamps and Woodruff (2017). It is important to stress that in this case it was not possible to obtain a perfect match with their predictors since some variables such as financial literacy or trust were not available in our dataset.

_Retail_ is an indicator variable reporting whether in the applicants reported that the sector of their current business (for existing owners) of their planned business (for start-ups) was market trader, street vendor/hawker, shop, street kiosk, or other retail trade.

_Self-confidence_ has been constructed by combining through principal component analysis the level of confidence reported by the applicant in carrying out the business tasks described in Section A.1.2.

_Motivation_ has been constructed by combining through principal component analysis the levels of importance assigned by the applicant to the different reasons to start a business instead of being employed for a salary described in Section A.1.2.

_Happiness_ was recorded in the first follow-up. Participants were asked the following question:
Imagine for a minute that you are living the best life you can possibly imagine. Now imagine that your life is the worst it could possibly be. Imagine a ladder with 10 steps. Suppose we say that the top of the ladder (step 10) represents the best possible life for you and the bottom (step 1) represents the worst possible life for you. Which step on the ladder best represents where you personally stand at the present time?

_Optimism_ was also recorded in the first follow-up. After the question above, participants were asked the following: Think about your life five years from today. Which step best represents where you personally will be on the ladder five years from now?

_Ability_ has been obtained by combining three indicators through principal component analysis: the number of correct answers in the Raven test described in Section A.1.5, the maximum correct numbers in the digitspan recall exercises described in Section A.1.5, and the educational level, i.e. a categorical variable constructed from the educational variables described in Section A.1.2 _Credit_ has been constructed by combining three indicators through principal component analysis: whether, at the time of the application, the business has a loan from a bank, a microfinance organization, or a cooperative society. Note that this question was asked only to applicants who already owned a business.
# **A.2 Robustness Tables**

**Table A1: Can Human Experts Predict Sales After 3 Years?**

|||Non-Winner<br>|s<br>||Wi<br>|nners<br>||
|---|---|---|---|---|---|---|---|
||(1)<br>|(2)<br>|(3)<br>|(4)<br>|(5)<br>|(6)<br>|(7)<br>|
||Judges<br>|McKenzie<br>|Fafchamps<br>|Judges<br>|Judges<br>|McKenzie<br>|Fafchamps<br>|
|Business Plan score|-0.0056<br>|-0.0138<br>|-0.0147<br>|0.0011<br>|-0.0065<br>|-0.0116<br>|-0.0110<br>|
||(0.0196)|(0.0204)|(0.0199)|(0.0127)|(0.0131)|(0.0148)|(0.0144)|
|Female||<br>-2.1786<sup>***</sup>|<br>-1.8214<sup>***</sup>|||<br>-0.7631|<br>-0.6095|
|||(0.5237)<br>|(0.5170)<br>|||(0.4225)|(0.4212)|
|Age||0.1618<sup>***</sup><br>|0.1194<sup>**</sup><br>|||0.0318<br>|0.0254<br>|
|||(0.0441)|(0.0437)|||(0.0370)|(0.0363)|
|Graduate Education||-1.0237||||-0.1670||
|||(0.8563)||||(0.5978)||
|Worked abroad||0.6766<br>||||-0.8457<br>||
|||(0.8431)||||(0.5784)||
|Lottery choice||<br>-0.0111||||<br>-0.3200||
|||(0.3986)||||(0.2905)||
|Wealth||0.0144||||0.0850||
|||(0.0779)||||(0.0444)||
|Grit||-0.3675||||-0.0154||
|||(0.3850)||||(0.2711)||
|Ability||0.7581<sup>**</sup>||||-0.1583||
|||(0.2302)||||(0.1617)||
|Agriculture||-0.3835||||-0.3796||
|||(0.4910)||||(0.3675)||
|IT||-0.6148||||-0.4697||
|||(0.6905)||||(0.4347)||
|Manufacture||0.5216|0.8233|||-0.1377|0.1598|
|||(0.5313)|(0.4768)|||(0.3860)|(0.3632)|
|Retail|||1.0215<br>||||1.7465<sup>**</sup><br>|
||||(1.0625)<br><sup>**</sup>||||(0.5318)<br>|
|Self-confidence|||0.3307||||0.1413|
||||(0.1255)||||(0.0937)|
|Motivation|||<br>0.0799||||<br>-0.1323|
||||(0.1469)||||(0.1002)|
|Happiness|||0.1005||||0.1112|
||||(0.1630)||||(0.1281)|
|Optimism|||0.4658<sup>*</sup>||||-0.1384|
||||(0.1862)||||(0.1348)|
|Ability|||<br>0.5508<sup>*</sup>||||<br>-0.1555|
||||(0.2209)||||(0.1503)|
|Credit|||<br>0.2675||||<br>0.0431|
||||(0.1611)||||(0.1960)|
|Eii|30799<sup>***</sup>|24890<sup>***</sup>|09603|10683<sup>***</sup>|10150<sup>***</sup>|07589<sup>*</sup>|04584|
|xstng|.|.|.|.|.|.|.|
||(0.5051)|(0.5406)|(0.6063)|(0.2874)|(0.2866)|(0.3621)|(0.4279)|
|Log(Total award paid)|||||0.5869<sup>*</sup>|0.5552|0.5839<sup>*</sup>|
||||||(0.2930)|(0.2944)|(0.2955)|
|Regional Indicators|No|Yes|Yes|No|No|Yes|Yes|
|Sample Size|1059|1059|1059|1047|1047|1047|1047|
|<br>P-value|,|,<br>0.000|,<br>0.000|,|,|,<br>0.176|,<br>0.005|
|AdjustedR<sup>2</sup>|0.037|0.098|0.138|0.011|0.020|0.024|0.030|
|Robust Standard Errors in<br>Aendix A1 for variable|parentheses.<br>definitions|*, **, *** de<br>eendent vari|note significance<br>able is inverse h|at the 10, 5,<br>erbolic sine|and 1 perce<br>of sales 3 ea|nt levels respe<br>s after ali|ctively. See<br>n coded as|
|pp .<br>zero for applicants witho<br>Fafchamps and Woodruff|.<br>ut an operati<br>(2017) (abbre|p<br>ng firm. P-val<br>viated as FaWo|y<br>ue is for testing<br>) are jointly zero|p<br>that set of co<br>.|y<br>ntrols added|ppy<br>by MacKenz|g,<br>ie (2015) or|
# **Table A2: Robustness to Including Judge Fixed Effects**

|||Non-win<br>|ners<br>|||Winne<br>|rs<br>||
|---|---|---|---|---|---|---|---|---|
||(1)<br>Survival|(2)<br>Employment|(3)<br> Sales|(4)<br>Profits|(5)<br>Survival|(6)<br> <br>Employment|(7)<br> Sales|(8)<br>Profits|
|**Panel A: Without Judg**|**e Fixed Effec**|**ts**|||||||
|Business Plan score|-0.0002|-0.0021|-0.0056|<br>-0.0041|0.0000|0.0897**|0.0011|<br>0.0009|
||(0.0014)|(0.0168)|(0.0196)|<br>(0.0180)|(0.0006)|<br>(0.0310)|(0.0127)|<br>(0.0127)|
|Sample Size|1,078|1,078|1,059|1,059|1,062|1,062|1,047|<br>1,047|
|R<sup>2</sup>||0.014|0.039|0.036||0.023|0.013|0.013|
|**Panel B: With Judge F**|**ixed Effects**||||||||
|Business Plan score|-0.0012|0.0015|-0.0135|<br>-0.0150|-0.0004|<br>0.1253***|-0.0059|<br>-0.0051|
||(0.0016)|(0.0175)|(0.0217)|<br>(0.0197)|(0.0008)|<br>(0.0338)|(0.0145)|<br>(0.0142)|
|Sample Size|1,067|1,078|1,059|1,059|963|1,062|1,047|<br>1,047|
|R<sup>2</sup>||0.028|0.058|0.058||0.051|0.053|0.046|

Robust standard errors in parentheses, *, **, *** denote significance at the 10, 5, and 1 percent levels respectively. See Appendix A.1. for variable definitions. Employment, Sales and Profits are unconditional outcomes, coded as zero for non-operating firms
# **Table A3: Do Sub-Scores of Judges Predict Outcomes?**

|||Non-Win|ners|||Winner|s||
|---|---|---|---|---|---|---|---|---|
||(1)<br>Survival|(2)<br>Employment|(3)<br>Sales|(4)<br>Profits|(5)<br>Survival|(6)<br>Employment|(7)<br>Sales|(8)<br>Profits|
|Score: ability to manage|0.030<sup>***</sup>|0.401<sup>***</sup>|0.348<sup>***</sup>|0.294<sup>**</sup>|0.002|0.231|0.094|0.049|
||(0.010)|(0.119)|(0.131)|(0.123)|(0.005)|(0.176)|(0.089)|(0.088)|
|Score: articulation|-0.003|-0.092|-0.168|-0.146|0.002|-0.157|0.173|0.159|
||(0.013)|(0.140)|(0.179)|(0.165)|(0.006)|(0.222)|(0.117)|(0.116)|
|Score: capstone|-0.008|-0.176|-0.196|-0.035|-0.007|0.187|-0.189|-0.220<sup>*</sup>|
||(0.014)|(0.198)|(0.191)|(0.175)|(0.007)|(0.236)|(0.124)|(0.125)|
|Score: financial sources|0.011|0.076|0.048|0.024|-0.003|0.425<sup>*</sup>|-0.027|0.039|
||(0.012)|(0.132)|(0.158)|(0.146)|(0.006)|(0.222)|(0.106)|(0.107)|
|Score: financial sustainability|<br>-0.011|<br>-0.072|<br>-0.163|<br>-0.115|<br>-0.004|<br>-0.106|<br>-0.015|<br>-0.006|
||(0.011)|(0.128)|(0.146)|(0.138)|(0.006)|(0.172)|(0.087)|(0.089)|
|Score: financial viability|<br>0.008|<br>0.007|<br>0.212|<br>0.151|<br>0.000|<br>-0.151|<br>-0.107|<br>-0.133|
||(0.017)|(0.191)|(0.224)|(0.211)|(0.008)|(0.315)|(0.139)|(0.140)|
|Score: job creation|0.000|0.068<sup>*</sup>|0.008|0.002|-0.001|0.184<sup>***</sup>|-0.012|0.020|
||(0.003)|(0.035)|(0.040)|(0.037)|(0.001)|(0.058)|(0.023)|(0.025)|
|Score: risk assessment|-0.002|-0.046|-0.012|0.010|-0.001|-0.044|-0.089|-0.063|
||(0.007)|(0.081)|(0.093)|(0.087)|(0.003)|(0.140)|(0.059)|(0.060)|
|Score: time to market|-0.003|-0.057|-0.022|-0.080|0.005|-0.322|0.090|0.098|
||(0.010)|(0.122)|(0.141)|(0.131)|(0.006)|(0.236)|(0.101)|(0.102)|
|Score: understanding|<br>-0.010|<br>-0.130|<br>0.007|<br>-0.082|<br>0.004|<br>-0.098|<br>0.061|<br>0.001|
||(0.012)|(0.138)|(0.162)|(0.147)|(0.006)|(0.193)|(0.107)|(0.109)|
|Existing|<br>0.174<sup>***</sup>|<br>1.042<sup>*</sup>|<br>2.566<sup>***</sup>|<br>2.429<sup>***</sup>|<br>0.028|<br>2.130<sup>***</sup>|<br>0.688<sup>**</sup>|<br>0.736<sup>**</sup>|
||(0.042)|(0.551)|(0.554)|(0.518)|(0.019)|(0.636)|(0.340)|(0.340)|
|Log(Total award paid)|||||0.026<sup>*</sup><br>(0.015)|3.617<sup>***</sup><br>(0.433)|0.580<sup>*</sup><br>(0.296)|0.386<br>(0.300)|
|Constant||3.500<sup>***</sup>|6.548<sup>***</sup>|5.894<sup>***</sup>||<br>-49.262<sup>***</sup>|<br>2.261|<br>4.182|
|||(0.956)|(1.153)|(1.060)||(6.892)|(4.566)|(4.630)|
|Sample Size|1,078|1,078|1,059|1,059|1,062|1,062|1,047|1,047|
|<br>Adjusted R<sup>2</sup>||0.020|0.038|0.033||0.066|0.021|0.019|
|P-valuethatscores jointlyzero|0.337|0.039|0.415|0.624|0.947|0.056|0.464|0.549|

Robust Standard errors in parentheses. *, **, *** denote significance at the 10, 5, and 1 percent levels respectively. P-value is for F-test that all ten sub-scores are jointly unrelated to the business outcome three years after application.
**Table A4: Predicting Outcomes for Non-Winners Conditional on Operating a Firm**

|||Employment|||Sales|||Profits||
|---|---|---|---|---|---|---|---|---|---|
||(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|
||Judges|McKenzie|FaWo|Judges|McKenzie|FaWo|Judges|McKenzie|FaWo|
|Business Plan score|-0.0015|0.0019|0.0144|-0.0057|-0.0013|-0.0016|-0.0035|0.0082|0.0087|
||(0.0237)|(0.0260)|(0.0260)|(0.0137)|(0.0149)|(0.0144)|(0.0136)|(0.0152)|(0.0147)|
|Female||-0.0764|0.2189||-0.6269|-0.5693||-0.3310|-0.1299|
|||(0.8279)|(0.8664)||(0.4890)|(0.5039)||(0.4833)|(0.4891)|
|Age||0.0643|0.0482||0.0440|0.0327||0.0042|-0.0032|
|||(0.0586)|(0.0572)||(0.0361)|(0.0356)||(0.0349)|(0.0352)|
|Graduate Education||1.7434|||-1.0594|||-1.0160||
|||(1.3193)|||(0.7996)|||(0.7678)||
|Worked abroad||0.1256|||1.0070|||1.2113||
|||(1.2563)|||(0.7131)|||(0.6602)||
|Lottery choice||1.1154<sup>*</sup>|||0.4861|||0.3256||
|||(0.4963)|||(0.3387)|||(0.3306)||
|Wealth||0.1906|||0.0734|||0.0104||
|||(0.1067)|||(0.0624)|||(0.0564)||
|Grit||<br>-0.3856|||<br>0.0174|||<br>-0.1183||
|||(0.5085)|||(0.3307)|||(0.3238)||
|Ability||0.1527|||0.2689|||0.2051||
|||(0.2741)|||(0.1568)|||(0.1705)||
|Agriculture||-0.4653|||0.3281|||0.1823||
|||(0.7257)|||(0.4211)|||(0.4023)||
|IT||-1.6241<sup>*</sup>|||0.1511|||-0.3364||
|||(0.6732)|||(0.4793)|||(0.5839)||
|Manufacture||-0.9482|-0.5424||0.7329|0.6019||0.3223|0.2243|
|||(0.6473)|(0.6012)||(0.4195)|(0.3652)||(0.4075)|(0.3458)|
|Retail|||<br>0.4425|||<br>0.6653|||<br>0.3885|
||||(1.1477)|||(0.7906)|||(0.7084)|
|Self-confidence|||0.0169|||0.1383|||0.0411|
||||(0.1402)|||(0.1236)|||(0.1137)|
|Motivation|||-0.0658|||-0.2129|||-0.1952|
||||(0.1507)|||(0.1277)|||(0.1190)|
|Happiness|||0.3065|||0.2219|||0.2249|
||||(0.1757)|||(0.1193)|||(0.1213)|
|Optimism|||-0.0798|||0.1290|||0.0990|
||||(0.2457)|||(0.1960)|||(0.1752)|
|Ability|||0.1236|||0.2375|||0.1297|
||||(0.2431)|||(0.1452)|||(0.1518)|
|Credit|||<br>-0.1577|||<br>0.1267|||<br>0.1257|
||||(0.1616)|||(0.0724)|||(0.0644)|
|Existing|0.3872|0.6218|<br>-0.0783|0.8039<sup>*</sup>|0.8170<sup>*</sup>|<br>0.2834|0.6643|0.9218<sup>*</sup>|<br>0.3345|
||(0.5611)|(0.5825)|(0.7337)|(0.3335)|(0.3564)|(0.3922)|(0.3525)|(0.3893)|(0.4224)|
|Regional Indicators|No|Yes|Yes|No|Yes|Yes|No|Yes|Yes|
|Sample Size|630|630|630|611|611|611|611|611|611|
|P-value||0.073|0.643||0.060|0.000||0.442|0.003|
|AdjustedR<sup>2</sup>|-0.002|0.020|0.004|0.007|0.022|0.038|0.004|0.013|0.040|

Robust Standard Errors in parentheses. *, **, *** denote significance at the 10, 5, and 1 percent levels respectively
# **A.2.1 Other Predictors Suggested by the Literature**

In addition to the models fitted in Tables 1-3 and A1, we examine whether there are differences in outcomes according to three other characteristics suggested by recent literature. The first is the reason for going into entrepreneurship. Acs (2006) distinguishes between “necessity entrepreneurship”, in which individuals go into entrepreneurship because they do not have better options; and “opportunity entrepreneurship”, in which individuals start new businesses to exploit better opportunities. Schoar (2010) makes a similar distinction between subsistence and transformative entrepreneurs. We compare individuals who rated “difficulty in finding a job” important or very important to those who rates it somewhat important, not important, or don’t know/not applicable. In panel A of Table A5 we see individuals who go into self-employment for necessity reasons have worse outcomes for all four outcome measures among non-winners, but that there are no statistically significant differences among winners.

Second, the entrepreneurship literature has recently given attention to the signal contained in the decision to name a firm after the owner. Belenzon et al. (2017) view this as raising the reputation costs of poor performance, causing the owners to exert more effort and have more successful outcomes. In contrast, Guzman and Stern (2014) view eponymous firms as a signal for lower entrepreneurial quality, being more likely to signal a family-owned lifestyle business than an ambitious growth-oriented firm. We only have the firm name for the winning firms, and find 4 percent of firms to have a name that is a match to the applicant’s surname, and 11 percent to be a partial match to the surname, first name, or initials. In line with the mixed view of the signal contained in the choice of name, we find no significant differences in firm outcomes for winning firms named after the owner compared to those not (panel B of Table A5).

Thirdly, we consider whether applicants were late to enroll. The original deadline was November 18, 2011 at midnight, and this was then extended for one more week. 10.5 percent of individuals enrolled after the deadline. This is motivated by Banerjee and Duflo (2014), who find that students who enrolled late for an online course perform worse, which they view as likely due to these individuals being more disorganized and less able to complete tasks on time. Panel C of Table A5 shows that we see no significant difference in outcomes for non-winners according to this status, but we do find that winners who applied late have significantly lower odds of business survival, and earn less sales.
# **Table A5: Other Predictors**

|||Non-Win<br>|ners<br>|||Winne<br>|rs<br>||
|---|---|---|---|---|---|---|---|---|
||(1)<br>Survival|(2)<br> <br>Employment|(3)<br> Sales|(4)<br>Profits|(5)<br>Survival|(6)<br> <br>Employment|(7)<br> Sales|(8)<br>Profits|
|**Panel A: Reason for Going**|**into Busine**|**ss**|||||||
|Necessity Entrepreneur|-0.0757*|<br>-0.8930*|-0.8349|<br>-0.5946|0.0314|-0.3666|0.6762*|<br>0.5239|
||(0.0384)|<br>(0.3808)|(0.5440)|<br>(0.5018)|(0.0239)|<br>(0.6538)|(0.3343)|<br>(0.3618)|
|Sample Size|1,078|1,078|1,059|<br>1,059|1,062|1,062|1,047|<br>1,047|
|R<sup>2</sup>||.0175|.0412|.0375||.0127|.0163|.0145|
|**Panel B: Firm is Named Aft**|**er the Own**|**er**|||||||
|Eponymous Firm (Narrow)|||||-0.0262|-1.1963|0.1178|<br>-0.4042|
||||||(0.0322)|<br>(1.0055)|(0.7134)|<br>(0.8179)|
|Sample Size|||||1,051|1,051|1,036|<br>1,036|
|R<sup>2</sup>||||||.0121|.0136|<br>.0124|
|Eponymous Firm (Broad)|||||0.0198|0.0461|0.5464|<br>0.3037|
||||||(0.0262)|<br>(0.8512)|(0.3768)|<br>(0.4190)|
|Sample Size|||||1,051|1,051|1,036|<br>1036|
|R<sup>2</sup>||||||.0116|.015|.0126|
|**Panel C: Time of Registrati**|**on for the C**|**ompetition**|||||||
|Late Entrant to Competition|0.0024|-0.4967|-0.0396|<br>-0.1541|-0.0606**|2.3158|-1.3276*|<br>-0.8912|
||(0.0440)|<br>(0.4364)|(0.6013)|<br>(0.5623)|(0.0185)|<br>(1.7820)|(0.5524)|<br>(0.5171)|
|Sample Size|1,078|1,078|1,059|1,059|1,062|1,062|1,047|<br>1,047|
|R<sup>2</sup>||.0152|.0391|.0364||.0172|.0211|<br>.0162|

Robust standard errors in parentheses. *, **, and *** denote significance at the 10, 5, and 1 percent levels respectively. Necessity entrepreneur defined as saying that difficulty finding a job is a very important or important reason for wanting to start a business. Eponymous firm (narrow) requires surname of owner to be in firm name. Eponymous firm (broad) requires partial match between firm name and owner's first name, surname, or initials. Late entrants are those who enrolled after the original deadline.
# **Table A6: Variables most commonly selected by Boosting**

|||Total Emp|loyment|Sale|s|Pro|fits|
|---|---|---|---|---|---|---|---|
|Predictors|Count|NoWin|Win|NoWin|Win|NoWin|Win|
|# Workers with tertiary education|23|24.79|22.16|2.34|2.03|2.26|5.60|
|Registration Time|21|2.50|7.84|2.86|5.03|3.42|3.88|
|# Workers with secondary education|21|5.86|0.38|2.35|6.71|2.25|10.23|
|Age|21|1.78|34.90|5.45|2.69|5.41|3.01|
|Taxes paid|18|0|3.05|1.58|3.45|2.00|3.37|
|Projected annual turnover|18|6.93|4.92|3.46|10.67|1.92|7.15|
|Initial application score|18|4.54|0.38|0.87|2.82|0.83|4.33|
|Length first answer application|17|2.51|1.30|2.50|3.42|2.33|4.27|
|# Brothers|17|0.93|1.73|1.59|3.32|1.31|2.13|
|Business plan scores|17|2.03|0.79|1.48|1.03|2.21|2.80|
|# Sisters|14|2.55|0.49|1.59|0.55|1.63|2.89|
|# Workers with primary education|14|7.70|4.37|1.46|1.16|2.87|1.38|
|Amount needed to expand|14|0|3.27|2.75|2.52|1.56|3.43|
|Length all answers application|13|0|0|1.63|0.71|2.45|2.30|
|# Children|12|0|2.69|0|3.55|0|6.32|
|YouWin: Total award paid|12|0|1.23|1.38|3.01|1.93|4.04|
|Business plan scores: component understanding|11|0|0|2.09|1.24|2.25|2.15|
|Business plan scores: component articulation|11|4.44|0|0.51|2.22|0.49|7.87|
|# Years living abroad|11|0|0|0.92|2.93|0.32|4.29|
|# Freezers|11|0|0|1.45|3.66|0.97|3.10|
|Gender|11|0|0|0.75|3.02|1.17|4.52|
|Raven test - Q11A1 (correct)|11|0|0|1.85|0.55|2.21|1.87|
|Projected employment|10|0|0|0.47|5.82|1.32|2.62|
|Marketing training|10|0|2.66|1.22|1.90|1.44|2.58|
|Raven test – Q1A8 (correct)|10|0|0|1.90|0|3.32|0|

Notes: NoWin and Win denote Non-winners and Winners respectively. Count denotes the number of times (out of 30) that the variable was chosen by boosting in predicting employment, sales, and profit outcomes. The remaining columns then show the average influence the variable has on the prediction.
# **Table A7: Variables most commonly selected by LASSO**

|Predictors|Count|
|---|---|
|[YouWin: Total award paid]<sup>3</sup>|5|
|[Business Sector: Shop] x [Raven test: Q1A8 (correct)]|4|
|[Grit: often new interests] x [Application includes “employer of labor”]|4|
|[# gas stoves]<sup>3</sup>|4|
|[Raven test: Q9A9 (incorrect)] x [Loan from cooperative missing]|3|
|[Raven test: Q11A9 (incorrect)] x [Loan from cooperative missing]|3|
|[# Workers with secondary education missing] x [Length all answers application]|3|

Notes: Count denotes the number of times (out of 30) that the variable was chosen by LASSO in predicting employment, sales, and profit outcomes.
# **Table A8: How much difference would this make for an investor taking a profit share in the top 100 firms?**

|||Model|Predictors|Non-Winners|Winners|
|---|---|---|---|---|---|
||1|OLS|Random|3,943,960|5,319,813|
||2|OLS|Application Score|4,958,110|8,774,139|
|Judges|3|OLS|Business Score|2,794,840|10,093,147|
||4|OLS|SubScore|3,631,660|4,904,253|
||5|OLS|Gender|3,832,820|9,852,094|
||6|OLS|Age|5,723,710|9,474,214|
|Sinle|7|OLS|Necessity Firm|2,971,470|7,216,015|
|g<br>Pdit|8|OLS|Grit|6,715,710|9,731,715|
|recor|9|OLS|Digit-span Recall|3,743,760|11,151,475|
||10|OLS|Raven Test|4,892,010|11,587,300|
||11|OLS|Registrationtime|4,308,850|6,969,259|
||12|OLS|McKenzie|5,008,910|10,140,434|
|EconomistModels|13|OLS|FaWo|6,395,860|22,344,000|
||14|OLS|FaWo+ BusScores|5,873,460|22,217,115|
||15|Post-LASSO|All|4,009,350|4,255,990|
|Machine-Learning|16|SVR|All|5,597,660|4,542,629|
||17|Boosting|All|5,494,360|7,152,821|

Notes: Profits in Nigerian Naira (1 Nigerian Naira equals around 0.0028 US$). For each model, we have taken the predicted profits computed in Table 5, sort firms in the test sample based on these predicted values, selected the top 20 firms (equivalent to around 10% of the test sample), summed the profits in these selected firms, and repeated this procedure for each of the 5 folds. We have reported the average total profits in the 5 folds. Models are estimating separately for competition non-winners and winners. Judges use scores from competition judges to predict the outcome. Single predictor models use a single survey measure. The economist models are the models of McKenzie (2015) and Fafchamps and Woodruff (2017) (abbreviated as FaWo) used in Tables 1-3. The three machine learning models are based on 393 predictors, along with interaction terms in the case of post-LASSO. All models include a dummy for whether the applicant was for an existing firm compared to a new firm, and models for winners also control for the total award received (in logs).
# **A.3 Technical Appendix – Machine Learning**

The aim of this technical appendix is to provide a quick overview of the machine learning algorithms used in the paper. A more detailed and comprehensive review of the tools available to practitioners is provided by Hastie et al. (2009) as well as by Ng (2016).

Machine Learning is the science of getting computers to learn, without being explicitly programmed. Standard econometric techniques, e.g. regressions, are considered supervised algorithms. In other words, supervised algorithms are provided with a certain number of “right” answers, i.e. actual y associated with certain x, and are asked to produce other correct answers, i.e. to predict new y given other combinations of x. On the other hand, unsupervised learning algorithms derive a structure for the data without necessarily knowing the effect of the variables. For instance, we could cluster genes into groups that are somehow related by lifespan.

# **A.3.1 LASSO**

LASSO (Least Absolute Shrinkage and Selection Operator) reduces the magnitude of the selected predictors, while at the same time imposing a zero coefficient to all the unselected variables. Assuming linearity, a simple version of this estimator is the following: Where n is the number of observations, k is the number of variables (potentially larger than n), and λ is the regularization (or penalization) term. Note that the first summation is the OLS objective function, while the second part introduces a penalization and is defined as: The usual caveat in these techniques is to normalize with zero mean and unit variance all the variables, or to restrict their domain between zero and one, so that the regularization is not inflated by the different scale of the variables. As also mentioned in Guenther and Schonlau (2016), both methods should work correctly.

As discussed in Belloni et al. (2014), the underlining key assumption in this context is the so-called approximate sparsity condition: only a (small) subset of the variables among _xij_ have associated coefficients _β_ j different from zero. In is also important to note that a simple way to overcome the bias towards zero in the LASSO coefficients is to use the two-step post-LASSO procedure. In the first step, LASSO is used to
BQ) = argmin—-BeR* »)i=1 yj log F(x}B) + (1 — yi) log(t — FOB) | + Allblle

B(C) = argminBeR* C, »)i=1 yj costs (x/B) +(1 — yi) costo(x/B) | + Il
cost, (x; PB) = max(0,1 — x;B)

> costo (x;PB) = max(0,x;B6 — 1)

<!-- Start of picture text --> \<br>\<br>\<br>\<br>\<br>\<br>\<br>\<br>\/<br>\ /<br>\/<br>\ /<br>\/<br>\ /<br>\/<br>\A<br>\/<br>\ /<br>\—-— a7<br>KS f<br>C. _—<br>a )<br><!-- End of picture text --> Kepr(%1,X2) aye ~~Ilx1 — x2~~ = similarity(x,,x2) = exp ~~——352—)~~ = exp(—y]||x, — x2

o* o2 ~~-1~~ y= <u>202</u>
B(C) = aemeeR Cy »)i=1n yi cost, (Kj B) +(1 — y;) costo (KB) | + |IP lle

Kiinear (%1, X2) = X4X2 Kpoty (%1,X2) = (0+ yXx{X2 a Ksigmoia (x1, X2) = tanh(@ + yx1Xx2 M BoBaoBRE Hen k >, 6} j=1 yi(xjB) = MA — &) vi n », €j < C2 i=1
Cz KA Oi — xj)? n P(A) = argminbeRK )’ max{|y; — x16 = el, 0}

~~_~~ 2 seme” Ill lv; — {Bl <e Vi n ~~_~~ 2 a: ay ene” WIP + Cs)i=1 Gi +.

Vi Vi -XjB S E+;
Note that the slack variables can have different upper ��� and lower values ���, as well as to change across observations. As in the binary SVM, non-linear function can be estimated by including kernels in the above minimization problem.

We have implemented this algorithm in Stata using the command _svmachines_ (Guenther and Schonlau 2016). We have included all possible predictors as inputs variables. A grid-search has been used to find the optimal parameters, that is the ones that maximize accuracy or minimized the MSE in the CV sample. In case of continuous outcome variables, we have considered 13 possible values for each parameter (the penalization term _C3_ , the kernel smoothing parameter _γ_ , and the band width _ε_ ), as well as one kernel (Guassian) for a total of 2197 combinations (1x13<sup>3</sup> ). A similar grid-search has been conducted with the binary outcome variable survival, although in that case it has been necessary to calibrate only two parameters (the penalization term _C2_ and the kernel smoothing parameter _γ_ ) combined with the same two kernels. We have considered 16 possible values for each parameter, as well as two kernels (Guassian and sigmoid) for a total of 512 combinations (1x16<sup>3</sup> ). As mentioned in Section 4.3, SVM does not produce estimated probabilities: it only predicts on which side of the margin is an observation. Therefore, when predicting firms in the top 10% in Section 5, we have estimated pseudo-probabilities through Platt Scaling (Guenther and Schonlau 2016). However, in this case data are sorted randomly using the operating system random-number generator. As a consequence, different runs may give different results, potentially making results for SVM in Table 6 not exactly replicable in other computers.

# **A.3.3 Boosted Regression**

A simple version of Boosted Regression (or Boosting) can be illustrated using the AdaBoost algorithm for binary classifications:

1. All observations have the same weight<sup>1</sup> ��

2. Estimate the first classifier (e.g. a logistic regression) with the equally weighted data 3. Compute the classification errors, increase the weights of the misclassified observations 4. Estimate the second classifier with the new observation weights 5. Repeat steps 3-4 until you have M classifiers 6. Combine all the M individual classifiers by giving more weight to the classifiers with higher accuracy.
The peculiarity of this algorithm is that it usually obtains good performances by just combining simple classifiers such as regression trees. A regression tree partitions the covariate space into a set of rectangles and then fit a simple model (constant) to each rectangle. Therefore, the estimated function is just the average of the outcomes included in a particular rectangle. In other words, the partition can be described as a series of if-then statements, and it can be visualized by a graph that looks like a tree. The simplest possible tree is called tree stump and it contains only one split and two terminal nodes. Tree stumps tend to work well in boosting (Schonlau 2005).

Boosted regression is actually implemented using the algorithm introduced by Friedman et al. (2000) since they were able to reinterpret it in a likelihood framework, thus allowing statisticians to compare its objective function to the OLS and logistic one.

To avoid overfitting, it is also possible to introduce a shrinking parameter. This reduces the contribution of each additional tree, thus decreasing the impact of an over-fitted tree. The cost of this procedure is a substantial increase in computational time, since it is necessary to increase the number of iterations in order to compensate this smaller steps.

Bagging is an additional technique which we have used to reduce the variance of the final prediction without influencing the bias. At each iteration, we have used only a random subset of the train set (50% of it) to build the tree. The random subset has been increased to 90% of the training set when looking at the tails given the limited number of ones, i.e. firms in the top 10% of the outcome distribution.

This algorithm has been implemented in Stata using the command _boost_ (Schonlau 2005). A grid-search has been used to find the optimal parameters, that is the ones that maximize accuracy or minimized the MSE in the CV sample. We have considered trees with up to 8 splits, up to 150 iterations (i.e. number of trees) and two values for the shrinkage parameter (1 and 0.1), for a total of 2400 combinations (8*150*2). In addition to this, for the binary dependent variable survival we have also considered two error distribution (Gaussian and Logistic), thus computing 5800 combinations for each outcome-fold pair.
# **References used only in the Online Appendix**

Acs Z (2006) How is Entrepreneurship Good for Economic Growth. _Innov. Technol. Governance, Glob._ 1(1):97–107.

Banerjee A V., Duflo E (2014) (Dis)Organization and success in an economics MOOC. _Am. Econ. Rev._ 104(5):514–518.

Belenzon S, Chatterji AK, Daley B (2017) Eponymous entrepreneurs. _Am. Econ. Rev._ 107(6):1638–1655. Belloni A, Chen D, Chernozhukov V, Hansen C (2012) Sparse Models and Methods for Optimal Instruments with an Application to Eminent Domain. _Econometrica_ 80(6):2369–2429.

Belloni A, Chernozhukov V, Hansen C (2013) Inference on treatment effects after selection among highdimensional controls. _Rev. Econ. Stud._ 81(2):608–650.

Belloni A, Chernozhukov V, Hansen C (2014) High-Dimensional Methods and Inference on Structural and Treatment Effects. _J. Econ. Perspect._ 28(2):29–50.

Fafchamps M, Woodruff C (2017) Identifying Gazelles: Expert Panels vs. Surveys as a Means to Identify Firms with Rapid Growth Potential. _World Bank Econ. Rev._ 31(3):670–686.

Friedman J, Hastie T, Tibshirani R (2000) Additive Logistic Regression: A Statistical View of Boosting. _Ann. Stat._ 28(2):337–407.

Guenther N, Schonlau M (2016) Support Vector Machines. _Stata J._ 16(4):917–937.

Guzman J, Stern S (2014) Nowcasting and placecasting entrepreneurial quality and performance. _MIT Work. Pap._ Hastie T, Tibshirani R, Friedman J (2009) _The Elements of Statistical Learning: Data Mining, Inference, and Prediction_ Second Edi. (Springer).

McKenzie D (2015) Identifying and Spurring High-Growth Entrepreneurship: Experimental Evidence from a Business Plan Competition. _World Bank Policy Res. Work. Pap._ 7391.

Ng A (2016) Machine Learning. _Coursera_ . Retrieved (January 1, 2016), https://www.coursera.org/learn/machine-learning.

Schoar A (2010) The Divide between Subsistence and Transformational Entrepreneurship. Lerner J, Stern S, eds. _Innov. Policy Econ. Vol. 10_ . (NBER, Cambridge, MA), 75–81.

Schonlau M (2005) Boosted regression (boosting): An introductory tutorial and a Stata plugin. _Stata J._ 5(3):330–354.