metadata: authors: 0: author: Dinarte Diaz,Lelys Ileana
1: author: Ferreyra,Maria Marta
2: author: Urzua,Sergio
3: author: Bassi,Marina
count: Latin America,Caribbean
docna: 0: docna: What Makes a Program Good ? Evidence from
            Short-Cycle Higher Education Programs in Latin America and
            the Caribbean
docty: Policy Research Working Paper
subtopic: Rural Labor Markets,Labor Markets,Educational Sciences,Tertiary Education,Employment and Unemployment
teratopic: Rural Development,Education,Social Protections and Labor
abstracts: cdata!: Short-cycle higher education programs
            (SCPs) can play a central role in skill development and
            higher education expansion, yet their quality varies greatly
            within and among countries. This paper explores the
            relationship between programs&#x2019; practices and
            inputs (quality determinants) and student academic and labor
            market outcomes. It designs and conduct a novel survey to
            collect program-level information on quality determinants
            and average outcomes for Brazil, Colombia, Dominican
            Republic, Ecuador, and Peru. Categories of quality
            determinants include training and curriculum,
            infrastructure, faculty, link with productive sector, costs
            and funding, and practices on student admission and
            institutional governance. The paper also collects
            administrative, student-level data on higher education and
            formal employment for SCP students in Brazil and Ecuador and
            match it to survey data. Machine learning methods are used
            to select the quality determinants that predict outcomes at
            the program and student levels. Estimates indicate that some
            quality determinants may favor academic and labor market
            outcomes while others may hinder them. Two practices predict
            improvements in all labor market outcomes in Brazil and
            Ecuador&#x2014;teaching numerical competencies and
            providing job market information&#x2014;and one
            practice&#x2014;teaching numerical
            competencies&#x2014;additionally predicts improvements
            in labor market outcomes for all survey countries. Since
            quality determinants account for 20-40 percent of the
            explained variation in student-level outcomes, quality
            determinants might have a role shrinking program quality
            gaps. Findings have implications for the design and
            replication of high-quality SCPs, their regulation, and the
            development of information systems.
repnme: repnme: What Makes a Program Good ? Evidence from
            Short-Cycle Higher Education Programs in Latin America and
            the Caribbean
display_title: What Makes a Program Good ? Evidence
            from Short-Cycle Higher Education Programs in Latin America
            and the Caribbean
listing_relative_url: /research/2021/06/33231528/makes-program-good-evidence-short-cycle-higher-education-programs-latin-america-caribbean
topicv3: Education, Skills Development and Labor
            Market m1326135 1484,Tertiary Education m1326349 482,Labor Markets m1326488 855,Education m1327048 470,Access to Formal Contracts m1504058 2754
publishtoextweb_dt: 2021-06-29T00:00:00Z
dois: 10.1596/1813-9450-9722
historic_topic: Rural Development,Education,Social Protections and Labor
origu: Chief Economist (LCRCE)
no_of_pages: 96
author_keys: 0: author_key: 1668999
1: author_key: 1339928
2: author_key: NA
3: author_key: 714949
model_extractions: Policy Research Working Paper 9722

# What Makes a Program Good?

Evidence from Short-Cycle Higher Education Programs in Five Developing Countries _Lelys Dinarte Diaz Maria Marta Ferreyra Sergio Urzua Marina Bassi_ Latin America and the Caribbean Region Office of the Chief Economist June 2021
Policy Research Working Paper 9722

# **Abstract**

Short-cycle higher education programs (SCPs) can play a central role in skill development and higher education expansion, yet their quality varies greatly within and among countries. This paper explores the relationship between programs’ practices and inputs (quality determinants) and student academic and labor market outcomes. It designs and conduct a novel survey to collect program-level information on quality determinants and average outcomes for Brazil, Colombia, Dominican Republic, Ecuador, and Peru. Categories of quality determinants include training and curriculum, infrastructure, faculty, link with productive sector, costs and funding, and practices on student admission and institutional governance. The paper also collects administrative, student-level data on higher education and formal employment for SCP students in Brazil and Ecuador and match it to survey data. Machine learning methods are used to select the quality determinants that predict outcomes at the program and student levels. Estimates indicate that some quality determinants may favor academic and labor market outcomes while others may hinder them. Two practices predict improvements in all labor market outcomes in Brazil and Ecuador—teaching numerical competencies and providing job market information—and one practice— teaching numerical competencies—additionally predicts improvements in labor market outcomes for all survey countries. Since quality determinants account for 20-40 percent of the explained variation in student-level outcomes, quality determinants might have a role shrinking program quality gaps. Findings have implications for the design and replication of high-quality SCPs, their regulation, and the development of information systems.

This paper is a product of the Office of the Chief Economist, Latin America and the Caribbean Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at mferreyra@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **What Makes a Program Good? Evidence from Short-Cycle Higher Education Programs in Five Developing Countries**

Originally published in the <u>Policy Research Working Paper Series on</u> _June 2021._ This version is updated on _February 2023._ To obtain the originally published version, please email prwp@worldbank.org.

**Keywords:** Higher education, Short-cycle degrees, quality. **JEL codes:** I22, I23, I26, J24.
# **1 Introduction**

Higher education can be a powerful source of social mobility and economic growth. Although bachelor’s programs attract about three quarters of higher education enrollment worldwide, their returns vary widely and are not necessarily the best option for all students.<sup>1</sup> Further, the recent COVID-19 pandemic has shown the urgent need for workforce upskilling and reskilling. Shorter and more practical than bachelor’s programs, short-cycle higher education programs (SCPs) are uniquely suited to these roles. SCPs are two- or three-years long and correspond to associate degrees in the US, where they are typically taught at community colleges.<sup>2</sup> They prepare students for the labor market, often with a strong occupational content. They can play a central role addressing to rapidly changing market needs, expanding higher education to nontraditional students, and providing lifelong learning opportunities.

Despite their promise, SCPs have some shortcomings. Crucially, student academic and labor market outcomes vary greatly among SCPs. This holds for multiple countries and after controlling for student and peer characteristics.<sup>3</sup> Taking program costs into account, net economic returns vary greatly as well and are negative for many SCPs. Whether program quality is measured by outcomes or returns (raw or adjusted by student characteristics), high- and low-quality programs clearly

> 1 For bachelor’s enrollment rates worldwide, see UNESCO (http://data.uis.unesco.org/). On the variation of returns to bachelor’s programs, see, for instance, Hoxby and Stange (2019) and Lovenheim and Smith (2022) for the US, and Ferreyra et al. (2017) for the developing world.

> 2 These programs have different names depending on the country. In this paper, we follow UNESCO’s nomenclature of short-cycle programs, corresponding to level-5 programs in the International Standard Classification of Education (ISCED).

> 3 See, for instance, Jepsen et al., 2014; Dadgar and Trimble, 2015; Stevens et al., 2019; Liu et al., 2015; Grosz, 2020, and Mountjoy, 2021 for the US; Aucejo et al. (2020) for the UK; and Ferreyra et al., 2021; Ferreyra et al, 2020 for Latin America and the Caribbean.

2
coexist in the market. Nonetheless, little is known about what makes a program “good”—what is distinctive of high-quality programs—and, as result, SCP quality remains a “black box”.

This knowledge gap is largely due to a data gap, as researchers often lack information on student outcomes and, more importantly, on program practices (e.g., how programs relate to local employers) and inputs (e.g., facilities for practical training)—henceforth, quality determinants. Thus, in this paper we open the SCP quality “black box” to identify the quality determinants associated with good academic and labor market outcomes for students.<sup>4</sup> We adopt two complementary approaches. The first one is a regional (aggregated) analysis of program-level average academic and labor market outcomes for five countries in Latin America and the Caribbean (LAC). The second is a country-level analysis for two of those countries using individual-level data on graduation and labor market outcomes.<sup>5</sup> Underpinning both approaches is a new and unique survey, the World Bank Short-Cycle Program Survey (WBSCPS), which we designed and implemented in five countries in LAC, namely Brazil (in the states of São Paulo and Ceará), Colombia, the Dominican Republic, Ecuador, and Peru (for licensed programs). Together, these countries account for more than half of the SCP enrollment in LAC. The survey contains data from nationally representative samples of SCP providers for a total of 2,103 programs. It collects rich program-level information about quality determinants that are not reported in administrative datasets but might be associated with student outcomes (Bailey et al, 2015). We group them in six categories: training and curriculum (T&C),

> 4 Throughout this paper, “determinants” refers to practices and inputs that programs can choose. Other program or institutions characteristics (for example, institution’s age or public/private status) are not quality determinants because they are not equally flexible. 5 We initially set out to collect individual-level data for each of these five countries. However, most countries in the region either do not collect this type of data, merge datasets as needed (for example, those from higher education and social security), or make datasets available to researchers. Building good information systems and facilitating data access remains a key task in developed and developing countries alike.

3
infrastructure, faculty, link with productive sector (LPS), costs and funding, and other practices related to student admission and institutional governance. By complementing WBSCPS data with administrative program-level information from official sources and program websites, we obtain a novel, multi-country dataset with program-level information on student academic and labor market outcomes as well as quality determinants and characteristics of the program, institution, and student body.

In the regional approach, we exploit this dataset to find the association between program-level outcomes (dropout rate, time to degree, formal employment, and wages) and quality determinants for the five survey countries. For the country-level approach, we complement program-level information with individual-level administrative datasets from Brazil and Ecuador, which combine information from higher education censuses, national high school learning assessments, and social security records for each country. We exploit these sources to estimate the contribution of program quality determinants to student-level outcomes (formal employment and wages for both countries, plus graduation rate for Brazil) while accounting for student and peer background characteristics and former labor market experience as well as program and higher education institution (HEI) characteristics. To our knowledge, this is the first paper to measure the associations between a large set of quality determinants and academic and labor market outcomes by exploiting programand individual-level data for multiple countries.

This wealth of information allows us to characterize the SCP sector well beyond what was previously possible and to examine the association between student outcomes and program quality determinants. The sheer amount of information, however, poses the challenges of selecting the right set of explanatory variables (to avoid both omitted variable bias and model overfit) and overcoming the confirmation bias from selecting only the variables that confirm researchers’ 4
priors. To address these challenges, we use a machine learning method implemented in a two-stage estimation approach. First, we select the set of explanatory variables for student outcomes by using the adaptive Least Absolute Shrinkage and Selection Operator (LASSO) technique.<sup>6</sup> Second, we estimate OLS regressions of academic and labor market outcomes on the set of explanatory variables selected by LASSO to estimate the associations between outcomes and quality determinants while accounting for student, program, and HEI characteristics.

Our findings highlight four main conclusions. First, our program-level survey data shows great variation—within and across countries—in program outcomes and quality determinants. While some practices—such as providing labor market information to students or updating the curriculum to meet firms’ needs—are commonly reported, others are reported by less than a fifth of the programs (for example, requiring a second language for graduation).

Second, outcomes are generally associated with quality determinants from multiple categories. The most frequently selected determinants—and those associated with the greatest outcome improvements—correspond to the T&C, LPS, faculty, and other practices categories. The specific predictors of outcome improvements are different in the regional and country-level analyses. In the regional analysis, improving academic and labor market outcomes is associated with practices such as teaching a fixed curriculum, training students to work under pressure, teaching numerical competencies, running an employment center, and providing labor market information. In Brazil, three practices are associated with an improvement in graduation and all labor market outcomes— frequent analysis of student performance, use of tests as a graduation requirement, and students’ access to bank loans—and one practice—teaching numerical competencies—is positively

> 6 See Zou, 2006; Bühlmann and Van de Geer, 2011; Chetverikov et al., 2019. A recent application of machine learning methods (including LASSO) is Filmer et al. (2021), which seeks to identify predictors of teacher effectiveness.

5
associated with all labor market outcomes. In Ecuador, improvements in all labor market outcomes are associated with four practices: teaching numerical competencies, gauging student satisfaction frequently , providing labor market information, and training students for job interviews. These differences in selected quality determinants are not surprising given the large variation in program outcomes and quality determinants across countries (e.g., 74 percent of program directors in Brazil report that almost all their graduates work in the formal sector relative to 39 percent in Ecuador). Third, we find important commonalities despite these differences. Two practices contribute to _all_ labor market outcomes in Brazil and Ecuador—teaching numerical competencies and providing labor market information—and one practice—teaching numerical competencies—contributes to _all_ labor market outcomes in _all_ analyses, including the regional one. Besides their intrinsic importance, numerical competencies may proxy for related skills such as logical reasoning, problem solving, and critical thinking. Further, these competencies may enhance students’ human capital given their serious cognitive deficiencies at entry. Providing labor market information, in turn, is quite rare in Latin America, where HEIs typically regard job placement as the students’ sole responsibility. By providing this information, HEIs take the first step towards the job placement of their graduates and show their engagement with this process.

Lastly, we find that program quality determinants account for a substantial share of the explained variation of academic and labor market outcomes. Across countries, quality determinants account for 50-60 percent of the explained variation in dropout rates, time to degree and formal employment but only 15 percent of the explained variation in wages, which are mostly explained by country fixed effects. Within countries, quality determinants account for a substantive 19-40 percent of the explained outcome variation in Brazil and 32-38 percent in Ecuador. Although most of the variation in program outcomes remains unexplained within and 6
across countries, these findings are consistent with the notion that shrinking the gap in quality determinants might also shrink the gap in outcomes.

This paper contributes to several literatures. First, we link to work on the role of program quality determinants in higher education on graduates’ outcomes. This literature typically focuses on bachelor’s degree programs, uses data from a single country, and examines a single determinant such as availability or use of funding options;<sup>7</sup> curriculum structure, training, and academic remediation;<sup>8</sup> availability of infrastructure for practical training;<sup>9</sup> practices related to faculty assessment, training, and hiring;<sup>10</sup> practices to promote student employability and industry engagement;<sup>11</sup> and other program and institution characteristics such as governance.<sup>12</sup> Bailey et al (2015) focus on community colleges in the US and describe the practices of successful institutions and programs yet do not provide systematic, quantitative evidence. Our paper is novel in collecting evidence and analyzing multiple determinants for several countries and specifically SCPs.<sup>13</sup>

- 7 See studies on scholarships, grants, and awards (Andrews et al., 2020; Barrow et al., 2014; Castleman et al., 2016; Cohodes and Goodman, 2014; Field, 2009), loan availability and take-up (Armona et al., 2018; Wiederspan, 2016), and tuition (Denning, 2017; Dynarski and Scott-Clayton, 2013; Martorell et al., 2014; Scott-Clayton, 2011).

- 8 Examples are studies on credits for longer degrees or transfers (Andrews and Lovenheim, 2014), teaching a second language (Chin et al., 2013), teaching math or reading skills (Xu and Jaggars, 2011), and providing academic remediation (Bettinger and Long 2009).

- 9 This literature refers mostly to online training (Bettinger et al., 2017; Cellini and Grueso, 2021; Deming et al., 2016; Figlio et al., 2013; Johnson and Mejia, 2014; Jaggars and Xu, 2016). Evidence is lacking on the effects of other types of infrastructure for practical training.

- 10 Evidence on faculty characteristics mostly refers to gender (Bettinger and Long, 2005; Carrell et al., 2010; Kofoed et al., 2019; Lim and Meer, 2019; Porter and Serra, 2020). On faculty-related practices, the literature focuses on faculty assessment (Hoffmann and Oreopoulos, 2009; Kane and Staiger, 2012; Daniel et al., 2007), without robust evidence on faculty training or hiring practices.

- 11 Most of these papers analyze internship agreements between firms and HEIs (Di Meglio et al. 2022; Jaeger et al. 2020; Wesley et al. 2019; Yi 2018); few papers study industry engagement in curriculum design (Plewa et al., 2015).

- 12 See Teixeira et al. (2013), Hoxby and Bulman (2015), and Bound, Lovenheim, and Turner (2010).

- 13 Our paper is also in the spirit of Dobbie and Fryer (2013) for K-12 education. The authors collect data on charter school practices and use it to explain the variation in charter school effectiveness.

7
Second, our paper contributes to evidence on SCP returns. Most of this literature focuses either on a single US state<sup>14</sup> or a single country.<sup>15</sup> Our results extend the current literature by highlighting—within and across countries—the variance of a large set of program quality determinants, the association between these determinants and student outcomes, and the heterogeneity of these associations. We do not estimate SCP returns relative to a high school diploma (as in some of this literature) but rather compare SCPs among themselves.

Finally, this paper also relates to the literature that uses surveys to collect international data on practices to compare across countries. Some recent efforts of this kind include surveys on managerial practices in schools, firms, and health care (Bloom et al., 2015; 2016; 2020) as well as on universities’ practices on governance and performance (Aghion et al., 2010; McCormack et al., 2014). While data from administrative sources and “big data” methods have become increasingly prevalent, survey data remains an attractive option for two reasons. First, granular data on program quality determinants is usually not collected administratively either in developed or developing countries. Even countries that collect general institution or program characteristics do not inquire about issues such as links with the productive sector, faculty, and infrastructure. Second, as Bloom et al. (2016) note, some practices rarely have comparable measures across countries. To our knowledge, this is the first paper to conduct a multi-country data collection initiative on higher education quality determinants and student outcomes for cross-country comparisons.

> 14 See Bahr (2013; 2016) and Carrell and Kurlaender (2016) for California; De Vlieger, Jacob, and Stange (2017) for University of Phoenix; Kane et al. (2021) and Boatman and Long (2018) for Tennessee; Liu et al. (2015) for North Carolina; and Xu & Dadgar (2018) for Virginia.

> 15 See Cellini and Turner (2019), Stange (2012), and Scott-Clayton and Rodriguez (2015) for the United States; Melguizo and Wainer (2016) for Brazil; Dinarte et al. (2022), Shavelson et al. (2016), Melguizo et al. (2016), Saavedra (2009), Barrera-Osorio and Bayona-Rodriguez (2019) for Colombia; and Rodriguez et al. (2016) for Chile.

8
The remainder of the paper is organized as follows. Section 2 describes the SCP landscape in the region. Section 3 presents the data and descriptive statistics. Section 4 describes our empirical approach and section 5 summarizes the main results. Section 6 concludes.

# **2 SCPs in Latin America and the Caribbean, and in Survey Countries**

Higher education in LAC has recently experienced a large, rapid expansion, with gross enrollment rates rising from 23 to 52 percent between 2000 and 2017 (Ferreyra et al 2017). Quality remains a challenge, with average graduation rates of only 47 percent across countries. Program and field variety are an additional challenge, as only 9 percent of higher education students in LAC are enrolled in SCPs (relative to the world average of 24 percent), and the average share of STEM graduates is lower in the region than the world (18 vs. 25 percent, respectively). Perhaps as a result, the share of firms reporting serious difficulties in finding qualified workforce is higher in LAC (32 percent) than any other region in the world (Ferreyra et al 2017, Ferreyra et al 2021).

Across our five survey countries, the institutional landscape and popularity of SCPs shows great variation (see Table A1 and Ferreyra et al 2021).<sup>16</sup> Relative to total higher education enrollment, SCPs capture a share between 4 percent in the Dominican Republic and 32 percent in Colombia. On average, SCPs last two or three years and cover a variety of fields ranging from traditional (e.g., nursing and tourism) to innovative areas (e.g., cybersecurity and digital animation). The number of SCPs varies between 209 in the Dominican Republic and 2,388 in Brazil, and the number of HEIs ranges from 28 in the Dominican Republic to 467 in Brazil. SCP providers include universities and non-university HEIs or providers in all countries except Peru, where they only include non-university HEIs. A public national institution exists in Colombia

> 16 In what follows, “Brazil” refers exclusively to the states of Ceará and São Paulo, and “Peru” to licensed programs.

9
(SENA), and a private one in Brazil (the S-System, including SENAI) and Peru (SENATI). Due to this institutional variety and lack of coordination among providers, there is usually no clear pathway from SCPs to bachelor’s programs.

In our survey countries, SCPs are offered by public and private HEIs; the private share varies from 21 percent in Colombia to 97 percent in Peru. For-profit HEIs are permitted only in Brazil and Peru. While programs in public institutions are free or highly subsidized due to government financial support, private institutions depend almost entirely on tuition revenues. In some cases, governments provide scholarships, student loans, or guarantees for student loans; students can usually borrow from commercial banks. For the most part, however, students pay tuition out of their own pockets (Ferreyra et al., 2021). HEIs generally need a license to open a program and must undergo a periodic evaluation for license renewal.

As in the US, SCPs in LAC attract more disadvantaged and nontraditional students than bachelor’s degree programs. Nonetheless, SCP students in LAC exhibit relatively favorable academic and labor market outcomes (Ferreyra et al. 2021). The SCP average completion rate is 57 percent, 11 percentage points higher than the average completion rate for bachelor’s degrees. SCP graduates in LAC have lower unemployment rates, higher formal employment rates, and higher salaries not only than high school graduates but also than dropouts from bachelor’s degree programs, even after controlling for observable characteristics. On average, _Mincerian_ returns to SCPs relative to high school or an incomplete bachelor’s program are equal to 60 and 25 percent, respectively. In other words, SCPs seem a promising option for postsecondary training.

# **3 Data and Descriptive Statistics**

## **3.1 Program-Level Data**

10
**The World Bank Short-Cycle Programs Survey (WBSCPS).** This novel and unique survey collects rich information at the program and institution levels about practices, inputs, and other characteristics typically not reported in administrative datasets. The WBSCPS was administered to SCP directors between September 2019 and October 2020 in five LAC countries: Brazil (states of Ceará and São Paulo), Colombia, the Dominican Republic, Ecuador, and Peru (licensed programs). These countries were chosen based on their availability of SCP sampling frames (see below), the interest of their educational authorities in participating in the study, and their research and policy relevance.

For the survey, the SCP universe (or sampling frame) is the set of all programs offered in a country as reported by official sources. Since the universe size varies greatly across countries, we surveyed all programs in the universe in the Dominican Republic, Ecuador, and Peru (including, for the latter, all licensed programs as of October 2019). For Colombia and Brazil, we applied a stratified random sampling procedure to obtain representative SCP samples at the national level in Colombia and the state level in Brazil. The total sample size is 3,656 programs relative to a universe of 5,657 programs over all five countries. Table A2 shows the sampling methodology by country as well as the assumptions used for power calculations.

We developed a survey instrument to gather comparable information across countries (available at <u>instrument ). The instrument was shared and validated with local authorities, piloted</u> in subsample of programs, and adjusted based on pilots. The instrument is organized around the following topics: student demographics and readiness for the program; admission and graduation requirements; faculty characteristics, hiring, and evaluation; curriculum and practical training; infrastructure; online teaching; costs and financing; oversight and regulation; institutional governance; interaction with industry; job search assistance; competition; and academic and labor 11
market outcomes. We mapped the information collected to program quality determinants, further grouped into six categories (T&C, LPS, faculty, infrastructure, costs and funding, and other practices); student, program, and HEI characteristics; and program-level outcomes.

Interviewers were trained in data-collection protocols. To assess training effectiveness, a practical exercise was carried out as in Bloom et al. (2020), during which trainees practiced how to read and complete the questionnaire. Interviewers contacted all program directors in the sample. Directors first received an email from the survey firm about the study (accompanied by a letter from the research team) announcing a forthcoming phone call or website link to complete the survey. Interviewers called each program director eight times on average and scheduled the interviews in advance for the directors’ convenience, sometimes splitting the interview into two calls to suit the directors’ schedule. These procedures helped us to obtain a response rate of 70 percent on average (Table A3) for a total of 2,103 interviews (67 percent online and 33 percent by phone). This is a high response rate, especially because the survey is relatively long and requests some information that the directors may have had to look up, and 48 percent of surveys were conducted under stay-at-home policies due to the COVID-19 pandemic.

**Addressing threats to survey data quality.** Two sources of bias can emerge with survey data: self-selection and self-reporting. Self-selection into responding the survey might have taken place, for instance, if the directors of higher-performing programs had been more motivated to answer. In addition, since some interviews took place while the HEIs were migrating from in-person to online delivery—when staff were hardly available—only the directors of programs with larger staff might have responded. To mitigate this potential bias, we followed two strategies. First, we verified that the share of programs declining to participate in the survey was low and similar across countries. Only 14 percent of program directors declined to participate in the survey (Table A3).

12
Second, we evaluated survey representativeness. For each country, we compared programs in the survey and others in the universe based on characteristics such as HEI governance (public / private; for-profit), tuition, enrollment, and institution type (university, non-university HEI, non-HEI). Although a few differences were statistically significant, they were small in magnitude (Table A4). We recalibrated the sampling weights used in the estimations to account for them. As a result, our results can be viewed as representative for the SCP universes.

Self-reporting could have biased our survey data if, for example, program directors had expected to receive some benefit from the educational authorities based on their responses, in which case they would have had incentives to misreport information. To mitigate this problem, we followed best practices for self-reported data collection and applied them to all programs. First, respondents received a letter from the research team indicating that all responses would remain confidential and anonymous and would be reported only in an aggregate fashion—thus favoring truthfulness—as in Bloom et al. (2020). Second, the letter informed respondents that the survey was being conducted exclusively by The World Bank for research purposes and made no mention of governmental units. Third, questions were designed to avoid common biases associated with self-reporting. For example, questions referred to specific time periods (such as the previous academic year) to avoid memory biases, and, where possible, included specific response options.

To address potential biases in self-reported program outcomes, we conducted an outcome validation exercise using country-specific administrative data and household surveys. On average, we found that directors’ outcome reports did not differ substantially from outcomes obtained from these other sources. This result was more salient when we restricted the comparison of average labor market outcomes to the age group that was more likely to be enrolled in SCPs (Table A5).

13
We supplement the survey data with administrative information on program tuition and highquality accreditation (above and beyond a regular license), and institution governance and type. Some of these variables are available in administrative sources such as higher education censuses, whereas we collected others from HEI and program websites.

# **3.2 Descriptive statistics of program-level data**

Tables 1 and A6 show descriptive statistics of quality determinants (panel A) and student, program, and HEI characteristics (panel B) using program-level data from the WBSCPS and administrative sources. We begin with the description of quality determinants. On average, the programs have desirable traits but also substantial variation. In terms of _infrastructure_ , most programs (72 percent) have enough equipment for practical training given enrollment. Online teaching was rare prior to the COVID-19 pandemic, suggesting that the adjustment to provide it must have been challenging. As for _T&C_ , most programs (70 percent) teach a fixed curriculum with structured pathways (last updated, on average, about three years ago), with substantial emphasis on practical training. They tend to teach both cognitive (e.g., numerical) and socioemotional competencies (e.g., working under pressure), and about half of them provide remedial education, before and/or during the program. Most programs (86 percent) analyze student performance more than once a year to address problems. About 60 percent of the programs require an internship outside the institution, and less than half of the programs have special graduation requirements such as specific exams, theses, or second language tests. Regarding _costs and funding_ , annual tuition is $2,244 on average; it varies between zero and $25,515 (note that all monetary variables are expressed in 2019 PPP dollars except where noted). Less than a third of the programs receive outside funding, and about 40 percent of programs report having some students who take on bank or government loans.

14
In terms of LPS, most programs (82 percent) have somebody in charge of industry relations. About half of the programs (52 percent) communicate with local firms to gauge their needs or collect data on graduates’ employment or employers’ satisfaction with the graduates. Less common (35-40 percent) are agreements with industry to hire program graduates or train faculty. Most programs support students’ job search; for instance, they train students for job interviews (69 percent), run an employment center (60 percent), and, most often, provide job market information (81 percent). As for _faculty_ , the average program has 20 instructors. On average, these are wellqualified (82 and 49 percent have bachelor’s and graduate degrees, respectively) and experienced (56 percent have at least five years of industry experience) although less than half currently work in industry. They are mostly male and teach part-time. In most programs (85 percent), faculty are evaluated more than once a year, and about half of the programs provided professional training to most faculty the previous year. When hiring faculty, most programs (88 percent) seek practical experience. In terms of _other practice_ s, two-thirds of programs frequently review and update their administrative records, and 50-70 percent rely on admission requirements such as exams, interviews, or test scores in mandatory national exams. In the average program, faculty members comprise the largest share of the HEI’s governing board (40 percent of board members).

As panel B shows, in the average program most students are part-time, male, and younger than 25 years old. Most incoming students lack basic skills; the main deficiency is in numerical skills (reported by 82 percent of programs), followed by reading and writing. The average program has a theoretical duration of 5.2 semesters, is 11.5 years old, and enrolled 222 students the year prior to the survey. HEIs in our sample are mostly private (70 percent), are not universities, and lack high-quality accreditation. The average institution is 38 years old and offers 22 programs.

15
**Program-level outcomes.** We measure these based on the aggregate information on academic and labor market outcomes reported by program directors. Academic outcomes include dropout rate and time-to-degree. Dropout rate is the percentage of students who dropped out among those who were supposed to graduate the previous academic year. Our time-to-degree measure is the extra time to graduate (ETG), equal to the average percentage of additional time that students take to graduate relative to the program’s theoretical duration, which is reported by the program director. Labor market outcomes include formal employment and wages. Formal employment is a binary variable that equals one when the director reports that almost all the program’s graduates from the previous year are currently employed or self-employed in the formal sector, and zero otherwise. Wages correspond to the average annual salary earned by last year’s graduates, whether they work in the formal or informal sector. Appendix I provides further detail on outcomes.

On average, dropout rate is 14 percent. Average dropout rates are similar in Brazil, Colombia, and the Dominican Republic and slightly lower in Ecuador and Peru (Figure 1 panel A). Average ETG is 19 percent, ranging from an average of 31 percent in the Dominican Republic to 9 percent in Peru (Figure 1 panel A). In terms of formal employment, 59 percent of directors reported that almost all their graduates were formally employed or self-employed. Formal employment is highest in Brazil and lowest in Ecuador (averages are 74 and 39 percent, respectively; see Figure 2 panel A). Wages vary greatly within and across countries (panel B Figure 2). The average ranges from $7,481 in Peru to $11,910 in Ecuador and is only 30- 40 percent above the minimum wages in all countries except Brazil, where it is more than twice as high (Table A7).

# **3.3 Individual-Level Data for Ecuador and Brazil**

Despite its novelty and richness, our program-level data gives us limited ability to estimate the value-added contribution of quality determinants to student outcomes. To control for student and 16
_— —_ peer characteristics including previous labor market outcomes we merged in individual-level data from multiple administrative sources for Brazil and Ecuador.

# **Brazil**

**_Higher Education Census (HEC)._** This is the universe of higher education students, programs, and HEIs in Brazil and comes from the Ministry of Education. For a given academic year, and for every student enrolled in higher education, the HEC provides demographics (age, gender, and race), initial enrollment date, and dropout or graduation status by year’s end. Focusing on the programs in our sample of effective surveys (henceforth, surveyed programs), we selected the students who entered them in 2014, and used the 2015 and 2016 HECs to establish whether they had graduated. We define a student’s peers as those who entered her same program in 2014.

**_National Educational Entrance Examination._** The _Exame Nacional de Ensino Médio_ ,

(ENEM) dataset comes from the Brazilian Ministry of Education and includes student demographic and family background variables. Although it includes individual ENEM test scores, we do not use these because they are missing for 60 percent of the sample.

**_Labor Market Outcomes._** Their source is the Annual Reports of the Social Administration ( _Relaҁão Anual de Informaҁões Sociais,_ RAIS), a matched employer-employee dataset of all workers and firms in the Brazilian formal sector. It is constructed by the Brazilian Ministry of Labor based on a mandatory annual survey filled by all firms in the formal sector. RAIS contains information on earnings (gross monthly wages), employment, occupation, and demographics for all individuals who are employed by a formal firm in a particular year. For every individual selected from the 2014 HEC, we use RAIS to measure employment status and earnings in the 12month period before she starts the program (namely, in 2013) and the 12-month period following her graduation (namely, in 2016 or 2017) provided she graduates within three years.

17
By merging these sources, we create an individual-level dataset for Brazil for students who entered our surveyed programs in 2014. It includes student gender, age, and mother’s education level; graduation status as of 2016; and pre- and post-program labor market outcomes for graduates who were formally employed after graduation. We merge the individual-level dataset with information at the program- and HEI- levels from the WBSCPS and other administrative data. The resulting dataset includes 29,453 students and 401 programs (relative to 601 surveyed programs).<sup>17</sup>

# **Ecuador**

**_2019 Higher Education Census (HEC)._** This is the universe of higher education graduates who obtained their degrees between January and December of 2019. It comes from the Science, Technology, and Innovation Secretariat ( _Secretaria Nacional de Ciencia, Tecnologia e Innovacion_ , SENESCYT), and contains information on approximately 29,000 SCP graduates, including field of study, institution, program name, and graduation date. Based on these data, we define a student’s peers as those who also graduated from her program in 2019.

**_National Educational Entrance Examination._** This dataset comes from the National Institute

for Educational Assessment ( _Instituto Nacional de Evaluacion,_ INEVAL) at the Ministry of Education. It records test scores on the mandatory high school exit exam ( _Ser Bachiller_ ) and selfreported student socioeconomic background at the time of the exam. We obtained access to a subset of this dataset through the Higher Education Access Unit ( _Subsecretaria de Acceso a la Educacion Superior_ ) at SENESCYT, with information on students who took the test in 2017 and

> 17 Some programs do not match because they did not yet exist in 2014, which is our cohort’s entry year. Others did exist in 2014 but did not have graduates who were formally employed during our sample period. Table A8 presents descriptive statistics for the subsample of 401 programs matched to individual-level data. It also shows t-tests for mean differences between those programs and the remaining 200. A few means are statistically different. For example, matched programs are more likely to have sufficient infrastructure for practical training or coordinate job interviews with firms than unmatched programs. They also tend to be older, are taught by smaller institutions, and have higher shares of part-time students.

18
2018 (age, gender, whether the student has children of her own, family of origin’s socioeconomic score, and mother’s education level.) Note that the subset of students to which we were given access may not be a representative sample of all students who took the test. Further, the scores of the _Ser Bachiller_ test were not shared with us.

**_Labor Market Outcomes._** Their source are individual-level records from the Ecuadorian Social Security Institute ( _Instituto Ecuatoriano de Seguridad Social,_ IESS) and the Ecuadorian Internal Revenue Service Unit ( _Servicio de Rentas Internas,_ SRI), which together yield the universe of individuals formally employed or self-employed, with monthly records of individual employment status and earnings between January 2018 and December 2020. We use these to construct labor market outcomes for the 12-month periods before and after graduation.

The final dataset for Ecuador is a sample of 2019 SCP graduates. It includes individual characteristics (gender, age, whether the student has children), socioeconomic background (mother’s education level and socioeconomic index), pre-graduation labor market outcomes, and post-graduation formal employment status and wages. We merge the individual-level dataset with information at the program and HEI levels from the WBSCPS and administrative sources. The resulting dataset includes 1,239 individuals and 92 programs (relative to the 245 programs with effective surveys in our sample).<sup>18</sup> It is worth emphasizing that our individual-level datasets for Brazil and Ecuador are different in that, for Brazil, we have information on all students who entered our survey programs in 2014

> 18 Some programs do not match to individual-level data for reasons similar to those in Brazil. In addition, some programs match but only have one student. Table A9 presents descriptive statistics for the subsample of the 92 surveyed programs that match to individual-level data and shows t-tests for mean differences between these programs and the 153 unmatched ones with 2+ students. Some differences are statistically significant. Matched programs are more likely to update the curriculum based on government standards and HEI labor market perceptions, require a professional test for graduation, or have agreements with firms to hire their graduates. They have more workshops for practice, charge a higher tuition, and have more faculty members. They are more likely to be taught by private, large HEIs, and have high-quality accreditation.

19
(whether they went on to graduate or not), whereas we only have information on graduates for Ecuador. Therefore, we will examine graduation outcomes only for Brazil, but will analyze the following labor market outcomes for graduates from both countries over the 12-month period following graduation: (i) whether the student was formally employed at least one month; (ii) what percent of those months she was formally employed; and average monthly wages (calculated as the average over her months of formal work, and equal to zero if she did not work formally at all). Further, we use individual-level data to construct average characteristics and previous labor market experience for peers.<sup>19</sup> Appendix I provides additional information on all variables.

# **3.4 Descriptive statistics of individual-level data**

Table 2 presents descriptive statistics of program graduates in Ecuador and Brazil, for whom we study labor market outcomes. In both countries, the average student is about 25 years old and about 60 percent of students are female (panel A). Students in Ecuador have more educated mothers than in Brazil and 27 percent of them have children of their own. The average student in Brazil has accumulated more formal labor market experience before entering the program than the average graduate in Ecuador (panel C): 64 percent of students in Brazil worked at least one month before entering the program and worked for 46 percent of the time on average, whereas only 26 percent of graduates in Ecuador worked at least one month before graduation and worked for 19 percent of the time on average. Students in Brazil, therefore, have peers with more previous labor market experience than in Ecuador. After graduating (panel D), the average graduate in Brazil is more likely than that in Ecuador to be employed formally for at least one month (70 vs. 35 percent),

> 19 For student _i_ , peer characteristics are the average characteristics of the _other_ students in her cohort (Sacerdote, 2011). Recall that a program’s cohort is set of all 2019 program graduates in Ecuador (as information on entry cohort is not available) and all 2014 program entering students in Brazil. Interviews with local experts from Ecuador confirmed that graduation and entry cohorts usually have similar characteristics.

20
spends a greater fraction of the time in formal employment (53 vs. 24 percent), and has a higher monthly wage (US$1,031 vs. US$876) conditional on being formally employed. Finally, Table A10 describes the full set of students who started an SCP in 2014 in Brazil, of whom only 30 percent graduated within three years.

# **4 Empirical Strategy**

## **4.1 Cross-Country Estimations Using Program-Level Data**

### **Data-driven selection of quality determinants**

The WBSCPS has the fascinating advantage of providing a large set of variables that proxy for program quality determinants. However, this large number of variables poses some challenges. The first is to select the “right” set of explanatory variables. Using too few controls—or the “wrong” ones—may create omitted variable biases while using too many may overfit the model. A second empirical challenge is the potential for researcher’s confirmation bias—selecting the variables in a way that confirms her hypotheses. We address these issues with our two-stage estimation approach. First, we use a data-driven method to select the parsimonious set of quality determinants that best fit the outcome data. Second, we estimate the association between the selected program quality determinants and the outcomes of interest.

To select quality determinants in the first stage, we use a supervised machine learning approach: the Least Absolute Shrinkage and Selection Operator (LASSO) technique. LASSO chooses a parsimonious set of controls that provide the best possible fit of the data and discards those that contribute little to the fit. For a given outcome in program _j_ and country _c_ , we estimate the following linear specification using LASSO: 21
where yjc represents one of the four program-level outcomes reported by program directors (dropout rate, ETG, formal employment, and wages). The vector Qdjc includes all the survey variables for category _d_ of quality determinants (recall the six categories: infrastructure, costs, T&C, LPS, faculty, and other practices; see Table 1 panel A.). We use Qjc and α1 to refer to the full set (over all categories) of quality determinants and their coefficients, respectively.

A potential concern from regressing yjc on Qjc alone is omitted variable bias, as other program or institution characteristics might determine yjc and be correlated with the determinants. Therefore, we control for observable characteristics at the program or institution level, Cjc (see Table 1 panel B). To avoid model oversaturation due to these additional variables, we use a data reduction strategy—principal components analysis (PCA)—and build indexes for student body, program, and HEI characteristics (Appendix I). We also include country and field fixed effects in all specifications (φc and φf, respectively) to account for systematic differences in program-level outcomes across countries and fields. Moreover, we add the statistical noise controls for survey measurement error (Table 1 panel C), Njc, including number of attempts to complete the survey, whether the interview was conducted during lockdown policies, and interview mode (phone or online). Finally, εjc is the error term.

We use the adaptive LASSO (Belloni et al., 2014 and 2015), which selects the tuning parameters (weights) used by LASSO to discard or keep determinants in order to minimize the out-of-sample mean squared error (MSE) of the predictions.<sup>20</sup> To ensure that we control for Cjc and Njc, LASSO includes them in every model it estimates, holding them “fixed” while it finds

> 20 Adaptive LASSO is more conservative than the cross-validation (CV) method, which tends to include extra covariates whose coefficients are zero. See Zou (2006), Bühlmann and Van de Geer (2011), and Chetverikov, Liao, and Chernozhukov (2019).

22
the best combination of quality determinants in Qjc. In other words, LASSO tries combinations of quality determinants (which are therefore “floating” variables) conditional on the “fixed” variables. The selected subset of determinants for category _d_ , Q∗djc , is an input for the second estimation stage.

## **Associations between quality determinants and outcomes**

LASSO’s ability to work as a covariate-selection method makes it a nonstandard estimator and prevents the estimation of standard errors. Therefore, we implement a second stage that predicts a given outcome as a function of its selected determinants through the following OLS regression: The estimated parameters of interest are in the βd1 vectors, reflecting the association between the selected quality determinants and the outcome. Standard errors are clustered at the HEI level.

# **4.2 Country-Specific Estimations Using Individual-Level Data**

Program-level data does not allow us to satisfactorily address student self-selection into programs. To address this limitation, we use the individual-level administrative data for Brazil and Ecuador and follow Dinarte et al. (2022), Melguizo et al. (2016), and Smith and Stange (2016) to estimate the following model for a given outcome and country (Brazil or Ecuador): where yij is an outcome for student _i_ in program _j;_ Ri is an index of individual characteristics (e.g., gender, age, socioeconomic status, parental education) and previous labor market experience; and Zij is an index of peer characteristics and previous labor market experience. These indexes were created (via PCA) to reduce the dimensionality of the corresponding variables (see Appendix I).

23
For Brazil estimations we also include state fixed effects (São Paulo and Ceará) and graduation year (2015 and 2016) fixed effects. Standard errors are clustered at the program level.

We estimate this equation following the same two-stage approach described above. Throughout, all regressors except for those in the Q<sup>d</sup> vectors are held fixed **.** Estimates of the second-stage coefficients on the LASSO-selected quality determinants measure these determinants’ contributions to student outcomes, net of the contributions from the student, her peers, or other program and institution characteristics.

## **4.3 Variance decompositions**

We conduct a Shapley–Owen R-squared decomposition of the second-stage regressions (Shapley 1953; Owen 1977; Huettner and Sunder 2012). For the program-level regressions, we quantify the fraction of explained variation attributable to the following sets of variables: quality determinants (overall and per category); student, program, and HEI characteristics (as captured by the corresponding PCA scores), field fixed effects, and country fixed effects. For the individual-level regressions, we additionally quantify the fraction of explained variation attributable to student and peer administrative variables (as captured by the PCA scores) as well as, for Brazil, state and graduation-year fixed effects.

# **5 Estimation results**

## **5.1 SCP Quality Determinants**

### **Using program-level data**

### **_Academic Outcomes_**

24
**Dropout rates.** Several determinants are associated with reductions in dropout rates of about 1.52 pp, relative to a sample average of 14% (Figure 3 panel a; Table A11 column 1).<sup>21</sup> The largest reductions are associated with one T&C determinant—having a fixed curriculum—and two LPS determinants—providing job market information and obtaining equipment from industry for student training. The fixed curriculum finding is consistent with the US community college evidence in Bailey et al (2015) that structured pathways promote graduation. Industry connections and job search assistance, in turn, might show students their labor market prospects and motivate them to graduate. Some determinants are associated with higher dropout rates (e.g., receiving outside funding, which might come with conditions that limit the program’s margin of action).

**Estimated Time to Degree (ETG).** The determinants associated with the greatest ETG reductions (3 to 7 pp, relative to an average ETG of 18.6%) come from multiple categories (Figure 3 panel b; Table A11 column 2). Unsurprisingly, programs that teach students to work under hardship or pressure have lower ETGs, presumably by teaching them to persevere and work efficiently, as do programs that evaluate faculty more than once per year (promoting frequent teaching adjustments), engage with industry for student evaluation or curriculum design (perhaps yielding a more efficient and engaging curriculum), or have higher tuition (as students would seek to graduate fast to avoid paying it). Programs with a higher ETG (by about 3 to 5 pp) are those whose curriculum updates rely heavily on the HEI’s perception of the labor market (which might make the program unnecessarily long or involved) or that require a thesis or research project for graduation. Anecdotal evidence indicates that such projects become a hindrance to students who do not have

> 21 Figures 3-6 presents the estimated association between student outcomes and quality determinants, focusing on significant coefficients (at the 1, 5 or 10 percent level). For binary determinants (e.g., whether the program provides job market information), the figure presents the coefficient estimate. For non-binary determinants (e.g., percent of female faculty), it presents the coefficient estimate multiplied by the determinant’s standard deviation. The full regressions of the corresponding outcome on the variables selected by LASSO (including those whose coefficients are not statistically significant) are in tables A11-A14.

25
the preparation or support necessary to complete the projects on time. Some determinants may improve one outcome but hurt others. For example, programs with a higher proportion of faculty working in industry have lower ETG but higher dropout rates. Such instructors may design engaging curricula (lowering ETG) but deviate students away from the program and into industry (raising dropout rates).

# **_Labor Market Outcomes_**

**Formal employment.** Teaching numerical competencies is associated with a 15-pp increase in formal employment, relative to an average formal employment of 59 percent (Figure 4 panel a; Table A12 column 1). While intrinsically valuable, these competencies may also capture related skills such as logical reasoning and critical thinking. Further, programs that provide remediation during the program also have higher formal employment—by providing a context for the remediation, they may be more effective at raising employability-related skills. Some LPS practices are associated with higher (4-9 pp) formal employment, such as running an employment center (9 pp), assigning staff to collect graduates’ employment data, and collecting data frequently on graduates’ employment or employers’ satisfaction with graduates. Further, graduates from programs with a higher proportion of experienced faculty or that have enough equipment for practice also have higher formal employment. On the other hand, formal employment is lower in programs with a higher proportion of young faculty, who may have little experience working in industry or teaching these programs.

**Wages.** Higher wages (by 3-7 percent) accrue to graduates from programs with specific T&C practices such as teaching numerical competencies, providing remediation during the program _,_ granting credits for longer degrees, and relying heavily on the HEI’s perception of the labor market for curriculum updates (Figure 4 panel b; Table A12 column 2). Further, graduates from programs

26
with a higher faculty-student ratio<sup>22</sup> and a higher share of faculty with bachelor’s degrees also have higher wages. Unsurprisingly, graduates from programs that use knowledge tests as admission requirements (and are therefore more selective) have higher wages. Wages are lower in HEIs with a higher representation of students in the governing body since student demands may not be driven by concerns over future earnings.

# **_Program-level analysis—taking stock_**

The program-level analysis shows that, across countries, T&C, LPS, and faculty determinants are associated to academic and labor market outcomes. Cost and infrastructure determinants, in contrast, are associated only to academic or labor market outcomes, respectively. The main determinants, based on the size of their association with academic and labor market outcomes, are those from T&C; they include teaching a fixed curriculum, training students to work under pressure, teaching numerical competencies, and providing credits for longer degrees.

In a theme of our findings, a few determinants favor some outcomes but hinder others. For example, relying on the HEI’s perception of the labor market for curriculum updates favors wages but hinders ETG. Although our estimates do not imply causality, these results may indicate the need to weigh these trade-offs when deploying the corresponding practices.

# **Using individual-level data for Brazil**

By using individual-level for Brazil and Ecuador, we gain variation in the outcomes and augment sample sizes relative to our program-level analysis. This gives the adaptive LASSO the ability to select a higher number of determinants in the first stage, and our second stage gains statistical power to find significant coefficients. Further, controlling for student and peer characteristics helps

> 22 Since we control for program enrollment, the coefficient on number of faculty can be interpreted in terms of facultystudent ratio.

27
us address student self-selection into the program and provides an approximation to the valueadded contributions from program determinants to student outcomes.

**Academic outcomes (graduation).** Given our short graduation window (three years), the graduation outcome is practically a measure of on-time graduation and is therefore related to the two academic outcomes measured in the survey. Consistent with our program-level findings, graduation in Brazil is sensitive to determinants from all categories (Figure 5 panel a; Table A13 column 1). The largest associations (6-9 pp, relative to a sample average graduation of 30.3 percent) correspond to two T&C practices, namely frequent analysis of student performance (to solve academic problems in real time) and the use of tests as graduation requirements (which may operate as a commitment device or may be a professional requirement akin to US bar exams). Graduation is also higher (by about 5 pp) in programs that offer at least one online class (providing students with greater coursework flexibility) or maintain their main labs periodically (ensuring that the equipment can be used when needed). External financing through bank loans is associated with higher graduation rates, perhaps by allowing students to focus on their studies rather than having to work. Programs with industry agreements to train faculty have higher graduation rates, likely by helping faculty stay current and connected with industry. HEIs with greater student representation in the governing board also have higher graduation rates, as students may negotiate either for laxer graduation requirements or for teaching practices that promote persistence. A few determinants are negatively associated with graduation rate, such as providing professional training for all or almost all of the faculty (consistent with program-level findings on ETG and suggestive that the professional training may detract from time with the students), having internships agreements with industry (as students may take longer to finish or be poached by

28
industry before they graduate), and updating the curriculum based on regulatory norms or student feedback, which may not reflect labor market needs.

**Labor market outcomes.** Recall that we focus on graduates’ labor market outcomes during the twelve months following graduation, including two formal employment measures—whether the student is ever employed, and percent of the time employed (sample means are 70 and 53 percent, respectively)—and average monthly wage (sample mean is $716 including zeros).

Several determinants have a positive and relatively large association to these three outcomes (Figure 5 panels b-d; Table A13 columns 2-4). From T&C, programs that analyze student performance frequently and those that teach numerical competencies have better outcomes for the two employment measures (by 4-6 pp) and higher wages (by about 30 percent). Programs that require a graduation test also have better labor market outcomes. From LPS, providing job market information is associated with better employment outcomes (by 4-6 pp) and wages (by about 40 percent), likely because it facilitates and kickstarts students’ job search. The positive association of labor market outcomes with practices related to faculty (frequent faculty evaluation) and administration (frequent review of administrative data) shows the usefulness of real-time reviews. Admission interviews have a positive association to labor market outcomes, likely by ensuring a student’s good fit to the program. Interestingly, the use of bank loans also has a positive association with labor market outcomes, perhaps by allowing students to choose high-return programs, and online provision helps students graduate and work a greater fraction of the time, perhaps by allowing them to work during the program and therefore improve labor market prospects. On the other hand, two practices are negatively associated to all labor market outcomes, namely training students for job interviews, and collecting data on employment or employers’ satisfaction more 29
than once a year. While seemingly good, these practices may detract from other productive uses of time or be interpreted by potential employers as an attempt to remediate other weaknesses.

All in all, three practices are associated with an improvement both in academic and labor market outcomes in Brazil: frequent analysis of student performance, use of tests as a graduation requirement, and students’ access to bank loans. These practices may allow students to choose high-return programs, may help the program identify academic problems in real time, and may constitute a commitment device for students to graduate. Programs with online provision of classes also perform better in several outcomes, likely because they are more flexible. In addition, numerical competencies are associated with better labor market outcomes.

# **Using individual-level data for Ecuador**

Since we do not observe academic outcomes for Ecuador, we discuss the same labor market outcomes as for Brazil during the twelve months following graduation: whether the student is ever employed formally and percent of the time employed formally (sample means are 35 and 24 percent, respectively), and average monthly wage (sample mean is $300 including zeros).

Several determinants from the T&C and LPS categories have a positive and large association to the three outcomes (Figure 6 panels a-c; Table A14 columns 1-3). From T&C, teaching numerical competencies is associated with a 20-pp improvement in the employment measures and a doubling of wages. Programs that collect student satisfaction information frequently also have better labor market outcomes, perhaps because they improve job placement based on the feedback. Updating the curriculum based on government standards also delivers better outcomes, perhaps because firms prefer to hire graduates from compliant programs. From LPS, programs that provide labor market information have better employment outcomes (as in Brazil), as do programs that train students for job interviews (unlike Brazil). Job interview training is associated with a 20-pp 30
improvement in the employment measures and more than a threefold-increase in wages. In Ecuador, students might need this training more than in Brazil because they have less previous labor market experience (section 3.4). At the same time, a couple of faculty determinants are negatively associated to both labor market outcomes. The main one is hiring faculty based on research skills, which are not necessarily desirable for SCP teaching.

Given Ecuador’s low formal employment, we further examine which variables contribute exclusively to formal employment (but not wages). These include having sufficient infrastructure for practical training (consistent with program-level findings), borrowing equipment from industry for practical training, and having agreements with industry to hire graduates. In contrast with Brazil, programs that require a graduation test have worse labor market outcomes.

**_Taking stock._** Overall, the set of quality determinants associated with program outcomes is different in Brazil and Ecuador. This is not surprising given the large variation in program characteristics, practices, and outcomes across countries. Further, some practices (such as requiring a graduation test or training students for job interviews) have opposite-sign associations with labor market outcomes in Brazil and Ecuador. This might indicate that how these practices are implemented (e.g., how much they take away from other activities) and in which context (e.g., how much students need them) likely affects their usefulness.

# **What makes a program good?**

We return to our original question of what makes a program good. Based on our estimates, the answer depends on the type of data used and the country of interest. This is not surprising given that the outcomes are not strictly comparable between program- and individual-level data; countryspecific contexts are different for Brazil and Ecuador and, of course, sample sizes and data variation are remarkably different across all our samples. Nonetheless, all our estimations tell a 31
consistent story: outcomes are indeed associated with program quality determinants; labor market outcomes, in particular, are strongly associated with T&C and LPS determinants.

We find two practices that contribute to _all_ labor market outcomes in Brazil and Ecuador— teaching numerical competencies and providing labor market information. One of these—teaching numerical competencies—is positively associated to _all_ labor market outcomes in the regional analysis for the five countries. Besides their intrinsic importance, numerical competencies may also proxy for related skills such as logical reasoning, problem solving, and critical thinking (World Bank 2019). Further, programs that teach these skills may contribute much to student outcomes given the serious mathematical deficiencies of incoming students. Providing labor market information, in turn, is quite rare in LAC, where HEIs do not view it as their responsibility to assist students in their job search. By providing that information, HEIs take a first step towards placing their graduates and engaging with this process.

# **5.2 Variance decompositions**

## **Program-level regressions**

Despite the rich set of explanatory variables in our LASSO regressions, we explain little of the observed variation in outcomes: about 8-9 percent for academic outcomes and 13-18 percent for labor market outcomes (see R-squared values in Table 3). This is consistent with recent work by Filmer et al. (2021) using machine-learning techniques to explore teacher value added and with Dinarte et al. (2022) on SCP value added in Colombia.

We now focus on the explained variation of outcomes based on Shapley-Owen decompositions (Table 3 and Figure A1). Taken together, quality determinants account for a sizable 50-60 percent of the explained variation in dropout rate, ETG and formal employment but only 15 percent for wages, for which 71 percent of the explained variation is accounted for by country fixed effects.

32
Quality determinants explain much of the variance in academic outcomes (determined mostly within the institution) and of formal employment (since institutions may vary in their ability to place students). However, they explain little of the variance in wages, which are monetary outcomes and therefore sensitive to the national context. Field of study accounts for a substantive share of explained outcome variation (6 to 21 percent, depending on the outcome), consistent with the documented cross-field variation in dropout rates, net returns, and SCP value added (Ferreyra et al. 2017, Dinarte et al. 2022, Ferreyra et al. 2021).

Among quality determinant categories, T&C accounts for the largest share of explained variation of ETG (25%), formal employment (22%) and wages (8%), and LPS accounts for the largest share of explained variation of dropout rates (18 percent). Costs, in turn, explain a nonnegligible share (13 percent) of the explained variation of academic outcomes but nothing of labor market outcomes. Student and program characteristics account for very little of the explained outcome variation but HEI characteristics account for more, particularly in the case of the academic outcomes (10 percent for dropout rates and 5 percent for ETG).

# **Individual-level regressions**

**Brazil.** Despite having student and peer administrative data in addition to survey-level data, we explain relatively little (12-15 percent) of the variation in graduation and labor market outcomes (Table 4). Nonetheless, the role of student and peer administrative variables is considerable for labor market outcomes: together they account for 30-40 percent of their explained variation (Table 4 and Figure A2). In contrast, they explain little of the variation in graduation, much of which is explained by field of study (24 percent) and HEI characteristics (32 percent), consistent with program-level findings. As in the latter, field of study explains much variation in graduation, while the geographic unit (state, in this case) accounts for most (26-37%) of the variation in labor market

33
outcomes. The role of quality determinants is non-negligible: overall, they account for a sizable 20 percent of the explained variation in labor market outcomes and 40 percent for graduation, mostly through T&C determinants.

**Ecuador.** Our regressions for Ecuador explain a higher share (between 27 and 37 percent) of variation in labor market outcomes than for Brazil or survey data (Table 5). The specification is not exactly comparable to that of Brazil because it does not include geographic unit fixed effects (due to small sample sizes). Still, individual and peer administrative variables account for about 40 percent of explained variation, as in Brazil (Table 5 and Figure A3). Consistent with previous results, field accounts for 10-15 percent of the explained variation and, overall, quality determinants account for 32-28 percent of it, mostly due to T&C and faculty, each of which captures about 10-15 percent of explained variation.

**_Program- and individual-level data—taking stock._** Overall, we do not explain much of the observed variation in student- or program-level outcomes. Nonetheless, quality determinants in the program-level regressions account for about 50-60 percent of the observed variation in dropout rates, ETG and formal employment, and about 20-40 percent of the individual-level regressions for academic and labor market outcomes, with the largest explanatory power generally accruing to T&C practices. Especially for academic outcomes, field and HEI characteristics are highly explanatory. Geography, in turn, is highly explanatory for labor-market outcomes. The salient role of quality determinants—together with our previous findings on the association of specific determinants to outcomes—suggests that adopting certain practices might improve outcomes for some programs and shrink their worrisome variation.

# **6 Conclusions**

34
Little is known about what determines higher education programs’ quality—namely, the program practices and inputs that contribute to good student outcomes. The rich data collected by the WBSCPS provides a unique opportunity to make inroads into this issue for a specific type of higher-education program, SCPs. We collected program-level data on quality determinants; HEI, student body, and program characteristics; and aggregate outcomes for 2,103 SCPs in five countries in LAC. We complemented this novel dataset with individual-level information on academic and labor market outcomes from Brazil and Ecuador. We document a large variation in program quality determinants and outcomes and exploit it to identify the practices and inputs associated with better outcomes after controlling for student, program, and HEI characteristics.

We find that outcomes are generally associated with quality determinants from multiple categories. While the specific outcome predictors vary by outcome and across analyses, two practices are positively associated to _all_ labor market outcomes based on individual-level data from Brazil and Ecuador—teaching numerical competencies and providing labor market information—and one of these—teaching numerical competencies—is positively associated with labor market outcomes in the survey countries based on program-level data. Besides their intrinsic importance, numerical competencies may proxy for related skills such as logical reasoning, problem solving, and critical thinking and may remediate students’ cognitive deficiencies at entry. By providing labor market information, programs take a first step towards placing their graduates and break with the LAC tradition of not assisting graduates in their job search. Further, we find that program quality determinants account for a substantial share (15-60 percent depending on the regression and outcome) of the explained variation in academic and labor market outcomes. Taken together, these findings suggest that the adoption of the quality determinants identified as outcome predictors—shrinking the gap in quality determinants—might also shrink the gap in outcomes.

35
Some final caveats are in order. First, a negative association between a determinant and an outcome does not indicate that the determinant is undesirable. Nonetheless, it indicates the need to focus on that specific determinant and assess how it fits with the program’s goals. For example, our estimates do not imply that training students for job interviews is undesirable but indicate the need to understand why this practice might detract from student outcomes in some settings or how it might be interpreted by employers. Second, we do not claim to have identified the program determinants that causally make one program better than another (the individual-level regressions, however, bring us closer to that point than the program-level ones). Nonetheless, our findings— the first of their kind—are of great interest for any country seeking to promote SCPs. They can inform the design and replication of high-quality programs as well as the regulatory mechanisms to ensure the adoption of good practices on the part of programs and institutions. They can also inspire more detailed, nuanced data collection on programs and institutions and encourage the development of effective individual-level information systems, an endeavor that would yield much deeper insights on what makes higher education good.

36
# **References**

- Aghion, P., Dewatripont, M., Hoxby, C., Mas-Colell, A., Sapir, A. and Jacobs, B. (2010), ‘The governance and performance of universities: evidence from Europe and the US’, _Economic Policy_ , 25(61), p7-59.

- Almeida, R., Anazawa, L., Menezes Filho, N., and Vasconcellos, L. (2015). ‘Investing in Technical and Vocational Education and Training’, _Policy Research Working Paper No.7246. World Bank, Washington, DC._

- Andrews, R., Li, J., Lovenheim, M. F. (2014). ‘Heterogeneous paths through college: Detailed patterns and relationships with graduation and earnings’, _NBER Working Paper_ , w19935.

- Andrews, C. J., Lowrie, K., Smart, M. & Tsoulou, I. (2020). ‘What Is the Role of International Scholarship in JPER?’, _Journal of Planning Education and Research_ , 40(1), p6-7.

- Armona, L., Chakrabarti, R., & Lovenheim, M. F. (2018), ‘How does for-profit college attendance affect student loans, defaults and labor market outcomes?’, _NBER Working Paper_ , w25042.

- Barrera-Osorio, F., & Bayona-Rodríguez, H.  (2019). ‘Signaling or Better Human Capital: Evidence from Colombia’, _Economics of Education Review_ 70, p20–34.

- Bahr, P. R. (2013), ‘Classifying Community Colleges Based on Students’ Patterns of Use’, _Res High Educ_ , 54, p433–460.

- Bahr, P. R. (2016). ‘The earnings of community college graduates in California’, _Center for Analysis of Postsecondary Education and Employment Working Paper_ .

- Bailey, T. R. (2015), ‘Redesigning America's Community Colleges: A Clearer Path to Student Success’, Harvard University Press.

- Barrow, L., Richburg-Hayes, L., Rouse, C. E. & Brock, T. (2014), ‘Paying for performance: The education impacts of a community college scholarship program for low-income adults’, _Journal of Labor Economics_ , 32(3), p563-599.

- Belloni, A., Chernozhukov, V.  & Hansen, C. (2014), “High-Dimensional Methods and Inference on Structural and Treatment Effects.” _Journal of Economic Perspectives_ , 28 (2), p29–50.

- Belloni, A., Chernozhukov, V., Chetverikov, D. & Kato, K. (2015), ‘Some new asymptotic theory for least squares series: Pointwise and uniform results’, _Journal of Econometrics_ , 186(2), p345-366.

- Bettinger, E. P. & Long, B. T. (2005), ‘Do Faculty Serve as Role Models? The Impact of Instructor Gender on Female Students’, _The American Economic Review_ , 95(2), p152-157.

- Bettinger, E. P. & Long, B. T. (2009), ‘Addressing the needs of underprepared students in higher education: does college remediation work?’, _Journal of Human Resources_ , 44(3), p736-771.

- Bettinger, E. P., Fox, L., Loeb, S. & Taylor, E. S. (2017), ‘Virtual Classrooms: How Online College Courses Affect Student Success’, _The American Economic Review_ , 107(9), p2855-2875.

- Bloom, N., Lemos, R., Sadun, R. & Van Reenen, J. (2015), ’Does management matter in schools?’, _The Economic Journal_ , 125, p647–674.

- Bloom, N. Draca, M. & van Reenen, J. (2016), ‘Trade induced technical change? The impact of Chinese imports on innovation, IT and productivity’, _Review of Economic Studies_ , 83, p87–117

- Bloom, N., Lemos, R., Sadun, R. & Van Reenen, J. (2020), ‘Healthy business? Managerial education and management in health care’, _The Review of Economics and Statistics_ , 102(3), p506–517.

37
- Boatman, A. & Long, B. T. (2018), ‘Does remediation work for all students? How the effects of postsecondary remedial and developmental courses vary by level of academic preparation’, _Educational Evaluation and Policy Analysis_ , 40(1), p29–58,

- Bound, J., Lovenheim, M. F. & Turner, S. (2010). "Why Have College Completion Rates Declined? An Analysis of Changing Student Preparation and Collegiate Resources", _American Economic Journal: Applied Economics_ , 2 (3), p129-57.

- Bühlmann P., van de Geer S. (2011) Lasso for linear models. In: Statistics for High-Dimensional Data. Springer Series in Statistics. Springer, Berlin, Heidelberg.

- Carrell, S. E., Page, M. E. & West, J. E. (2010), ‘Sex and science: How professor gender perpetuates the gender gap’, _The Quarterly Journal of Economics_ , 125(3), p1101–1144.

- Carrell, S. E. & Kurlaender, M. (2016), ‘Estimating the productivity of community colleges in paving the road to four-year success’, _NBER Working Paper_ , w22904.

- Castleman, B. L. & Terry, B. (2016), ‘Long Looking beyond enrollment: The causal effect of need-based grants on college access, persistence, and graduation’, _Journal of Labor Economics_ , 34(4), p1023-1073.

- Cellini, S. R. & Turner, N. (2019), ‘Gainfully employed?: Assessing the employment and earnings of forprofit college students using administrative data’, _Journal of Human Resources_ , 54(2), p342-370.

- Cellini, S. R., & Grueso, H. (2021), ‘Student learning in online college programs’, _AERA Open_ , 7(1), p1– 18

- Chetverikov, D., Liao, Z. Chernozhukov, V. (2019). ‘On cross-validated lasso’ _The Annals of Statistics_ (forthcoming).

- Chin, A., Daysal, M. N. & Imberman, S. A. (2013), ‘Impact of bilingual education programs on limited English proficient students and their peers: Regression discontinuity evidence from Texas’, _Journal of Public Economics_ , 107, p63–78.

- Cohodes, S. R. & Goodman, J. S. (2014), ‘Merit aid, college quality, and college completion: Massachusetts' Adams scholarship as an in-kind subsidy’, _American Economic Journal: Applied Economics_ , 6(4), p251-285.

- Dadgar, M. and Trimble, M. J. (2015), ‘Labor market returns to sub-baccalaureate credentials: How much does a community college degree or certificate pay?’, _Educational Evaluation and Policy Analysis_ , 37(4), p399–418.

- Daniel, H. D., Mittag, S. and Bornmann, L. (2007), ‘The Potential and Problems of Peer Evaluation in Higher Education and Research’, In Quality Assessment for Higher Education in Europe, edited by A. Cavalli, 71–82. London: Portland Press.

- De Vlieger, P., Jacob, B. & Stange, K. (2017), ‘Measuring instructor effectiveness in higher education’, _NBER Working Paper_ , w22998.

- Deming, D. J., Lovenheim, M. & Patterson, R. W. (2016), ‘The competitive effects of online education’, NBER Working Paper, no. w22749.

- Denning, J. T. (2017), ‘College on the cheap: Consequences of community college tuition reductions’, _American Economic Journal: Economic Policy_ , 9(2), p155-188.

- Di Meglio, G., Barge-Gil, A., Camiña, E., & Moreno, L. (2022), ‘Knocking on employment’s door: internships and job attainment’, _Higher Education_ , 83(1), p137-161.

- Dobbie, W. & Fryer, R. G. (2013), ‘Getting beneath the Veil of Effective Schools: Evidence from New York City’, _American Economic Journal: Applied Economics_ , 5(4), p28-60.

38
- Dynarski, S. and Scott-Clayton, J. (2013), ‘Financial aid policy: Lessons from research’, _The Future of Children_ , 23(1), p67-91.

- Ferreyra, M., Avitabile, C., Botero, J., Haimovich, F. & Urzúa, S.  (2017), ‘At a Crossroads: Higher Education in Latin America and the Caribbean’, Washington, DC: World Bank Group.

- Dinarte, Lelys; Ferreyra, M. M., Melguizo, T., & Sanchez, A. (2022), “The Contribution of Short-Cycle Programs to Student Outcomes : Evidence from Colombia.” _Policy Research Working Paper_ 9424, World Bank, Washington, DC.

- Ferreyra, M.M, Dinarte, L., Urzúa, S., & Bassi, M. (2021). “The Fast Track to New Skills: Short-Cycle Higher Education Programs in Latin America and the Caribbean.” _Forthcoming_ . World Bank, Washington, DC.

- Field, E. (2009), ‘Educational debt burden and career choice: Evidence from a financial aid experiment at NYU law school’, _American Economic Journal: Applied Economics_ , 1(1), p1-21.

- Figlio, D., Rush, M. & Yin, L. (2013), ‘Is it live or is it internet? Experimental estimates of the effects of online instruction on student learning’, _Journal of Labor Economics_ , 31(4), p763-784.

- Filmer, D., Nahata, V., & Sabarwal, S. (2021). ‘Preparation, Practice, and Beliefs: A Machine Learning Approach to Understanding Teacher Effectiveness,’ _Policy Research Working Paper,_ WPS 9847. Washington, D.C.: World Bank Group.

- Grosz, M. (2020), ‘The returns to a large community college program: Evidence from admissions lotteries’, _American Economic Journal: Economic Policy_ , 12(1), p226–25

- Hoffmann, F. & Oreopoulos, P. (2009), ‘A Professor like Me: The Influence of Instructor Gender on College Achievement’, _The Journal of Human Resources_ , 44(2), p479-494.

- Hoxby, C. M. & Bulman, G. B. (2015), ‘The effects of the tax deduction for postsecondary tuition: Implications for structuring tax-based aid’, _NBER Working Paper_ , w21554.

- Hoxby, C. M., & Stange, K. (Eds.). (2019), ‘Productivity in higher education’, Chicago, IL: University of Chicago Press.

- Huettner, F., & Sunder, M. (2012), ‘Axiomatic arguments for decomposing goodness of fit according to Shapley and Owen values’, _Electronic Journal of Statistics_ , 6, p1239-1250.

- Jaeger, D. A., Nunley, J. M., Seals, A., & Wilbrandt, E. J. (2020), ‘The demand for interns’, _NBER Working Paper_ , w26729.

- Jaggar, S. S. & Xu, D. (2016), ‘How do online course design features influence student performance?’, _Computers & Education_ , 95, p270-284.

- Jepsen, C., Troske, K., and Coomes, P. (2014), ‘The Labor-Market Returns to Community College Degrees, Diplomas, and Certificates’, _Journal of Labor Economics_ , 32(1):95–121.

- Johnson, H. & Mejia, M. C. (2014), ‘Online learning and student outcomes in California’s community colleges’, Public Policy Institute of California, viewed June 18 2021, https://www.ppic.org/wpcontent/uploads/R_514HJR.pdf

- Kane, T. J. & Staiger, D. O. (2012), ‘Gathering Feedback for Teaching: Combining High-Quality Observations with Student Surveys and Achievement Gains’, Research Paper, Measures of Effective Teaching Project, Bill & Melinda Gates Foundation.

- Kane, T. J., Boatman, A., Kozakowski, W., Bennett, C., Hitch, R. & Weisenfeld, D. (2021), ‘Is college remediation a barrier or a boost? Evidence from the Tennessee SAILS program’ _, Journal of Policy Analysis and Management_ , 40(3), p1–31.

39
- Kofoed, M. S. & McGovney, E. (2019), ‘The effect of same-gender or same-race role models on occupation choice evidence from randomly assigned mentors at West Point’, _Journal of Human Resources_ , 54(2), p430-467.

- Kutscher, M., Nath, S. & Urzua, S. S. (2020), Centralized Admission Systems and School Segregation: Evidence from a National Reform’, _IZA Discussion Paper_ , 13305.

- Lim, J. & Meer, J. (2019), ‘Persistent effects of teacher-student gender matches’, _Journal of Human Resources_ , 0218-9314R4.

- Liu, V. Y. T., Belfield, C. R. & Trimble, M. J. (2015), ‘The medium-term labor market returns to community college awards: Evidence from North Carolina’, _Economics of Education Review_ , 44, p4255.

- Lovenheim, M. F., & Smith, J. (2022). ‘Returns to Different Postsecondary Investments: Institution Type, Academic Programs, and Credentials’, _NBER Working Paper_ , w29933.

- Martorell, P., McCall, B. & McFarlin, I. (2014), ‘Do public tuition subsidies promote college enrollment? Evidence from community college taxing districts in Texas’, _US Census Bureau Center for Economic Studies Paper_ , CES-WP- 14-32, p1-48.

- McCormack, J., Propper, C., & Smith, S. (2014), ‘Herding cats? Management and university performance.’, _The Economic Journal_ , 124(578), pF534-F564.

- Melguizo, T. & Wainer, J. (2016), ‘Toward a set of measures of student learning outcomes in higher education: evidence from Brazil’, _High Education_ , 72, p381–401.

- Melguizo, T., Sanchez, F. & Velasco, T. (2016), ‘Credit for low-income students and access to and academic performance in higher education in Colombia: a Regression Discontinuity Approach’, _World Development_ , 80, p61-77.

- Mountjoy, J. (2021), ‘Community colleges and upward mobility’, _NBER Working Paper_ , w29254,

- Owen, G. (1977), ‘Values of games with a priori unions.’ In _Essays in Mathematical Economics & Game Theory_ (R. Henn and O. Moeschlin, eds.) 76–88. Springer, Berlin

- Porter, C., & Serra, D. (2020), ‘Gender differences in the choice of major: The importance of female role models’, _American Economic Journal: Applied Economics_ , 12(3), p226-54.

- Plewa, C., Galán-Muros, V., & Davey, T. (2015), ‘Engaging business in curriculum design and delivery: a higher education institution perspective’, _Higher Education_ , 70(1), p35-53.

- Saavedra, J. E. (2009). ‘The Returns to College Quality: A Regression Discontinuity Analysis’ Harvard University, Cambridge, MA.

- Sacerdote, B. (2011). ‘Peer effects in education: How might they work, how big are they and how much do we know thus far?’ In _Handbook of the Economics of Education_ (Vol. 3, pp. 249-277). Elsevier.

- Stange, K. (2012), ‘Ability Sorting and the Importance of College Quality to Student Achievement: Evidence from Community Colleges’ _Education Finance and Policy_ , 7(1), p74-105.

- Scott-Clayton, J. (2011), ‘On money and motivation: A quasi-experimental analysis of financial incentives for college achievement’, _Journal Human Resources_ , 46(3), p614-646.

- Scott-Clayton, J. & Rodriguez, O. (2015), ‘Development, discouragement, or diversion? New evidence on the effects of college remediation policy’, _Education Finance and Policy_ , 10(1), p4-45.

- Shapley, L. S. (1953). ‘A value for n-person games.’ In _Contributions to the Theory of Games_ , (H. W. Kuhn and A. W. Tucker, eds.) II 307–317. Princeton University Press, Princeton.

40
- Shavelson, R. J., Domingue, B. W., Mariño, J. P., Mantilla, A. M., Forero, A. M. & Wiley, E E. (2016), ‘On the practices and challenges of measuring higher education value added: the case of Colombia’, _Assessment & Evaluation in Higher Education_ , 41(5), p695-720.

- Smith, J. & Stange, K. (2016), ‘A New Measure of College Quality to Study the Effects of College Sector and Peers on Degree Attainment’, _Education Finance and Policy_ , 11(4), p369-403.

- Stevens, A. H., Kurlaender, M., and Grosz, M. (2019), ‘Career technical education and labor market outcomes evidence from California community colleges’, _Journal of Human Resources_ , 54(4), p986– 1036.

- Rodriguez, J., Reyes, L. & Urzua, S. (2016), “Heterogenous Economic Returns to Post-secondary Degrees: Evidence from Chile,” _The Journal of Human Resources_ , Spring 2016, Vol. 51, n. 2, p416-460.

- Teixeira, P., Rocha, V., Biscaia, R. & Cardoso, M. F. (2013), ‘Competition and diversification in public and private higher education’, _Applied Economics_ , 45(35), p4949-4958.

- Wesley Routon, P., & Walker, J. K. (2019), ‘College internships, tenure gaps, and student outcomes: A multiple-treatment matching approach’, _Education Economics_ , 27(4), p383-400.

- Wiederspan, M. (2016), ‘Denying loan access: The student-level consequences when community colleges opt out of the Stafford loan program’, _Economics of Education Review,_ 51, p79-96.

- World Bank (2019). World Development Report 2019: The Changing Nature of Work. Washington, DC: World Bank. doi:10.1596/978-1-4648-1328-3.

- Xu, D. & Jaggars, S. S. (2011), ‘The effectiveness of distance education across Virginia's community colleges: Evidence from introductory college-level Math and English courses’, _Educational Evaluation and Policy Analysis_ , 33(3), p360-377.

- Xu, D. & Dadgar, M. (2018), ‘How effective are community college remedial math courses for students with the lowest math skills?’, _Community College Review_ , 46(1), p62-8.

- Yi, G. (2018), ‘Impact of internship quality on entrepreneurial intentions among graduating engineering students of research universities in China’, _International Entrepreneurship and Management Journal_ , 14(4), p1071-1087.

- Zou, H (2006), ‘The adaptive LASSO and its oracle properties’, _Journal of the American Statistical Association_ , 101(476), p1418-1429.

41
# **Tables and Figures**

## **Table 1. Descriptive Statistics** **_Using Program-Level Data from the WBSCPS and Administrative Sources_**

|**Variable**|**Mean**|**Std.**<br>**Dev.**|**Min.**|**Median**|**Max.**|**Obs.**|
|---|---|---|---|---|---|---|
|**Panel A. Programquality determinants**|||||||
|_Infrastructure_|||||||
|Has enough equipment or tools for practice|0.72|0.45|0|1|1|2,103|
|Program offers at least one online class|0.35|0.48|0|0|1|2,103|
|_Training and curriculum_|||||||
|Teaches numerical competencies|0.80|0.40|0|1|1|2,103|
|Promotes work under hardship or pressure|0.84|0.36|0|1|1|2,103|
|Curriculum is fixed|0.70|0.46|0|1|1|2,103|
|Time assigned to practical training (%)|46.45|16.56|4|50|80|2,072|
|<br>Graduation requirement|||||||
|Test|0.40|0.49|0|0|1|2,103|
|Thesis or research project|0.37|0.48|0|0|1|2,103|
|Internships outside institution are mandatory|0.58|0.49|0|1|1|2,103|
|Remediation support|||||||
|Remediation classes before starting the program|0.51|0.50|0|1|1|2,103|
|Remediation classes during the program|0.57|0.50|0|1|1|2,103|
|Years since last update to curriculum|2.91|2.69|0|2|25|1,946|
|More than once per year:|||||||
|Analyze student performance to solve problems|0.86|0.35|0|1|1|2,103|
|Collect student satisfaction data|0.69|0.46|0|1|1|2,103|
|_Costs_|||||||
|Annual tuition (2019 PPP USD)|2,244|1,762|0|2,367|25,515|2,103|
|HEI has received funding from|||||||
|<br>Private sector|0.20|0.40|0|0|1|2,103|
|Government|0.34|0.47|0|0|1|<br>2,103|
|_Link with productive sector_|||||||
|Engagement with firms|||||||
|Somebody in charge of industry relations|0.82|0.38|0|1|1|2,103|
|Industry has agreements with HEI to hire program grads|0.39|0.49|0|0|1|2,103|
|Industry has agreements to train faculty|0.35|0.48|0|0|1|2,103|
|Collect data on employment or employers' satisfaction|0.56|0.50|0|1|1|2,103|
|Communicate with local firms about their needs|0.52|0.50|0|1|1|2,103|
|Job search assistance|||||||
|HEI trains students for job interviews|0.69|0.46|0|1|1|2,103|
|HEI provides job market information|0.81|0.40|0|1|1|2,103|
|<br>HEI has an employment center|0.60|0.49|0|1|1|2,103|
|_Faculty_|||||||
|Number of faculty|20.00|18.70|1|15|200|2,076|
|<br>Percent of faculty|||||||
|<br>with BA degree|82.20|29.40|0|100|100|2,043|
|with graduate degree|48.60|32.60|0|43|100|1,998|
|working full-time|38.40|30.30|0|31|100|1,987|
|with 5yrs+ industry experience|55.70|33.10|0|56|100|1,988|
|<40 years old|40.00|29.20|0|34|100|1,961|

41
|that are women|34.00|23.50|0|30|100|2,021|
|---|---|---|---|---|---|---|
|Faculty are evaluated more than once per year|0.85|0.360|0|1|1|2,103|
|Almost all or all faculty participated in professional training last year|0.55|0.50|0|1|1|2,103|
|_Other Practices_|||||||
|Update or review admin data more than once per year|0.66|0.47|0|1|1|2,103|
|Admission requirements|||||||
|General or specific knowledge test|0.59|0.49|0|1|1|2,103|
|Interview|0.50|0.50|0|1|1|2,103|
|Min. score in HS GPA or national entry test|0.66|0.47|0|1|1|2,103|
|Percent of governing body that belongs to:|||||||
|Private sector|18.90|21.20|0|12|100|2,103|
|Government|11.90|16.40|0|1|100|2,103|
|Faculty|39.00|27.80|0|33|100|2,103|
|**Panel B. Student body, program, and institution characteristics**|||||||
|_Student body characteristics_|||||||
|Academic deficiencies|||||||
|Mathematics is lacking in incoming students|0.82|0.39|0|1|1|2,103|
|Reading is lacking in incoming students|0.70|0.46|0|1|1|2,103|
|Writing is lacking in incoming student|0.68|0.47|0|1|1|2,103|
|Percent of students that are|||||||
|25+ years old|28.94|29.30|0|20|100|2,103|
|full-time|43.89|39.00|0|33|100|2,103|
|Women|38.19|29.20|0|40|100|2,103|
|Student body characteristics (PCA score)|-0.02|1.40|-2.78|-0.31|4.88|2,103|
|_Program characteristics_|||||||
|Program duration (semesters)|5.20|0.97|2|6|8|2,101|
|Program has high quality accreditation|0.19|0.39|0|0|1|2,103|
|Total number of students in the program last year|221.60|332.80|1|125|4,321|2,030|
|Program age (years)|11.50|9.46|0|10|70|2,103|
|Program characteristics (PCA score)|0.02|1.25|-7.59|0.20|2.43|2,082|
|_Institution characteristics_|||||||
|HEI is public|0.30|0.46|0|0|1|2,103|
|HEI is a university|0.21|0.41|0|0|1|2,103|
|HEI is for profit|0.20|0.40|0|0|1|2,103|
|HEI age|37.84|30.80|1|32|481|2,094|
|Number of programs in the HEI|21.59|36.30|1|10|268|2,103|
|<br>HEI characteristics (PCA score)|0.08|1.16|-1.84|-0.21|9.57|2,094|
|**Panel C. Noise controls**|||||||
|Survey conducted during COVID|0.48|0.50|0|0|1|2,103|
|Number of attempts to complete the survey|8.36|3.16|1|9|17|2,103|
|Survey completed by phone|0.33|0.47|0|0|1|2,103|

_Sources_ : Own calculations using WBSCPS and administrative data.

_Notes_ : This table shows the descriptive statistics of the main variables used in the analysis. An observation corresponds to a program. Dummy variables included in the list are those with means between 0.1 and 0.9. Statistics are weighted by WBSCPS sampling weights. Panel A refers to quality determinants, presented by category. Panel B refers to characteristics of the student body, program, and higher education institution (HEI); Panel C refers to survey noise controls. Total number of surveys completed is 2,103. Values in the “Obs.” column vary depending on the number of valid responses. Tuition is presented in dollars but transformed in logs for estimation. Mean of PCA Scores are different than zero due to the use of sampling weights. All variables in panel B are included in the corresponding indexes (PCA Scores), except for “HEI is public,” which is included separately in the corresponding regressions as a “fixed” control.

42
# **Table 2. Descriptive Statistics for Students in Brazil and Ecuador**

**_Based on Individual-Level Data for Students Matched to Surveyed Programs_**

|**Variable**|**Br**|**azil**|**Ecu**|**ador**|
|---|---|---|---|---|
||**Mean**|**Std. Dev.**|**Mean**|**Std. Dev.**|
|**Panel A.Student characteristics**|||||
|Age|24.53|5.65|24.66|3.40|
|Female|0.59|0.49|0.62|0.49|
|Mother's education:|||||
|Less than primary|0.12|0.32|-|-|
|Primary school|0.16|0.36|0.15|0.36|
|High school|0.19|0.39|0.28|0.45|
|Higher education|0.07|0.26|0.52|0.50|
|Unknown|0.46|0.49|0.05|0.22|
|Student has children|-|-|0.27|0.44|
|Socioeconomic index (std)|-|-|0.01|0.97|
|Student administrative variables(PCA Score)|0.01|1.25|0.07|1.32|
|**Panel B. Peer(average) characteristics**|||||
|Age|24.59|1.70|24.66|1.81|
|Percentage of female peers|0.56|0.27|0.62|0.30|
|Percentage of peers by mother's education:|||||
|Less than primary|0.12|0.06|-|-|
|Primary school|0.15|0.06|0.15|0.14|
|High school|0.18|0.07|0.28|0.19|
|Higher education|0.06|0.06|0.52|0.24|
|Unknown|0.49|0.13|0.05|0.06|
|Percentage of peers with children|-|-|0.27|0.18|
|Peers' socioeconomic index (std)|-|-|0.01|0.37|
|Peers' administrative variables(PCA Score)|-0.22|1.57|0.14|1.36|
|**PanelC. Own andpeers’previous labor market outcomes**|||||
|Percent of time employed before graduation/entry|46.32|43.50|19.02|36.24|
|Average peers' percent of time employed before graduation/entry|49.13|13.50|19.02|21.26|
|Employed at least one month before graduation/entry|0.64|0.48|0.26|0.44|
|Pct. ofpeers employed at least one month beforegraduation/entry|0.65|0.18|0.26|0.25|
|**Panel D.Outcomes**|||||
|Average monthly wage conditional on working (USD, PPP)|1,031.40|577.56|876.06|381.36|
|Average monthly wage (USD, PPP)|715.62|678.84|300.29|472.02|
|Percent of time employed after graduation|53.37|43.97|24.01|38.23|
|Employed at least one month aftergraduation|0.70|0.46|0.35|0.48|
|_Number of observations_|7,843||1,239||

_Sources_ : Own calculations using individual-level administrative data for Brazil and Ecuador. For more details on data sources and variable definitions, see Section 3.3 and Appendix 1.

_Notes_ : In this table, the unit of observation is a program graduate. Statistics are weighted using WBSCPS sampling weights. A higher value of the socioeconomic index indicates a higher socioeconomic status. For a given student, her peers are the other students in her program and cohort; cohorts are defined in Section 3. Means of PCA scores are different than zero due to the use of sampling weights. In Panel C, previous labor market outcomes are pre-graduation (Ecuador) and pre-enrollment (Brazil). Average monthly wage conditional on working corresponds to individuals who are employed at least one month after graduation. For the remaining individuals, average monthly wage equals zero; therefore, the “Average monthly wage” row shows unconditional average monthly wages (which equal zero for individuals who do not work formally at all). Wage statistics trim off the 1<sup>st</sup> and 99<sup>th</sup> percentile. Purchasing power parity (PPP) adjustment of wages uses the 2019 (Ecuador) or 2017 (Brazil) PPP conversion factor.

43
# **Table 3. R-Squared Shapley-Owen Decomposition** **_Estimations using Program-Level Data_**

|**Outcome:**|**(1)**<br>**Dropout**<br>**Rate**|**(2)**<br>**Extra**<br>**Time to**<br>**Graduate**|**(3)**<br>**Formal**<br>**Employment**|**(4)**<br>**Wages**|
|---|---|---|---|---|
|_Percent of explained variation attributable to:_|||||
|All quality determinants|60.22|51.95|50.51|15.48|
|Infrastructure|-|-|6.54|0.25|
|Costs|12.85|12.92|-|-|
|Training and curriculum|11.45|24.59|22.36|7.68|
|Faculty|17.22|12.47|12.41|4.03|
|Link with productive sector|18.46|1.97|6.99|0.84|
|Other practices|0.23|-|2.21|2.68|
|Student characteristics (PCA Score)|0.24|0.25|2.15|0.04|
|Program characteristics (PCA Score)|0.60|0.31|0.23|1.84|
|HEI characteristics (PCA Score)|10.48|4.93|2.30|3.32|
|Country fixed effects|7.37|31.57|38.58|71.19|
|Field of study fixed effects|21.08|10.99|6.21|8.13|
|R-squared|0.080|0.087|0.134|0.178|
|Obs.<br>_Source_: Own estimations using WBSCPS data. For varia|1,526<br>ble definitions, see|1,693<br>Section 3 and A|1,270<br>ppendix 1.|1,752|

_Source_ : Own estimations using WBSCPS data. For variable definitions, see Section 3 and Appendix 1. _Notes_ : This table present results from the R-squared Shapley-Owen decomposition for the regressions reported in Tables A11 (dropout rate and extra time-to-graduate) and A12 (formal employment and wages), estimated with WBSCPS program-level data. For each regression, the table shows R-squared (net of the variation explained by survey noise variables) and the percent of the (net) explained variation attributable to each set of variables. “Obs.” indicates number of programs (equal to the number of observations in the underlying regression).

44
# **Table 4. R-Squared Shapley-Owen Decomposition for Brazil** **_Estimations Using Individual-Level Data_**

||**(1)**|**(2)**|**(3)**|**(4)**|
|---|---|---|---|---|
|||**Ever**|**Percent**||
|_Outcomes:_|**Graduation**|**employed**<br>**after**<br>**graduation**|<br>**of time**<br>**employed**|**Wages**|
|_Percent of explained variation attributable to:_|||||
|All quality determinants|39.98|19.54|24.62|18.89|
|Infrastructure|4.19|-|0.99|-|
|Costs|7.34|0.45|1.02|0.47|
|Training and curriculum|10.81|7.90|8.30|7.51|
|Faculty|5.49|3.32|2.82|3.48|
|Link with productive sector|8.05|2.37|4.73|2.16|
|Other Practices|4.11|5.50|6.77|5.27|
|Student characteristics (PCA Score)|0.25|1.30|1.63|1.32|
|Program characteristics (PCA Score)|1.03|1.51|1.18|1.51|
|HEI characteristics (PCA Score)|32.42|0.81|0.63|0.73|
|Student administrative variables (PCA Score)|0.41|14.33|24.23|18.09|
|Peer administrative variables (PCA Score)|1.83|18.17|15.36|18.03|
|Field of study fixed effects|23.92|7.34|5.67|7.64|
|State fixed effects|0.16|36.71|25.79|33.51|
|Graduation year fixed effects|-|0.29|0.90|0.27|
|R-squared|0.134|0.135|0.122|0.149|
|Obs.|22,663|7,177|6,827|7,089|

_Source_ : Own estimations using data from the WBSCPS and Brazil individual-level data. For variable definitions, see Section 3 and Appendix 1.

_Notes_ : This table present results from the R-squared Shapley-Owen decomposition for the regressions reported in Table A13, estimated with individual- and program-level data for Brazil (states of Ceara and Sao Paulo). For each regression, the table shows R-squared (net of the variation explained by survey noise variables) and the percent of the (net) explained variation attributable to each set of variables. “Obs.” indicates number of students (equal to the number of observations in the underlying regression).

45
# **Table 5. R-Squared Shapley-Owen Decomposition for Ecuador** **_Estimations Using Individual-Level Data_**

||**(1)**|**(2)**|**(3)**|
|---|---|---|---|
|_Outcomes:_|**Ever**<br>**employed**<br>**after**<br>**graduation **|**Percent**<br>**of time**<br>**employed**|**Wages**|
|_Percent of explained variation attributable to:_||||
|All quality determinants|35.00|37.64|32.06|
|Infrastructure|-|1.98|-|
|Costs|-|0.84|-|
|Training and curriculum|15.53|12.39|12.45|
|Faculty|9.46|15.81|10.08|
|Link with productive sector|8.78|5.83|8.09|
|Other Practices|1.23|0.80|1.44|
|Student characteristics (PCA Score)|1.71|1.42|1.52|
|Program characteristics (PCA Score)|2.27|3.65|2.49|
|HEI characteristics (PCA Score)|6.52|2.75|6.32|
|Student administrative variables (PCA Score)|19.90|27.60|22.99|
|Peer administrative variables (PCA Score)|20.01|17.48|21.04|
|Field of study fixed effects|14.59|9.47|13.59|
|R-squared|0.265|0.354|0.282|
|Obs.|1,214|1,201|1,206|

_Source_ : Own estimations using data from the WBSCPS and Ecuador individual-level data. For variable definitions, see Section 3 and Appendix 1.

_Notes_ : This table present results from the R-squared Shapley-Owen decomposition for the regressions reported in Table A14, estimated with individual- and program-level data for Ecuador. For each regression, the table shows R- squared (net of the variation explained by survey noise variables) and the percent of the (net) explained variation attributable to each set of variables. “Obs.” indicates number of students (equal to the number of observations in the underlying regression).

46
<!-- Start of picture text --> Brazil Colombia Dominican Republic<br>8 =<br>8 8wo.<br>3<br>”<br>&3=<br>°o °o<br>0 20 40 «=260—Ss 80<br>0 20 40«=—60——s«80 0 20 40 «=660—Ss 80 Percent<br>Ecuador Peru<br>8 eo<br>8 S<br>$ 8ES<br>o o<br>0 20 40 «8660 S80 0 20 40 «#260 ~— 80<br>Percent Percent<br>Dropout by country<br><!-- End of picture text --> <!-- Start of picture text --> Brazil Colombia Dominican Republic<br>8 8 8<br>3 3 $<br>a<br>x<br>o<br>°o °o<br>-100 © 100 200 300<br>-100 0 100 200 300 -100 0 100 200 300 Percent<br>Ecuador Peru<br>8 .<br>& z5<br>o oOo<br>-100 0 100 200 300 -100 0 100 200 300<br>Percent Percent<br>Extra-time to degree by country<br><!-- End of picture text -->
<!-- Start of picture text --> fo}<br>ie}<br>Cc<br>vu<br>fl)<br>aot<br>°N<br>oio)Thin<br>Brazil Colombia Dominican Republic Peru Ecuador<br>Formal employment by country.<br><!-- End of picture text --> <!-- Start of picture text --> wo Brazil w Colombia “ Dominican Republic<br>+ v<br>Tv<br>m”<br>”<br>doNn N<br>oO o 85 9 95 10<br>8.5 9 95 10 8.5 9 95 10 Wages (Log, USD-PPP)<br>Ecuador Peru<br>wo<br>w<br>o><br>Tt<br>m<br>”<br>N. ZAIN N. D4 aa<br>8.5 7 95 10 8.5 9 9.5 10<br>Wages (Log, USD-PPP) Wages (Log, USD-PPP)<br>Wages by country<br><!-- End of picture text -->
<!-- Start of picture text --> Peer evaluation is very important in the faculty evaluation<br>* Percent offaculty working in the industry<br>HEI provides job market information @<br>Industry lends or provides equipment to program for student training @<br>HEI has received funding from government e<br>HEI has received funding from private sector @<br>Curriculum is fixed @<br>-4 -2 0 2 4<br>Percentage points<br>@ Training and curriculum @ Costs @ Link with productive sector Faculty<br><!-- End of picture text --> <!-- Start of picture text --> Evaluate faculty more than once per year<br>* Percent of faculty with SCP degree<br>* Percent of faculty working in the industry<br>Almost all or all faculty participated in professional training last year<br>Industry helps with student evaluation or curriculum design @<br>* Annual tuition 2019 PPP USD (log) @<br>Loans by banks are used by some students @<br>Teaches work under hardship / pressure @<br>Thesis or research project required for graduation @<br>HEI perception of labor market is very important to update program @<br>-10 5 0 5 10<br>Percentage points<br>@ Training and curriculum @ Costs @ Link with productive sector Faculty<br><!-- End of picture text -->
<!-- Start of picture text -->
* Percent of <40 years old faculty<br>* Percent of faculty with 5yrs+ industry experience<br>Collect data on employment or employers’ satisfaction more than once per year @<br>Program has staff assigned to collect grads' employment data @<br>HEI has an employment center @<br>Offers remediation classes during the program @<br>Teaches numerical competencies @<br>Has enough equipment or tools for practice<br>-10 0 Percentage points. 10 20<br>Infraestructure @ Training and curriculum @ Link with productive sector Faculty<br><!-- End of picture text -->

<!-- Start of picture text -->
* Percent of governing body that belongs to: current students e@<br>General or specific knowledge test is an admission requirement e@<br>* Percent of faculty that are women<br>* Percent of faculty with BA degree<br>* Number of faculty (log)<br>Offer non-class-based remediation @<br>Offers remediation classes during the program e<br>HEI perception of labor market is very important to update program e<br>Teaches numerical competencies e<br>Offers credits for longer degrees @<br>5 0 5 10<br>Percent<br>@ Training and curriculum Faculty @ Other Practices<br><!-- End of picture text -->
<!-- Start of picture text -->
* Percent of governing body that belongs to: current students @<br>Almost all or all faculty participated in professional training last year<br>* Percent of faculty working in the industry<br>Industry has internship agreements with HEI @<br>Communicate with local firms about their needs more than once per year @<br>Industry has agreements to train faculty @<br>Loans by banks are used by some students e<br>Collect student satisfaction data more than once per year @<br>Student feedback is very important to update program @<br>Government standards are very important to update program e<br>* Years since last update to curriculum e<br>Test required for graduation @<br>Analyze student performance to solve problems more than once per year @<br>Maintenance of largest lab: every year<br>Program offers at least one online class<br>-1 -.05 (e) 05 J 15<br>Percentage points<br>Infraestructure @ Training and curriculum © Costs<br>@ Link with productive sector Faculty @ Other Practices<br><!-- End of picture text -->

<!-- Start of picture text --> Update / review admin data more than once per year @<br>Interview is an admission requirement e@<br>Class planning is very important in faculty evaluation<br>Students and peers’ informal comments are very important in faculty evaluation<br>* Percent of faculty with graduate degree<br>Classroom observation is very important in faculty evaluation<br>Peer evaluation is very important in the faculty evaluation<br>Evaluate faculty more than once per year<br>HEI trains students for job interviews e@<br>Collect data on employment or employers' satisfaction more than once per year @<br>HEI providesjob market information e@<br>Loans by banks are used by some students @<br>Offers remediation classes during the program @<br>Offers remediation classes before starting the program @<br>Teaches persistence in complex tasks @<br>Student feedback is very important to update program @<br>Test required for graduation @<br>Teaches numerical competencies @<br>Analyze student performance to solve problems more than once per year @<br>-10 -5 0 5 10<br>Percentage points<br>Infraestructure @ Training and curriculum @ Costs<br>@ Link with productive sector Faculty @ Other Practices<br><!-- End of picture text -->
<!-- Start of picture text -->
* Percent of governing body that belongs to: government @<br>* Percent of governing body that belongs to: private sector @<br>Update / review admin data more than once per year @<br>General or specific knowledge test is an admission requirement @<br>Interview is an admission requirement r )<br>* Percent of faculty with BA degree<br>* Percent of faculty with graduate degree<br>Evaluate faculty more than once per year<br>Industry helps with student evaluation or curriculum design @<br>Collect data on employment or employers' satisfaction more than once per year @<br>HEI trains students for job interviews @<br>HEI provides job market information @<br>HEI has received funding from government @<br>Loans by banks are used by some students @<br>Collect student satisfaction data more than once per year @<br>Teaches numerical competencies e<br>Analyze student performance to solve problems more than once per year @<br>Test required for graduation @<br>>30% classes can be taken online<br>-10 -5 0 5 10<br>Percentage points<br>Infraestructure @ Training and curriculum @ Costs<br>@ Link with productive sector Faculty @ Other Practices<br><!-- End of picture text -->

<!-- Start of picture text --> Interview is an admission requirement s<br>Update / review admin data more than once per year &<br>Class planning is very important in faculty evaluation<br>Students and peers’ informal comments are very important in faculty evaluation<br>* Percent of faculty that are women<br>* Percent of faculty with graduate degree<br>Classroom observation is very important in faculty evaluation<br>Peer evaluation is very important in the faculty evaluation<br>Evaluate faculty more than once per year<br>HEI trains students for job interviews @<br>Collect data on employment or employers' satisfaction more than once per year s<br>HEI provides job market information s<br>Loans by banks are used by some students @<br>Offers remediation classes during the program @<br>Test required for graduation @<br>Student feedback is very important to update program @<br>Analyze student performance to solve problems more than once per year .<br>Teaches numerical competencies @<br>-50 -25 0 25 50 75<br>Percent<br>Infraestructure @ Training and curriculum @ Costs<br>@ Link with productive sector Faculty @ Other Practices<br><!-- End of picture text -->
<!-- Start of picture text --> Min scores in HS GPA or national entry test is an admission requirement e<br>Research skills are very important when hiring faculty<br>* Percent of faculty working in the industry<br>HEI provides job market information @<br>Somebody is in charge of industry relations @<br>HEI trains students for job interviews e@<br>Test required for graduation @<br>Teaches a foreign language e<br>Collect student satisfaction data more than once per year e@<br>Teaches numerical competencies e<br>-20 0 20 40<br>Percent<br>@ Training and curriculum = @ Link with productive sector Faculty @ Other Practices<br><!-- End of picture text --> <!-- Start of picture text --> Research skills are very important when hiring faculty<br>Students and peers' informal comments are very important in faculty evaluation<br>* Percent of faculty with SCP degree<br>* Percent of <40 years old faculty<br>* Percent of faculty working in the industry<br>* Percent of faculty with 5yrs+ industry experience<br>Classroom observation is very important in faculty evaluation<br>HEI coordinates job interviews with firms @<br>HEI has agreements with private firms to hire graduates .<br>Industry lends or provides equipment to program for student training @<br>HEI trains students for job interviews és<br>HEI provides job market information ©<br>Government scholarships are used by some students €<br>Analyze student performance to solve problems more than once per year &<br>Time assigned to practical training (%) 3<br>Collect student satisfaction data more than once per year @<br>Student feedback is very important to update program @<br>Teaches numerical competencies @<br>Government standards are very important to update program @<br>Number of workshops or labs available for practice<br>-40 -20 0 20 40<br>Percent<br>Infraestructure @ Training and curriculum @ Costs<br>@ Link with productive sector Faculty<br><!-- End of picture text -->
<!-- Start of picture text --> Min scores in HS GPA or national entry test is an admission requirement e<br>Research skills are very important when hiring faculty<br>* Percent of faculty working in the industry<br>HEI provides job market information e<br>Somebody is in charge of industry relations e<br>HEI trains students for job interviews @<br>Test required for graduation @<br>Government standards are very important to update program e<br>Collect student satisfaction data more than once per year @<br>Teaches numerical competencies @<br>-200 0 200 400 600<br>Percent<br>@ Training and curriculum = @ Link with productive sector Faculty @ Other Practices<br><!-- End of picture text -->
# **Data Appendix**

## **Appendix I. Description of Outcomes and Other Variables**

## **Outcomes from individual-level administrative data**

## _A. Previous labor market experience_

These variables refer to the 12-month period before graduation (Ecuador) or entry into the program (Brazil). For a given student, her peers are those who graduated from the same program as her in the same year (Ecuador) or who entered the same program as her in the same year (Brazil). Previous labor market experience outcomes include the following: _Percent of time employed before graduation/entry_ is the number of months the individual was employed during that 12-month period divided by 12.

_Peer average percent of time employed before graduation/entry_ is the average of the previous variable for the student’s peers.

_Employed at least one month before graduation/entry_ is a dummy that takes the value of one if the individual was employed for at least one month within the corresponding 12-month period.

_Percent of peers employed at least one month before graduation/entry_ is the average of the previous variable for the student’s peers.

## _B. Graduation_

_Graduation_ (only for Brazil) is a dummy that takes the value of one if the individual graduated within three years of entering the program.

## _C. Labor Market Outcomes_

These variables (for Ecuador and Brazil) refer to the 12-month period following graduation. They include the following: _Employed at least one month after graduation_ is a dummy that takes the value of one if the individual was employed for at least one month during that period.

_Percent of time employed after graduation_ is the number of months the individual was employed during that 12-month period divided by 12 56
_Average monthly wage (USD, PPP)_ is the average monthly wage received by the individual during that 12-month period. If the student worked at least one month, average wage is equal to total wages divided by the number of months he worked (i.e., it is equal to her average monthly wage conditional on working). If she did not work at all, average wage is zero. Purchasing power parity (PPP) adjustment of wages was done using the 2019 (Ecuador) or 2017 (Brazil) PPP conversion factor.

# **Other variables based on individual-level administrative data**

_Student administrative variables (PCA score)_ consist of an index created for Brazil and Ecuador through a Principal Component Approach with the goal of reducing the dimensionality of the student characteristics included in estimation. The index includes the following student characteristics: age, gender, mother’s education level (less than primary, primary, high school, higher education, unknown), whether the student has children (Ecuador), and socioeconomic status index (Ecuador).

_Peer administrative variables (PCA score)_ follows the same logic, but for peers’ (average) characteristics. The variables included in the calculation of the index were average characteristics at the student’s cohort level, which is defined as all students (excluding the student herself) who entered (for Brazil) or graduate in (for Ecuador) the same program in the same year. Thus, the calculation of the index includes peers’ average age, percentage of peers who are women, percentage of peers with each level of mother’s education, percentage of peers with children (Ecuador), and peers’ average socioeconomic index (Ecuador).

# **Outcomes from survey data**

_Dropout rate_ is the percentage of students that dropped out among those who were supposed to graduate the academic year before the survey.

_Extra time to graduate (ETG)_ is the average percentage of additional time that students take to graduate relative to the theoretical duration of the program. For example, if a program is supposed to last two years and students take, on average, three years to graduate, then ETG=50% (=1 extra year *100 / 2 years). To calculate ETG, we asked directors (i) the average time it took the last cohort to graduate from the program and (ii) the theoretical duration of the program.

57
_Formal employment_ is a binary variable that equals one when the director reports that almost all the program graduates from the previous year are employed or self-employed in the formal sector. Directors were given three possible options regarding how many of the past year’s graduates found employment or self-employment in the formal sector: almost all; some; almost none or none.

_Wages_ is the average salary of the graduates from last year, whether they work in the formal or informal sector. Purchasing power parity (PPP) adjustment of wages was done using the 2019 PPP conversion factor.

# **Other variables based on survey data**

_Student characteristics (PCA Score)_ consist of an index created to reduce the dimensionality of the average student characteristics at the program level available in the survey data. It was built using a Principal Component Analysis approach and includes the following student characteristics (see Panel B in Table 1): percent of full-time students, percent of female students, percent of students that are 25+ years old, academic deficiencies in incoming students. The latter are indicators of whether program directors reported that missing skills among incoming students in math, oral expression, reading, writing or other competencies.

_Program characteristics (PCA Score)_ follows a similar logic and includes the following program characteristics (see Panel B in Table 1): program duration (semesters), number of cities where the HEI offers the program, whether the program has high quality accreditation, total number of students in the program last year, and program age (years).

_HEI characteristics (PCA Score)_ also follows a similar logic and includes the following institution characteristics (see Panel B in Table 1): HEI age, whether the HEI is a university, whether the HEI is for profit, and number of programs in the HEI. Note that whether the HEI is public is not included in this index; it is included as a separate “fixed” control in our estimation strategy.

58
# **Appendix Tables and Figures**

## **Table A1. Institutional Framework for WBSCPS Countries**

|**Indicator**|**Brazil**|**Colombia**|**Dominican Republic**|**Ecuador**|**Peru**|
|---|---|---|---|---|---|
|**1. SCP enrollment**<br>**share (%)**|12|32|4|14|25|
|**2. Program types**|Technological (2-3 yrs.)|Technical (2 yrs.)<br>Technological (3 yrs.)|Technical (2 yrs.)<br>Prof. technical (_tecnico_<br>_superior_; 2-3 years)|Technical<br>Technological (mostly<br>2 yrs.)|Technical (2 yrs.)<br>Prof. technical (3-4 yrs.)<br>Tech. bachelor’s (3-4 yrs.)|
|**3. Number of HEIs**<br>**offering SCPs**|HEI in Brazil: 1,700<br>HEI in SP + Ceará: 467|217|28|182|HEIs w/ licensed prog.: 75<br>Other HEIs: 747|
|**4. Private enrollment**<br>**(%)**|Brazil: 84<br>SP + Ceará: 77|21|49|53|HEIs w/ licensed prog.: 97<br>Other HEIs: 50|
|**5. HEI types and**<br>**enrollment shares**<br>**(%)**|SP + Ceará:<br>Universities 41<br>Univ. Centers 17<br>Schools 38<br>IF and CEFET 3|Universities 9<br>Univ. Institutes 13<br>Technol. Inst. 7<br>Technical Inst. 6<br>SENA 65|Universities 47<br>Tech. Institutes 53|Tech. & Technol. Inst.:<br>95<br>Univ. and Polytechnic<br>Schools:<br>5|<br>HEIs w/ licensed programs:<br>Higher Education Institutes:<br>100|
|**6. Number of SCPs**|SP + Ceará: 2,388|2,130|209|543|Licensed programs:<br>392|
|**7. Public funding to**<br>**students at public**<br>**HEIs**|Zero tuition|Zero tuition at SENA<br>Public HEIs: government<br>scholarships; loans from<br>public institution(ICETEX)|Zero tuition, but “academic<br>fees.”|Zero tuition|Zero tuition|
|**8. Funding for**<br>**students at private**<br>**HEIs**|ProUni (government<br>scholarship for low-<br>income, high ability<br>students)<br>FIES (government and<br>state-guaranteed student<br>loans banks)<br>FUNDACRED (loans)|Government scholarships<br>Loans from public<br>institution (ICETEX)|Government scholarships<br>Loans from public<br>institution (FUNDAPEC)|Loans from public bank<br>(_Banco del Pacifico_)|Public loans and scholarships<br>(PRONABEC) for low-<br>income, high-ability students|
|**9. Public funding to**<br>**public HEIs**|Yes. Sources: federal,<br>state, municipality|For SENA: yes (dedicated<br>taxes)<br>For other public HEIs: yes|Yes|Yes|Yes|
|**10. Public funding to**<br>**private HEIs**|No|No|n/a|Partial for some private<br>HEIs (_cofinanciadas_)|No|

59
|**11. National**<br>**mandatory entry**<br>**exam**|ENEM (high school<br>graduation exam) required<br>by public HEIs<br>Vestibular (HE entry<br>exam) required by some<br>HEIs|Saber 11 (mandatory for<br>high school graduation)|POMA or PAA (mandatory<br>for HE entry)|Ser Bachiller<br>(mandatory for high<br>school graduation)|Only for students applying to<br>PRONABEC scholarship|
|---|---|---|---|---|---|
|**12. National**<br>**mandatory exit exam**|ENADE (mandatory for<br>HE graduation; only some<br>majors tested in a given<br>year)|Saber T&T (mandatory for<br>SCP graduation)|n/a|No|No|
|**13. Operating license**|Mandatory (initial<br>accreditation) by HEI and<br>program.|Mandatory by HEI and<br>program; must be renewed<br>periodically|Mandatory; evaluation<br>required to renew license<br>every 5 years.|Mandatory by HEI|Mandatory by HEI and<br>program; lasts 5 years.|
|**14. Accreditation**|At private HEIs,<br>accreditation must be<br>renewed, generally every 3<br>years.|High-quality accreditation<br>is voluntary for HEIs and<br>programs, mandatory for<br>teaching programs. Lasts 4<br>years.|HEIs can voluntarily pursue<br>international accreditation.|Periodic mandatory<br>accreditation is needed<br>to remain open.|Voluntary for HEIs and<br>programs; mandatory for<br>education, law and health<br>programs.|
|**15. For-profit HEIs**|Allowed (36% of SCPs in<br>Brazil; 39% in SP+ Ceará)|Not allowed|Not allowed|Not allowed|Allowed (75% of licensed<br>SCPs)|

# _Sources:_ Ferreyra et al. (2021).

_Notes:_ HEIs in Brazil’s S-System count as university centers or schools. For Colombia, _Servicio Nacional de Aprendizaje_ (SENA) is not counted as an HEI in row (3) but its students are counted as part of public enrollment in row (4). For Ecuador, private share includes enrollment in private institutions with public funding ( _cofinanciadas_ ). When an HEI has branches in multiple cities, each branch is counted separately. CEFET = Federal Centers for Technological Education ( _Centros Federais de Educação Tecnológica_ ) (Brazil); ENADE = National Exam of Student Achievement ( _Exame Nacional de Desempenho dos Estudantes_ ) (Brazil); ENEM = National Assessment of Secondary Education ( _Exame Nacional de Ensino Médio_ ) (Brazil); FIES = Student Financing Fund ( _Fundo de Finaciamiento Estudantil_ ) (Brazil); FUNDAPEC = Pro-Education and Culture Action Educational Credit Foundation ( _Fundación Acción Pro Educación y Cultura [APEC] de Crédito Educativo_ ) (Dominican Republic); HEI = higher education institution; HE= higher education; IF = Federal institute (Instituto Federal) (Brazil); ICETEX = Colombian Institute of Educational Credit and Technical Training Abroad ( _Instituto Colombiano de Crédito Educativo y Estudios Técnicos en el Exterior_ ) (Colombia); PAA = Academic Aptitude Test ( _Prueba de Aptitud Académica_ ) (Dominican Republic); POMA = ( _Prueba de Orientación y Medición Académica_ ) (Dominican Republic); PRONABEC = National Program of Scholarships and Educational Credit ( _Programa Nacional de Becas y Crédito Educativo_ ) (Peru); Prof. = professional; SCP = short-cycle program; Tech. = technical. WBSCPS = data from the World Bank Short-Cycle Program Survey.

60
# **Table A2. Sampling frame and survey structure, by country**

|**Country**|**Sampling**<br>**Procedure**<br>(1)|**SCPs**<br>**universe**<br>(2)|**Replace-**<br>**ment rate**<br>(3)|**Assumptions for**<br>**power calc.**<br>(4)|**Stratification levels**<br>(5)|**Sampling Frame**<br>**(Universe) Source**<br>(6)|
|---|---|---|---|---|---|---|
|**Brazil**|Represen-<br>tative<br>sample|2,388|50%|Effect size: 0.08<br>Power: 80%<br>Alpha: 0.05<br>Formal<br>employment at<br>baseline: 9.23%|_State_: Ceara, Sao Paulo<br>_HEI Administration_: Public, private<br>_HEI academic organization_:<br>Universidade, Centro Universitário,<br>Faculdade, Instituto Federal de<br>Educação, Ciência e Tecnologia|Higher Education Census (_Censo_<br>_da Educação Superior_), 2017.|
|**Colombia**|Represen-<br>tative<br>sample|2,130|50%|Effect size: 0.08<br>Power: 80%<br>Alpha: 0.05<br>Formal<br>employment at<br>baseline: 10%|_Region:_Caribe, Centro-Oriente,<br>Centro-Sur, Cafetero-Antioquia,<br>Pacifico<br>_HEI Administration_: Public, private<br>_HEI academic organization_:<br>University, technical center,<br>technological center.|National Information System of<br>Higher Education (_Sistema_<br>_Nacional de Información de la_<br>_Educación Superior_), 2017|
|**Dominican**<br>**Republic**|Census|209|n.a.|n.a.|All programs|Ministry of Economics, Planning,<br>and Development, 2019|
|**Ecuador**|Census|543|n.a.|n.a.|All programs|Secretariat of Higher Education,<br>Science, Technology, and<br>Innovation, 2019|
|**Peru**|Census|387|n.a.|n.a.|All licensedprograms|Ministryof Education,2019|

_Notes:_ For each country included in the WBSCPS, this table shows the sampling procedure (whether using the universe of programs or a representative sample) in column (1), size of program universes in column (2), replacement rate in column (3), and assumptions for power calculations in column (4). To determine the sample size, details on stratification levels used to calculate the number of programs to be included Colombia and Brazil samples and the source of sampling frame (universe or initial list of programs) are in columns (5) and (6). The assumptions to estimate sample size for Brazil were based on Almeida et al (2015). “n.a.” = not applicable.
**Table A3. Universes, Samples, and Response Rates, by country**

|||||**S**|**urvey status**<br>|||**Adjusted**|
|---|---|---|---|---|---|---|---|---|
|**Country**|**Universe**<br>**size**<br>**(1)**|**Sample**<br>**size**<br>**(2)**|**Effective**<br>**Surveys**<br>**(3)**|**Declined**<br>**to answer**<br>**(4)**|**Not effective**<br>**due to**<br>**COVID**<br>**(5)**|**Not effective**<br>**due to other**<br>**reasons**<br>**(6)**|**Closed**<br>**Programs**<br>**(7)**|<br>**response**<br>**rate**<br>**(8)**|
|Brazil|2,388|1,203|601|162|0|174|266|64%|
|Colombia|2,130|1,314|900|93|15|99|207|81%|
|Dominican<br>Republic|209|209|80|13|0|0|116|86%|
|Ecuador|543|543|294|66|0|124|59|61%|
|Peru|387|387|228|83|0|67|9|60%|
|Total|5,657|3,656|2,103|417|15|464|657|70%|

# _Source_ : Own calculations using WBSCPS.

_Notes_ : This table shows the universe, sample size, and effectiveness measures for the WBSCPS. For Dominican Republic and Ecuador, the universe of programs was included in the survey. For Peru, the universe of licensed programs was included. For Brazil and Colombia, representative samples were drawn from the States of Sao Paulo and Ceara and at the national level, respectively. Columns (1) and (2) show the number of programs in the universe and sample sizes for each country in the study, respectively. Columns (3) through (6) indicate the number of surveys by status for each country: column (3) indicates completed surveys; columns (4) and (5) indicate the number of surveys that SCP directors declined to answer or that did not answer because they were preparing for distance learning, respectively; and column (6) indicates surveys not answered for other reasons, such as lack of provost authorization. Column (7) shows the number of programs that were closed at the time of the survey but that were reported as open in the sampling frames. Columns (3) through (7) add up to total sample size (column 2). Adjusted response rate is the share of effective surveys (column 3) relative to all active programs in our sample (i.e., Sample size (2) – Closed programs (7)). For example, for Brazil, adjusted response rate = 601/ (1,203 – 266) = 64%.
# **Table A4. Survey representativeness checks, by country**

|Dependent variable:|Program is included in<br>the survey|Program is an<br>effective survey|Program is an effective<br>survey, excluding closed<br>programs|
|---|---|---|---|
|**_Panel A. Brazil_**|(1)|(2)|(3)|
|HEI is public|0.028**|0.042*|0.042*|
|Obs.|2,367|2,367|2,101|
|HEI is for-profit|0.039|0.032|0.042|
|Obs.|2,362|2,362|2,096|
|Total enrollment|-4.953|-2.232|-5.738|
|Obs.|2,362|2,362|2,097|
|Total graduates|-1.249|-0.506|-1.504|
|Obs.|2,359|2,359|2,094|
|Annual tuition cost|419.1*|288.3|305.0|
|Obs.|2,388|2,388|2,122|
|**_Panel B. Colombia_**||||
|Technical program|0.008|0.011|0.009|
|Obs.|2,130|2,130|2,015|
|Distance (virtual) program|0.011|-0.003|-0.001|
|Obs.|2,130|2,130|2,015|
|Annual tuition cost (in COP)|-28,285.2|80,234.1|87,502.3|
|Obs.|2,130|2,130|2,015|
|Total enrollment in 2017|66.7|-65.3|-88.9|
|Obs.|2,130|2,130|2,015|
|HEI is SENA|0.003|-0.014|-0.019|
|Obs.|2,130|2,130|2,015|
|Program has high quality||||
|accreditation|-0.006|-0.003|-0.002|
|Obs.|2,130|2,130|2,015|
|**_Panel C. Dominican Republic_**||||
|<br>HEI is a university||-0.376**|-0.038|
|Obs.||209|93|
|HEI is public||0.073|-0.039|
156 86 Obs.

|**_Panel D. Ecuador_**|||
|---|---|---|
|HEI is public|0.346***|0.372***|
|Obs.|543|483|
|Total graduates|20.2**|19.9**|
|Obs.|465|405|
|Annual tuition cost|-387.2|-107.4|
|Obs.|333|291|
|Program is taught online|0.058*|0.052|
|Obs.|543|484|
|**_Panel E. Peru_**|||
|Technical program|-0.027|-0.026|
|Obs.|387|383|
|Professional program|0.027|0.026|
|Obs.|387|383|
|Annual tuition cost (in SOL)|1061.9|1460.4|
|Obs.|387|383|
|Program has high quality<br>accreditation|-0.002|-0.007|
|Obs.|387|383|
|Face-to-face program|0.033|0.030|
|Obs.<br>_Source_: Own estimations using WBSCPS.<br>_Notes_: This table summarizes the results from represe|387<br>ntativeness checks performed|383<br>using available program-|

_Notes_ : This table summarizes the results from representativeness checks performed using available programlevel administrative data, by country. In each column, we present estimated coefficients from the following model: yp = α0 + βDp + S+ εp for Brazil and Colombia and yp = α0 + βDp + εp for the rest of countries. In this model, yp is the characteristic of program _p_ (e.g., whether the program has high quality accreditation). We estimate a model for each characteristic available in the country’s administrative data. _S_ is the sampling strata variable, and Dp indicates a dummy that varies by model depending on the comparison. Column (1) presents the comparison of programs selected for the survey relative to the rest of non-selected programs within the universe of programs for which we have administrative information (Dp = 1 if the program was selected for the survey, and 0 otherwise). This check only applies to Brazil and Colombia, the countries for which we drew samples. Column (2) shows results from testing for differences between programs with an effective survey and all other programs in the universe. Column (3) is similar to column (2) but compares effective surveys with all other programs in the universe _that have not closed_ . All regressions estimate robust standard errors clustered at the HEI level. Significance levels: * p<0.1, ** p< 0.05, *** p<0.01.

**`**
**Table A5. Validation of Outcomes Reported by Principals Using Administrative Data and Household Surveys**

||Outcome data source|Survey Data<br>(WBSCPS)|Survey Data,<br>employment<br>rate|Administrative<br>Data|HH Survey<br>(21-25 years old)|HH Survey<br>(25-29 years old)|HH Survey<br>(21-29 years old)|
|---|---|---|---|---|---|---|---|
|||(1)|(2)|(3)|(4)|(5)|(6)|
||Graduation rate (%)|62.19||30.30||||
||Dropout rate (%)|14.90||||||
|Brazil|Extra time to graduate (%)|11.99||||||
||Employment rate (%)|73.52|69.65|65.6|76.00|82.00|80.00|
||Wages (USD,PPP 2019)|10,730.00||10,006.40*|10,155.00|13,440.00|12,472.00|
||Graduation rate (%)|58.41||61.00||||
||Dropout rate (%)|14.25||23.00||||
|Colombia|Extra time to graduate (%)|24.04||25.00||||
||Employment rate (%)|57.25|65.42|63.00|75.00|78.00|77.00|
||Wages (USD,PPP 2019)|10,313.00||12,192.00|8,892.00|10,212.00|9,684.00|
||Graduation rate (%)|58.49||||||
||Dropoutrate (%)|14.95||||||
|Dominican<br>Republic|Extra time to graduate (%)<br>Employment rate (%)|30.97<br>52.50|65.42|||||
||Wages (USD,PPP 2019)|11,275.00||||||
||Graduation rate (%)|54.71||||||
||Dropoutrate (%)|12.45||||||
|Ecuador|Extra time to graduate (%)|18.52||||||
||Employment rate (%)|38.56|53.40|40.18|48.00|63.00|57.00|
||Wages (USD,PPP 2019)|11,910.00||10,900.00|<br>10,222.00|<br>13,689.00|<br>12,511.00|
||Graduation rate (%)|64.06||||||
|Peru|Dropoutrate (%)<br>Extra time to graduate (%)|12.82<br>8.95||||||
||Employment rate (%)|42.32|56.40|40.00|38.00|46.00|43.00|
||Wages (USD,PPP 2019)|7,481.00||12,167.00|<br>7,545.00|<br>8,829.00|<br>8,229.00|

65
_Source_ : Own calculations using WBSCPS, administrative data, and households’ surveys. The administrative data sources are the following: Annual Reports of the Social Security Institution ( _Relaҁão Anual de Informaҁões Sociais,_ RAIS) for Brazil for 2017, Social Security and Taxes database for 2019-2020 for Ecuador, _Observatorio Laboral para la Educacion (_ OLE) for Colombia for 2016, and _Ponte en Carrera_ for Peru for 2018. The household surveys are the original (unharmonized) surveys from these sources: Brazil: PNADC ( _Pesquisa Nacional por Amosta de Domicílios Contínua_ ) for 2018; Colombia: GEIH ( _Gran Encuesta Integrada de Hogares_ ) for 2018; Ecuador: ENEMDU ( _Encuesta Nacional Empleo, Desempleo y Subempleo_ ) for 2018; Peru: ENAHO ( _Encuesta Nacional de Hogares sobre Condiciones de Vida y Pobreza_ ) for 2017.

_Notes_ : This table presents a validation exercise for the outcomes reported in the survey data. Graduation, dropout, time to degree, and employment are expressed as percentages. * indicates that wages are expressed in annual USD PPP (2017) for Brazil when using administrative data (column 3). Column 1 presents outcome averages as reported by program directors in the WBSCPS; means are weighted by the WBSCPS sampling weights (see definitions of outcomes from survey data in Appendix 1). In column (2), we impute a formal employment rate based on the following survey questions: "Regarding the graduates of the program in recent years, how many were employed by a firm in the formal sector?" and "Regarding the graduates of the program in recent years, how many were self-employed in the formal sector?" Directors had to choose among three possible answers: almost all, some; almost none or none. We assume the following formal employment rates for those answers: 80, 40, and 10% respectively. Column (2) shows the resulting formal employment rate imputation. In column (3) we show average outcomes calculated using individual-level administrative data; wages refer to individuals working in the formal sector and the formal employment rate is the percentage of SCP graduates who work formally. Formal workers include the following: employed and self-employed in Colombia, Peru, and Ecuador; and employed in Brazil. Columns 4-6 report validation outcomes using household survey data for different age groups; wages refer both to formal and informal employees.

66
# **Table A6. Descriptive Statistics** **_Using Program-Level Data from the WBSCPS and Administrative Sources_**

|**Variable**|**Mean**|**Std.**<br>**Dev.**|**Min.**|**Median**|**Max.**|**Obs.**|
|---|---|---|---|---|---|---|
|**Panel A. Programquality determinants**|||||||
|_Infrastructure_|||||||
|Maintenance of largest lab: every year|0.57|0.49|0|1|1|<br>2,103|
|Has enough equipment or tools for practice|0.72|0.45|0|1|1|<br>2,103|
|Number of workshops or labs available for practice|6.12|6.15|0|4|65|<br>2,031|
|>30% classes can be taken online|0.15|0.36|0|0|1|<br>2,103|
|Program offers at least one online class|0.35|0.48|0|0|1|<br>2,103|
|_Training and curriculum_|||||||
|Teaches numerical competencies|0.80|0.40|0|1|1|<br>2,103|
|Teaches a foreign language|0.60|0.49|0|1|1|<br>2,103|
|Teaches persistence in complex tasks|0.76|0.42|0|1|1|<br>2,103|
|Promotes work under hardship or pressure|0.84|0.36|0|1|1|<br>2,103|
|Offers credits for longer degrees|0.89|0.31<br>|0|1|1|<br>2,103|
|Curriculum is fixed|0.70|0.46|0|1|1|<br>2,103|
|Time assigned to practical training (%)|46.45|16.56|4|50|80|<br>2,072|
|Graduation requirement|||||||
|Test|0.40|0.49|0|0|1|<br>2,103|
|Thesis or research project|0.37|0.48|0|0|1|<br>2,103|
|Second language|0.11|0.31|0|0|1|<br>2,103|
|Internships outside institution are mandatory|0.58|0.49|0|1|1|<br>2,103|
|Mandatory internships during the program+|0.16|0.37|0|0|1|<br>2,103|
|<br>Mandatory internships at the end of the program+|0.22|0.41|0|0|1|<br>2,103|
|<br>Remediation support|||||||
|<br>Remediation classes before starting the program|0.51|0.50|0|1|1|<br>2,103|
|<br>Remediation classes during the program|0.57|0.50|0|1|1|<br>2,103|
|Non-class-based remediation|0.43|0.50|0|0|1|<br>2,103|
|Years since last update to curriculum|2.91|2.69|0|2|25|<br>1,946|
|Important reasons to update program:|||||||
|Government standards+|0.70|0.46|0|1|1|<br>2,103|
|Employment outcomes or employers' requests+|0.87|0.34|0|1|1|<br>2,103|
|HEI perception of labor market+|0.83|0.38|0|1|1|<br>2,103|
|Enrollment trends+|0.56|0.50|0|1|1|<br>2,103|
|Student feedback+|0.78|0.41|0|1|1|<br>2,103|
|More than once per year:|||||||
|Analyze student performance to solve problems|0.86|0.35|0|1|1|<br>2,103|
|Collect student satisfaction data|0.69|0.46|0|1|1|<br>2,103|
|_Costs_|||||||
|Annual tuition (2019 PPP USD)|2,244|1,762|0|2,367|25,515|<br>2,103|
|<br>HEI has received funding from||||||<br>|
|<br>Private sector|0.20|0.40|0|0|1|<br>2,103|
|Government|0.34|0.47|0|0|1|<br> <br>2,103|
|Funding options used by some students|||||||
|<br>Loans by banks|0.41|0.49|0|0|1|<br>2,103|

67
|Loans by government program|0.45|0.50|0|0|1|2,103|
|---|---|---|---|---|---|---|
|Loans by HEI|0.29|0.46|0|0|1|2,103|
|Other loans|0.34|0.47|0|0|1|2,103|
|Government scholarships|0.66|0.48|0|1|1|2,103|
|HEI scholarships|0.77|0.42|0|1|1|2,103|
|_Link with productive sector_|||||||
|Engagement with firms|||||||
|Somebody in charge of industry relations|0.82|0.38|0|1|1|2,103|
|Industry helps with student evaluation or curriculum design|0.70|0.46|0|1|1|2,103|
|Industry has agreements with HEI to hire program grads|0.39|0.49<br>|0|0|1|2,103|
|Industry has agreements to train faculty|0.35|0.48|0|0|1|2,103|
|Industry lends or provides equipment to program for student training|0.50|0.50|0|1|1|2,103|
|Collect data on employment or employers' satisfaction|0.56|0.50|0|1|1|2,103|
|Communicate with local firms about their needs|0.52|0.50|0|1|1|2,103|
|Program has staff assigned to collect grads' employment data|0.68|0.47|0|1|1|2,103|
|Job search assistance|||||||
|HEI coordinates job interviews with firms|0.60|0.49|0|1|1|2,103|
|HEI trains students for job interviews|0.69|0.46|0|1|1|2,103|
|HEI provides job market information|0.81|0.40|0|1|1|2,103|
|HEI has an employment center|0.60|0.49|0|1|1|2,103|
|_Faculty_|||||||
|Number of faculty|20.00|18.70|1|15|200|2,076|
|Percent of faculty|||||||
|<br>with BA degree|82.20|29.40|0|100|100|2,043|
|with graduate degree|48.60|32.60|0|43|100|1,998|
|with SCP degree|20.00|30.00|0|6|100|1,916|
|working full-time|38.40|30.30|0|31|100|1,987|
|<br>working in the industry|42.10|30.90|0|37|100|2,008|
|with 5yrs+ industry experience|55.70|33.10|0|56|100|1,988|
|<40 years old|40.00|29.20|0|34|100|1,961|
|that are women|34.00|23.50|0|30|100|2,021|
|Faculty are evaluated more than once per year|0.85|0.360|0|1|1|2,103|
|Important for faculty evaluation|||||||
|Classroom observation+|0.60|0.49|0|1|1|2,103|
|Class planning+|0.61|0.49|0|1|1|2,103|
|Student evaluation+|0.70|0.46|0|1|1|2,103|
|Students and peers' informal comments+|0.32|0.47|0|0|1|2,103|
|<br>Peer evaluation+|0.31|0.46|0|0|1|2,103|
|Almost all or all faculty participated in professional training last year|0.55|0.50|0|1|1|2,103|
|<br>Important for faculty hiring|||||||
|<br>Practical experience|0.88|0.32|0|1|1|2,103|
|Research skills|0.62|0.48|0|1|1|2,103|
|_Other Practices_|||||||
|Update or review admin data more than once per year|0.66|0.47|0|1|1|2,103|
|Admission requirements|||||||
|<br>General or specific knowledge test|0.59|0.49|0|1|1|2,103|
|Interview|0.50|0.50|0|1|1|2,103|

68
|Min. score in HS GPA or national entry test|0.66|0.47|0|1|1|2,103|
|---|---|---|---|---|---|---|
|Percent of governing body that belongs to:|||||||
|Private sector|18.90|21.20|0|12|100|2,103|
|Government|11.90|16.40|0|1|100|2,103|
|Faculty*|39.00|27.80|0|33|100|2,103|
|Current students or other sectors|25.60|22.30|0|21.5|100|2,103|
|**Panel B. Student body, program, and institution characteristics**|||||||
|_Student body characteristics_|||||||
|Academic deficiencies|||||||
|Mathematics is lacking in incoming students|0.82|0.39|0|1|1|2,103|
|Oral expression is lacking in incoming students|0.57|0.49|0|1|1|2,103|
|Reading is lacking in incoming students|0.70|0.46|0|1|1|2,103|
|Writing is lacking in incoming student|0.68|0.47|0|1|1|2,103|
|Other competencies are lacking in incoming students|0.15|0.36|0|0|1|2,103|
|Percent of students that are|||||||
|25+ years old|28.94|29.30|0|20|100|2,103|
|full-time|43.89|39.00|0|33|100|2,103|
|Women|38.19|29.20|0|40|100|2,103|
|Student body characteristics (PCA score)|-0.02|1.40|-2.78|-0.31|4.88|2,103|
|_Program characteristics_|||||||
|Program duration (semesters)|5.20|0.97|2|6|8|2,101|
|Number of cities where the HEI offers the program|3.55|8.32|1|1|151|2,083|
|Program has high quality accreditation|0.19|0.39|0|0|1|2,103|
|Total number of students in the program last year|221.60|332.80|1|125|4,321|2,030|
|Program age (years)|11.50|9.46|0|10|70|2,103|
|Program characteristics (PCA score)|0.02|1.25|-7.59|0.20|2.43|2,082|
|_Institution characteristics_|||||||
|HEI is public|0.30|0.46|0|0|1|2,103|
|HEI is a university|0.21|0.41|0|0|1|2,103|
|HEI is for profit|0.20|0.40|0|0|1|2,103|
|HEI age|37.84|30.80|1|32|481|2,094|
|Number of programs in the HEI|21.59|36.30|1|10|268|2,103|
|<br>HEI characteristics (PCA score)|0.08|1.16|-1.84|-0.21|9.57|2,094|
|**Panel C. Noise controls**|||||||
|Survey conducted during COVID|0.48|0.50|0|0|1|2,103|
|Number of attempts to complete the survey|8.36|3.16|1|9|17|2,103|
|Survey completed by phone|0.33|0.47|0|0|1|2,103|

_Sources_ : Own calculations using WBSCPS and administrative data.

_Notes_ : This table shows the descriptive statistics of the main variables used in the analysis (Table 1 presents an abridged version). An observation corresponds to a program. Dummy variables included in the list are those with means between 0.1 and 0.9. Statistics are weighted by WBSCPS sampling weights. Panel A refers to quality determinants, presented by category. Panel B refers to characteristics of the student body, program, and higher education institution (HEI); Panel C refers to survey noise controls. Total number of surveys completed is 2,103. Values in the “Obs.” column vary depending on the number of valid responses. * indicates the omitted category in the analysis. + indicates that the variable is conditional on another. For example, “Mandatory internships during the program” is conditional on “Internships outside institution are mandatory” = 1; therefore, its coefficient is relative to not having mandatory internships outside the institution or having them but not during the program. Tuition is 69
presented in dollars but transformed in logs for estimation. “Test as graduation requirement” includes professional, industry, and national exit tests. Mean of PCA Scores are different than zero due to the use of sampling weights. All variables in panel B are included in the corresponding indexes (PCA Scores), except for “HEI is public,” which is included separately in the corresponding regressions as a “fixed” control. “HS” = high school.

70
# **Table A7. Average Annual Wages of SCP Graduates, by Country**

|**Average wage**|**Brazil**|**Colombia**|**Dominican**<br>**Republic**|**Ecuador**|**Peru**|
|---|---|---|---|---|---|
||**(1)**|**(2)**|**(3)**|**(4)**|**(5)**|
|In dollars (USD PPP 2019)|$10,730|$10,313|$11,275|$11,910|$7,481|
|In terms of the minimum<br>wage|2.2|1.3|1.3|1.4|1.3|

_Source_ : Own calculations using WBSCPS and administrative data.

_Note:_ This table reports the average annual wage per country, expressed in purchasing power parity (PPP) dollars or as a multiple of the country’s annual minimum wage. Average wage in terms of the minimum wage (MW) is calculated using each country’s MW in 2019 (Brazil, Ecuador, and Peru) and 2020 (Colombia and Dominican Republic). Average annual wage in terms of minimum wages = average annual wage / annual MW, where annual MW = monthly MW * 12. Percentiles 1 and 99 are excluded from these calculations. Statistics are weighted by WBSCPS sampling weights. Only São Paulo and Ceará are included for Brazil, and licensed programs for Peru.

71
# **Table A8. Descriptive Statistics for Programs in Brazil**

## **_Using Program-Level Data from the WBSCPS and Administrative Sources_**

|**Variable**|**Progr**<br>**matc**<br>**indivi**<br>**Mean**<br>**(1)**|**ams not**<br>**hed with**<br>**dual data**<br>**Std.Dev.**<br>**(2)**|**Program**<br>**with in**<br>**d**<br>**Mean**<br>**(3)**|**s matched**<br>**dividual**<br>**ata**<br>**Std.Dev.**<br>**(4)**|**P-value**<br>**diff.**<br>**(1) – (3)**<br>**(5)**|
|---|---|---|---|---|---|
|**Panel A. Programquality determinants**||||||
|_Infrastructure_||||||
|Program offers at least one online class|0.34|0.47|0.33|0.47|0.74|
|>30% classes can be taken online|0.13|0.34|0.10|0.30|0.19|
|Number of workshops or labs available for practice|4.57|4.09|5.72|5.61|**0.01**|
|Has enough equipment or tools for practice|0.80|0.40|0.88|0.33|**0.01**|
|Maintenance of largest lab: every year|0.67|0.47|0.70|0.46|0.56|
|_Training and curriculum_||||||
|Curriculum is fixed|0.75|0.43|0.80|0.40|0.23|
|Teaches numerical competencies|0.83|0.38|0.80|0.40|0.47|
|Teaches a foreign language|0.30|0.46|0.34|0.47|0.32|
|Teaches persistence in complex tasks|0.64|0.48|0.70|0.46|0.13|
|Remediation support||||||
|Remediation classes before starting the program|0.46|0.50|0.46|0.50|0.84|
|Remediation classes during the program|0.87|0.34|0.89|0.31|0.47|
|Non-class-based remediation|0.44|0.50|0.45|0.50|0.83|
|Graduation requirements||||||
|Professional association test|0.17|0.37|0.17|0.37|0.99|
|Thesis or research project|0.13|0.34|0.14|0.35|0.81|
|Years since last update to curriculum|1.63|1.66|1.97|2.01|**0.04**|
|Important reasons to update program:||||||
|Government standards|0.48|0.50|0.53|0.50|0.22|
|Employment outcomes or employers' requests|0.85|0.35|0.88|0.32|0.29|
|HEI perception of labor market|0.86|0.35|0.88|0.32|0.42|

72
|Enrollment trends|0.50|0.50|0.47|0.50|0.48|
|---|---|---|---|---|---|
|Student feedback|0.85|0.36|0.83|0.38|0.53|
|More than once per year:||||||
|Analyze student performance to solve problems|0.85|0.36|0.89|0.31|0.13|
|Collect student satisfaction data|0.71|0.45|0.67|0.47|0.30|
|Time assigned to practical training (%)|44.79|16.38|46.13|15.86|0.34|
|Internships outside institution are mandatory|0.25|0.44|0.30|0.46|0.20|
|Mandatory internships at the end of the program|0.10|0.30|0.11|0.31|0.77|
|_Costs_||||||
|Funding options used by some students||||||
|HEI scholarships|0.78|0.42|0.69|0.46|**0.02**|
|Government scholarships|0.80|0.40|0.74|0.44|**0.09**|
|Loans by HEI|0.26|0.44|0.27|0.45|0.70|
|Loans by banks|0.18|0.38|0.19|0.39|0.75|
|Other loans|0.24|0.43|0.31|0.46|**0.08**|
|HEI has received funding from||||||
|Government|0.30|0.46|0.31|0.46|0.91|
|Private sector|0.14|0.35|0.19|0.39|0.21|
|Annual tuition (2019 PPP USD)|2,735.20|1,748.13|2,728.69|2,061.10|0.97|
|_Link with productive sector_||||||
|Engagement with firms||||||
|Collect employment data for graduates more than once per year|0.51|0.50|0.50|0.50|0.76|
|Communicate with local firms about their needs more than once per year|0.55|0.50|0.59|0.49|0.30|
|Industry helps with student evaluation or curriculum design|0.45|0.50|0.49|0.50|0.27|
|Industry has internship agreements with HEI|0.82|0.38|0.83|0.38|0.80|
|Industry has agreements with HEI to hire program grads|0.36|0.48|0.37|0.48|0.79|
|Industry has agreements to train faculty|0.27|0.44|0.26|0.44|0.84|
|Industry lends or provides equipment to program for student training|0.40|0.49|0.41|0.49|0.81|
|Somebody in charge of industry relations|0.83|0.38|0.80|0.40|0.45|
|Program has staff assigned to collect grads' employment data|0.42|0.49|0.39|0.49|0.45|

73
|Job search assistance||||||
|---|---|---|---|---|---|
|HEI coordinates job interviews with firms|0.56|0.50|0.65|0.48|**0.04**|
|HEI trains students for job interviews|0.68|0.47|0.63|0.48|0.29|
|HEI provides job market information|0.75|0.43|0.81|0.39|**0.07**|
|HEI has an employment center|0.35|0.48|0.34|0.47|0.80|
|_Faculty_||||||
|Number of faculty|15.45|9.17|16.08|8.61|0.41|
|Percent of faculty||||||
|<40 years old faculty|38.44|27.25|34.86|26.25|0.14|
|that are women|40.95|25.21|38.83|23.31|0.32|
|with BA degree|85.89|25.91|84.41|26.62|0.52|
|with graduate degree|70.12|28.25|65.40|29.85|**0.07**|
|with 5yrs+ industry experience|63.32|27.65|62.17|28.59|0.65|
|working in the industry|46.39|29.79|41.84|27.82|**0.07**|
|Important for faculty hiring||||||
|Practical experience|0.87|0.34|0.88|0.33|0.75|
|Research skills|0.37|0.48|0.44|0.50|0.13|
|Important for faculty evaluation||||||
|Classroom observation|0.53|0.50|0.49|0.50|0.26|
|Class planning|0.49|0.50|0.44|0.50|0.26|
|Student evaluation|0.79|0.41|0.70|0.46|**0.03**|
|Students and peers' informal comments|0.24|0.43|0.24|0.43|0.86|
|Peer evaluation|0.26|0.44|0.24|0.43|0.52|
|Almost all or all faculty participated in professional training last year|0.65|0.48|0.67|0.47|0.68|
|Faculty are evaluated more than once per year|0.77|0.42|0.77|0.42|0.97|
|_Other Practices_||||||
|Update or review admin data more than once per year|0.62|0.49|0.55|0.50|0.13|
|Percent of governing body that belongs to:||||||
|Private sector|17.47|19.70|14.12|16.96|**0.03**|
|Government|5.47|11.29|7.78|13.92|**0.04**|

74
|Current students|15.34|14.51|16.42|15.64|0.42|
|---|---|---|---|---|---|
|Admission requirements||||||
|General or specific knowledge test|0.83|0.37|0.90|0.30|**0.02**|
|Interview|0.23|0.43|0.18|0.39|0.12|
|Min. scores in HS GPA or national entry test|0.87|0.33|0.88|0.33|0.84|
|**Panel B. Student body, program, field of study, and institution characteristics**||||||
|_Student body characteristics_||||||
|Percent of students that are full-time|18.74|27.64|14.11|22.90|**0.03**|
|Academic deficiencies||||||
|Mathematics is lacking in incoming students|0.79|0.41|0.83|0.37|0.15|
|Reading is lacking in incoming students|0.64|0.48|0.61|0.49|0.47|
|Writing is lacking in incoming students|0.67|0.47|0.69|0.46|0.57|
|Oral expression is lacking in incoming students|0.50|0.50|0.47|0.50|0.58|
|_Program characteristics_||||||
|Program duration (semesters)|4.81|0.96|4.81|0.91|0.99|
|Program age (years)|8.71|5.19|10.55|5.41|**0.00**|
|Total number of students in the program last year|151.55|275.94|144.11|163.48|0.69|
|Program has high quality accreditation|0.23|0.42|0.21|0.41|0.65|
|_Institution characteristics_||||||
|HEI is for profit|0.48|0.50|0.45|0.50|0.45|
|HEI is a university|0.29|0.45|0.33|0.47|0.30|
|Number of programs in the HEI|36.60|75.98|9.86|7.87|**0.00**|
|HEI is public|0.16|0.37|0.19|0.39|0.44|
|_Field of study_||||||
|Education, Arts, Humanities, and Social|0.14|0.34|0.07|0.25|**0.01**|
|Economics, Management and Accounting|0.55|0.50|0.64|0.48|**0.04**|
|Health, Agronomy, Vet, Engineering and Architecture|0.31|0.46|0.29|0.45|0.60|
|**Panel C. Noise controls**||||||
|Number of attempts to complete the survey|9.17|2.91|9.33|2.66|0.51|
|Survey completed by phone|0.45|0.50|0.42|0.49|0.47|

75
**<u>Observations</u>** <u>200 401</u>

# _Source_ : Own calculations using WBSCPS and administrative data.

_Notes_ : This table shows descriptive statistics for the 601 surveyed programs in Brazil. The unit of observation is a program. Columns (1) and (2) refer to the 200 programs that do not match to students in the individual-level dataset; columns (3) and (4) refer to the 401 programs that match (with 2+ students per program); and column (5) presents the p-value for the t-test of the difference in means between columns (1) and (3). P-value is in boldface when the difference is significant at the 1, 5 or 10% level. Panel A refers to program quality determinants, classified by category. Panel B refers to student body, program, and HEI characteristics, and Panel C to noise controls. In the case of dummy variables, only those with a mean between 0.10 and 0.9 are included in this table and in the estimation. Statistics are weighted by the WBSCPS sampling weights.

76
# **Table A9. Descriptive Statistics for Programs in Ecuador** **_Using Program-Level Data from the WBSCPS and Administrative Sources_**

|**Variable**|**Progr**<br>**match**<br>**individ**<br>**Mean**<br>**(1)**|**ams not**<br>**ed with**<br>**ual data**<br>**Std.Dev.**<br>**(2)**|**Program**<br>**with in**<br>**d**<br>**Mean**<br>**(3)**|**s matched**<br>**dividual**<br>**ata**<br>**Std.Dev.**<br>**(4)**|**P-value**<br>**diff.**<br>**(1) – (3)**<br>**(5)**|
|---|---|---|---|---|---|
|**Panel A. Programquality determinants**||||||
|_Infrastructure_||||||
|Program offers at least one online class|0.37|0.48|0.35|0.48|0.78|
|>30% classes can be taken online|0.25|0.44|0.32|0.47|0.27|
|Number of workshops or labs available for practice|3.14|2.64|3.99|3.63|**0.03**|
|Has enough equipment or tools for practice|0.56|0.50|0.64|0.48|0.22|
|Maintenance of largest lab: every year|0.52|0.50|0.58|0.50|0.41|
|_Training and curriculum_||||||
|More than once per year||||||
|Analyze student performance to solve problems|0.86|0.35|0.85|0.36|0.88|
|Collect student satisfaction data|0.67|0.47|0.60|0.49|0.27|
|Important reasons to update program||||||
|Government standards|0.61|0.49|0.73|0.45|**0.05**|
|Employment outcomes or employers' requests|0.76|0.43|0.75|0.44|0.85|
|HEI perception of labor market|0.69|0.46|0.83|0.37|**0.01**|
|Enrollment trends|0.55|0.50|0.63|0.49|0.21|
|Student feedback|0.68|0.47|0.65|0.48|0.61|
|Teaches numerical competencies|0.80|0.40|0.80|0.40|0.88|
|Teaches a foreign language|0.62|0.49|0.68|0.47|0.28|
|Graduation requirements||||||
|Professional association test|0.52|0.50|0.71|0.45|**0.00**|
|Thesis or research project|0.95|0.22|0.91|0.28|0.26|
|Second language|0.13|0.33|0.11|0.32|0.76|

77
|Internships outside institution are mandatory|0.94|0.24|0.94|0.24|0.90|
|---|---|---|---|---|---|
|Mandatory internships during the program|0.54|0.50|0.36|0.48|**0.01**|
|Time assigned to practical training (%)|52.37|13.78|51.06|13.14|0.47|
|Remediation support||||||
|Remediation classes before starting the program|0.54|0.50|0.65|0.48|0.11|
|Remediation classes during the program|0.30|0.46|0.41|0.49|**0.08**|
|Non-class-based remediation|0.30|0.46|0.37|0.49|0.21|
|Years since last update to curriculum|2.26|2.09|2.08|2.16|0.53|
|_Costs_||||||
|Annual tuition (2019 PPP USD)|1,049.59|1,389.20|1,546.45|1,701.53|**0.01**|
|Funding options used by some students||||||
|Loans by banks|0.25|0.43|0.30|0.46|0.35|
|Loans by government program|0.36|0.48|0.38|0.49|0.80|
|Other loans|0.22|0.42|0.37|0.49|**0.01**|
|Government scholarships|0.45|0.50|0.38|0.49|0.26|
|HEI scholarships|0.64|0.48|0.75|0.44|**0.07**|
|_Link with productive sector_||||||
|Engagement with firms||||||
|Collect employment data for graduates more than once per year|0.44|0.50|0.52|0.50|0.22|
|Communicate with local firms about their needs more than once per year|0.38|0.49|0.39|0.49|0.79|
|Program has staff assigned to collect grads' employment data|0.71|0.46|0.76|0.43|0.33|
|Industry helps with student evaluation or curriculum design|0.68|0.47|0.61|0.49|0.24|
|Industry has agreements with HEI to hire program grads|0.30|0.46|0.45|0.50|**0.02**|
|Industry has agreements to train faculty|0.32|0.47|0.40|0.49|0.22|
|Industry lends or provides equipment to program for student training|0.54|0.50|0.50|0.50|0.53|
|Somebody in charge of industry relations|0.85|0.36|0.89|0.31|0.29|
|Job search assistance||||||
|HEI coordinates job interviews with firms|0.42|0.50|0.46|0.50|0.57|
|HEI trains students for job interviews|0.48|0.50|0.58|0.50|0.11|

78
|HEI provides job market information|0.78|0.42|0.81|0.39|0.54|
|---|---|---|---|---|---|
|HEI has an employment center|0.72|0.45|0.74|0.44|0.66|
|_Faculty_||||||
|Important for faculty evaluation||||||
|Classroom observation|0.78|0.42|0.76|0.43|0.79|
|Class planning|0.76|0.43|0.75|0.43|0.92|
|Students and peers' informal comments|0.41|0.49|0.40|0.49|0.89|
|Peer evaluation|0.60|0.49|0.53|0.50|0.27|
|Almost all or all faculty participated in professional training last year|0.71|0.46|0.69|0.47|0.77|
|Research skills are very important when hiring faculty|0.58|0.50|0.69|0.46|**0.08**|
|Number of faculty|12.18|8.55|18.69|17.91|**0.00**|
|Percent of faculty||||||
|with BA degree|84.12|26.93|86.20|23.55|0.54|
|with 5yrs+ industry experience|61.88|33.41|58.43|38.12|0.47|
|working full-time|55.01|35.17|57.45|32.05|0.59|
|with graduate degree|34.68|23.98|38.56|24.70|0.23|
|working in the industry|43.58|37.50|38.46|35.76|0.30|
|with SCP degree|18.00|28.94|14.02|24.20|0.28|
|<40 years old|55.39|33.48|51.42|29.92|0.35|
|that are women|38.64|23.30|38.56|21.85|0.98|
|_Other Practices_||||||
|Update or review admin data more than once per year|0.70|0.46|0.75|0.43|0.39|
|Percent of governing body that belongs to:||||||
|Private sector|12.85|22.49|6.76|16.16|**0.02**|
|Current students|13.57|13.38|11.93|9.38|0.29|
|Other sector|9.57|18.82|20.21|26.19|**0.00**|
|Admission requirements||||||
|General or specific knowledge test|0.65|0.48|0.65|0.48|0.91|
|Interview|0.44|0.50|0.35|0.48|0.15|

79
|Min. score in HS GPA or national entry test is an admission requirement|0.50|0.50|0.46|0.50|0.60|
|---|---|---|---|---|---|
|**Panel B. Student,program, field of study, and institution characteristics**||||||
|_Student body characteristics_||||||
|Percent of students that are full-time|61.01|40.41|65.47|39.01|0.39|
|Academic deficiencies||||||
|Mathematics is lacking in incoming students|0.67|0.47|0.81|0.40|**0.02**|
|Reading is lacking in incoming students|0.50|0.50|0.51|0.50|0.96|
|Writing is lacking in incoming students|0.68|0.47|0.54|0.50|**0.02**|
|Oral expression is lacking in incoming students|0.71|0.45|0.58|0.50|**0.03**|
|_Program characteristics_||||||
|Program duration (semesters)|5.05|0.60|5.06|0.73|0.85|
|Program age (years)|10.54|9.28|12.80|9.27|**0.06**|
|Total number of students in the program last year|162.88|291.68|393.95|613.07|**0.00**|
|_Institution characteristics_||||||
|HEI is public|0.40|0.49|0.25|0.44|**0.01**|
|Program has high quality accreditation|0.26|0.44|0.45|0.50|**0.00**|
|Number of programs in the HEI|4.96|3.31|6.18|4.56|**0.02**|
|_Field of study_||||||
|Health|0.04|0.19|0.08|0.27|0.17|
|Education, Arts, Humanities, and Social|0.49|0.50|0.27|0.44|**0.00**|
|Economics, Management and Accounting|0.22|0.42|0.27|0.45|0.41|
|Agronomy and Vet, Engineering and Architecture|0.24|0.43|0.38|0.49|**0.02**|
|**Panel C. Noise controls**||||||
|Number of attempts to complete the survey|9.85|3.46|9.39|3.95|0.34|
|Survey completed by phone|0.22|0.42|0.18|0.39|0.42|
|**Observations**|1|53|9|2||

# _Source_ : Own calculations using WBSCPS and administrative data.

_Notes_ : This table shows descriptive statistics for the 245 surveyed programs in Ecuador. The unit of observation is a program. Columns (1) and (2) refer to the 153 programs that do not match to students in the individual-level dataset and have at least two students; columns (3) and (4) refer to the 92 programs that match (with 2+ students per program); and column (5) presents the p-value for the t-test of the difference in means between columns (1) and (3). P-value is in boldface 80
when the difference is significant at the 1, 5 or 10% level. Some programs match but have only one student; they are not included in this table or in the estimation using individual-level data. Panel A refers to program quality determinants, classified by category. Panel B refers to student body, program, and HEI characteristics, and Panel C to noise controls. For dummy variables, this list and the corresponding estimations only include those with a mean between 0.1 and 0.9 Statistics are weighted by WBSCPS sampling weights.

81
# **Table A10. Descriptive Statistics for Students in Brazil** **_Based on Individual-Level Data for Students Matched to Surveyed Programs_**

|**Variable**|**Mean**|**Std. Dev.**|
|---|---|---|
|**Panel A. Student characteristics**|||
|Age|24.60|5.57|
|Female|0.47|0.50|
|Mother's education:|||
|Less than primary|0.11|0.31|
|Primary school|0.14|0.35|
|High school|0.19|0.39|
|Higher education|0.07|0.26|
|Unknown|0.49|0.50|
|Student administrative variables (PCA Score)|0.00|1.15|
|**Panel B. Peer(average) characteristics**|||
|Age|24.65|1.77|
|Percentage of female peers|0.47|0.28|
|Percentage of peers by mother's education:|||
|Less than primary|0.11|0.05|
|Primary school|0.14|0.06|
|High school|0.19|0.07|
|Higher education|0.07|0.06|
|Unknown|0.48|0.14|
|Peers'administrative variables (PCAScore)|-0.07|1.45|
|**Panel C. Outcomes**|||
|Graduated|0.30|0.46|

_Sources_ : Own calculations using individual-level administrative data for Brazil. For more details on data sources and variable definitions, see Section 3.3 and Appendix 1.

_Notes_ : In this table, the unit of observation is a student. Statistics are weighted using WBSCPS sampling weights. For a given student, her peers are the other students who started her program in 2014. Means of PCA scores are different than zero due to the use of sampling weights. Number of observations is 29,453.

82
# **Table A11. Associations between SCP Quality Determinants and Academic Outcomes** **_Using Program-Level Data for all Survey Countries_**

|_Dependent variable:_|_Dropout_<br>_Rates_|_Extra time-to-_<br>_graduate_|
|---|---|---|
||(1)|<br>(2)|
|_Costs_|||
|HEI scholarships are used by some students|1.438<br>(0.876)||
|Loans b banks are used b some students|-1.284|2.595*|
|y y|(0.834)<br>|(1.555)|
|HEI has received funding from government|1.309*<br>(0.739)<br>||
|HEI has received funding from private sector|1.466*<br>(0.815)||
|||-0987**|
|Annual tuition (2019 PPP USD, log)||.<br>(0.427)|
|_Training and curriculum_|||
||-2.111**||
|Curriculum is fixed|(0.899)||
|Offers remediation classes during the program|-1.003<br>(0.782)||
|Years since last update to curriculum|-0.0439<br>(0.111)<br>||
|Enrollment trends are very important to update program|-1.122<br>(0.703)||
|||-7.358***|
|Promotes work under hardship or pressure||(2.236)|
|Test required for graduation||1.988<br>(1.623)|
|Thesis or research project required for graduation||3.153*<br>(1.778)|
|HEI perception of labor market is very important to update||<br>5.069***|
|program<br>||(1.646)<br>|
|Analyze student performance to solve problems more than||-2.547|
|once per year||(2.358)<br>|
|Collect student satisfaction data more than once per year||-1.135<br>(1.582)|
|_Link with productive sector_|||
||-0.667|-3.652**|
|Industry helps with student evaluation or curriculum design|(0.790)|(1.752)|
|Industry lends or provides equipment to program for student|<br>-1.769***||
|<br>training|(0.670)<br>||
|Somebody in charge of industry relations|-1.507<br>(1.198)||
|HEI i d f b ii|<br>-0.817||
|trans stuents or jo ntervews|(0.876)||
||-2.198**||
|HEI provides job market information|<br>(1.036)||
|||1289|
|HEI has an employment center||.<br>(1.485)|
|_Faculty_|||
|Number of faculty (log)|||
|||-00524**|
|Percent of faculty working in the industry||.<br>(0.0205)|
|Percent of facult with SCP deree|-0.0132|<br>-0.0546**|
|y g|(0.0126)|(0.0266)|
||<br>0.00629||
|Percent of faculty with graduate degree|(0.0127)||
||0.0217*||
|Percent of faculty working in the industry|<br>(0.0118)||
|Research skills are very important when hiring faculty|-1.096<br>(0.801)||

83
|Students’ and peers' informal comments are very important in|-0.320||
|---|---|---|
|faculty evaluation|(0.705)<br>||
|Peer evaluation is very important in the faculty evaluation|-1.463*<br>(0.788)||
|Practical experience is very important when hiring faculty||-3.417<br>(2.234)|
|Almost all or all faculty participated in professional training last year||2.663*<br>(1.467)<br>|
|Faculty are evaluated more than once per year<br>||-4.393*<br>(2.305)|
|_Other Practices_|||
|Update or review admin data more than once per year|-0.274<br>(0.725)||
|Student body, program, and HEI characteristics (PCA scores)|✓|✓|
|Noise controls, country-and field-fixed effects|✓|✓|
|Observations|1,525|1,692|
|Mean of dependent variable|14.03|18.58|
|R-squared|0.085|0.107|
|Adj. R squared|0.0613|0.0890|

# _Source_ : Own estimations using WBSCPS data.

_Notes_ : This table shows coefficients from the (second stage) OLS regressions of dropout rate and extra time to graduation (ETG) on the determinants selected by LASSO in the first stage. The unit of observation is a program. Regressions are weighted using WBSCPS sampling weights. See definitions of outcomes and PCA scores in Appendix 1. Specifications control for PCA scores of characteristics of the student body, program, and HEI as well as for survey noise controls, country fixed effects, and field fixed effects. Standard errors clustered at HEI level are in parenthesis. Significance levels: * p<0.1, ** p< 0.05, *** p<0.01.

84
**Table A12. Associations between SCP Quality Determinants and Labor Market Outcomes** **_Using Program-Level Data for all Survey Countries_**

|_Ddt ribl_|_Formal employment_|_Log Wages_|
|---|---|---|
|_epenen vaae:_|(1)|(2)|
|_Infrastructure_|||
|Has enough equipment or tools for practice|0.0702**|0.0200|
||(0.0352)|(0.0167)|
|_Training and curriculum_|||
|Teaches numerical competencies|0.133***|0.0561***|
||(0.0394)|(0.0189)|
|Offers remediation classes during the program|0.0525*|0.0275*|
||(0.0317)|(0.0146)|
|Offer non-class-based remediation||-0.0257*<br>|
|||(0.0152)|
|Employment outcomes or employers' requests are very|||
|important to update program|0.0678||
||<br>(0.0425)||
|HEI perception of labor market is very important to update|<br>||
|program|0.0400|0.0395**|
||(0.0356)|(0.0179)|
|Enrollment trends are very important to update program||-0.0169<br>(0.0130)|
|Offers credits for longer degrees||<br>0.0615***<br>(0.0207)|
|_Link with productive sector_|||
|Collect data on employment or employers' satisfaction more|||
|than once per year|0.0483*||
||(0.0292)||
|Industry has agreements to train faculty|-0.0488||
||(0.0320)||
|Industry lends or provides equipment to program for student|||
|<br>training|-0.0266||
||(0.0286)||
|Program has staff assigned to collect grads' employment data|0.0576*||
||(0.0313)||
|HEI provides job market information||0.0126|
|||(0.0188)|
|HEI has an employment center|0.0830***|<br>0.0252|
||(0.0311)|(0.0161)|
|_Faculty_|||
|Percent of <40 years old faculty|-0.00175***||
||(0.000501)||
|Percent of faculty with 5yrs+ industry experience|0.00101**||
||(0.000441)||
|Percent of faculty that are women||-0.000683**|
|||(0.000324)|
|Percent of faculty with BA degree||0.000469*|
|||(0.000250)|
|Class planning is very important in faculty evaluation|0.0470<br>(0.0293)||
|Classroom observation is very important in faculty evaluation||0.0157|
|||(0.0137)|
|Number of faculty (log)||<br>0.0229**|
|||(00109)|
|Research skills are very important when hiring faculty||.<br>0.0207<br>(00160)|
|_Other Practices_||.|
|<br>Update or review admin data more than once per year|0.0468||
||(0.0302)||
|Percent of governing body that belongs to current students|<br>-0.000798|-0.000614*|
||(0.000678)|(0.000318)|
|Percent of governing body that belongs to private sector||0000367|
|||.<br>(0.000349)|

85
|Interview is an admission requirement<br>General or specific knowledge test is an admission requirement||0.0235<br>(0.0168)<br>0.0281*<br>(0.0150)|
|---|---|---|
|Student body, program, and HEI characteristics (PCA scores)|✓|✓|
|Noise controls, country- and field-fixed effects|✓|✓|
|Observations|1,270|1,751|
|Mean of dependent variable|0.591|9.209|
|Average wage (USD 2019 PPP)||10,424.17|
|R-squared|0.136|0.175|
|Adj R squared|0.113|0.158|

# _Source_ : Own estimations using WBSCPS data.

_Notes_ : This table shows coefficients from the (second stage) OLS regressions of formal employment and log wages on the determinants selected by LASSO in the first stage. The unit of observation is a program. Formal employment equals one if the director reports that almost all of the program graduates are employed or self-employed in the formal sector. Regressions are weighted using sampling weights from the WBSCPS. See definitions of outcomes and PCA scores in Appendix 1. Number of observations vary across variables due to differences in share of missing values. Specifications control for PCA scores of characteristics of the student body, program, and HEI as well as for survey noise controls, country fixed effects, and field fixed effects. See Appendix 1 for the list of the variables included in the PCA indexes. Standard errors clustered at HEI level are in parenthesis. Significance levels: * p<0.1, ** p< 0.05, *** p<0.01.

86
# **Table A13. Associations between SCP Quality Determinants and Labor Market Outcomes**

**_Using Individual- and Program-Level Data for Brazil_**

||**Graduation**|**Ever**<br>**employed the**<br>**year after**<br>**graduation**|**Percent of time**<br>**employed after**<br>**graduation**|**Log**<br>**average**<br>**monthly**<br>**wage**|
|---|---|---|---|---|
||(1)|<br>(2)|(3)|<br>(4)|
|_Infrastructure_|||||
|Program offers at least one online class|0.0467**<br>(0.0193)||||
|>30% classes can be taken online|||2.560*<br>||
||||(1.497)||
|Number of workshops or labs available for practice|0.00149<br>(0.00236)||||
||<br>-0.0211||-1.506||
|Has enough equipment or tools for practice|(0.0240)||(1.893)||
||0.0345*||0.0472||
|Maintenance of largest lab: every year|(0.0202)||(1.369)||
|_Costs_|||||
|HEI scholarships are used by some students|0.00990<br>(0.0197)||||
||0.0173||||
|Loans by HEI are used by some students|(0.0207)||||
||<br>0.0500**|0.0285*|4.994***|0.202*|
|Loans by banks are used by some students|(0.0223)|(0.0148)|(1.569)|(0.104)|
|Other loans are used by some students|0.0136<br>(0.0190)||||
|HEI has received fundin from overnment|<br>-0.0290||-2.682*||
|g g|(0.0211)||(1.438)||
|HEI has received funding from private sector|0.0342<br>(0.0221)||||
|_Training and curriculum_|||||
||||-1.558||
|Curriculum is fixed|||(1.459)||
||-0.0285|0.0382**|<br>4.177**|0.292**|
|Teaches numerical competencies|(0.0255)|(0.0190)|(1.822)|(0.133)|
||0.0177||||
|Teaches a foreign language|(0.0210)||||
||<br>0.00888|0.0290**|1.654|0.158|
|Teaches persistence in complex tasks|(0.0199)|<br>(0.0141)|(1.400)|(0.0996)|
|Offers remediation classes before starting the program|0.0171<br>|0.0228*<br>||0.132<br>|
||(0.0165)|(0.0118)||(0.0820)|
|Offers remediation classes durin the roram||-0.0512**|-0.614|-0.375***|
|g pg||(0.0208)|(2.026)|(0.140)|
|Off lbd dii|-0.0232||||
|er non-cass-ase remeaton|(0.0180)||||
|fil ii i f i|<br>0.0549*|0.0348**|7.062***|0.201*|
|Proessona assocaton test requred or graduaton|(0.0280)|(0.0172)|(1.763)|(0.114)|
|Thesis or research project required for graduation|0.0259<br>(0.0235)|0.0166<br>(0.0200)|1.941<br>(1.773)|0.104<br>(0.133)|
|Years since last update to curriculum|0.00675*<br>(0.00357)||||
|Government standards are very important to update program|-0.0359**<br>(0.0167)||||
|Employment outcomes or employers' requests are very important to|<br>-0.0152||-2.561||
|<br>update program|(0.0295)||(2.007)<br>||
|Ellt td itt t dt|||-1.152||
|nromen rens are very mporan o upae program|||(1153)||
||-0.0366*|0.0320**|.|0.241**|
|Student feedback is very important to update program|(0.0215)|(0.0144)||(0.101)|
|Analyze student performance to solve problems more than once per|<br>0.0919***|<br>0.0476**|5.854**|<br>0.267*|
|year|(0.0298)<br>|(0.0211)|(2.313)<br>|(0.146)|
|Collect student satisfaction data more than once per year|-0.0398**<br>(0.0199)||2.386*<br>(1.261)||

87
|Time assigned to practical training (%)|-0.000699<br>(0.000557)||||
|---|---|---|---|---|
|Internships outside institution are mandatory|0.0169<br>(0.0205)||||
|Mandatory internships at the end of the program|-0.0448<br>(0.0294)||||
|_Link with productive sector_|||||
|Collect employment data for graduates more than once per year|-0.0186<br>|-0.0260*<br>|-2.686**<br>|-0.198**<br>|
||(0.0187)|(0.0132)|(1.350)|(0.0910)|
|Communicate with local firms about their needs more than once per|-0.0371*<br>||||
|year|(0.0204)<br>||***||
|Industry helps with student evaluation or curriculum design|-0.0137<br>||-4.414<br>||
||(0.0169)<br>||(1.214)<br>||
|Industr has internshi areements with HEI|-0.0440**||-1.799||
|y p g|(0.0218)<br>||(1.492)||
|Industr has areements to train facult|0.0363*||||
|y g y|(0.0198)||||
|Industry lends or provides equipment to program for student|-0.0174||||
|training|(0.0188)<br>||||
|Sbd (f bd tff) i i h f idt lti|-0.0271||1.680||
|omeoy rom oar or sa s n carge o nusry reaons|(0.0202)<br>||(1.506)||
|Program has staff assigned to collect grads' employment data|0.0320<br>(0.0201)||||
|HEI i d f b ii||-0.0392***|-2.599*|-0.258***|
|trans stuents or jo ntervews||(0.0136)|(1.500)|(0.0958)|
|HEI coordinates job interviews with firms|0.0121<br>(0.0179)||||
||-0.0199|0.0574***|3.871**|0.376***|
|HEI provides job market information|(0.0228)<br>|(0.0155)|(1.656)|(0.109)|
|HEI has an employment center|-0.0212<br>(0.0192)||||
|_Faculty_|||||
|||0.0419***|3.131**|0.302***|
|Faculty are evaluated more than once per year||(0.0153)|(1.587)|(0.106)|
|Percent of <40 years old faculty|0.000567<br>(0.000351)|||-|
|||-0.000444|-0.0276|0.00425**|
|Percent of faculty that are women||(0.000287)|(0.0271)<br>|(0.00201)|
|Percent of faculty with BA degree|0.000433<br>(0.000344)||-0.0636**<br>(0.0253)||
|Percent of faculty with graduate degree||0.000633***<br>|0.0929***<br>|0.00531***<br>|
|||(0.000199)|(0.0213)|(0.00140)|
|Percent of faculty with 5yrs+ industry experience|-0.000571<br>||||
||(0.000399)||||
|Percent of faculty working in the industry|0.000721**<br>||-0.0287<br>||
||(0.000348)<br>||(0.0227)||
|Practical experience is very important when hiring faculty|0.0207<br>||||
||(0.0265)||||
|Classroom observation is very important in faculty evaluation|<br>0.00654<br>|0.0228*<br>||0.183**<br>|
||(0.0189)<br>|(0.0116)<br>**||(0.0808)<br>**|
|Class lannin is ver imortant in facult evaluation|0.0171|-0.0329|-1.345|-0.218|
|pg y p y|(0.0185)|(0.0129)|(1.284)|(0.0895)|
|Students and peers' informal comments are very important in faculty|<br>-0.0134|<br>-0.0290*||<br>-0.191*|
|<br>evaluation|(0.0189)<br>|(0.0148)<br>||(0.104)<br>|
|P lti i itt i th flt lti|0.0199|0.0308**||0.210**|
|eer evauaon s very mporan n e acuy evauaon|(00184)|(00139)||(00980)|
||.<br>-0.0459**|.|-1.955|.|
|Almost all or all faculty participated in professional training last year|(00195)||(1228)||
|_Other practices_|.||.||
|Udt i di dt th|0.0130|0.0473***|4.222***|0.345***|
|pae or revew amn aa more an once per year|(0.0169)|(0.0131)<br>|(1.194)<br>|(0.0900)<br>|
|Percent of governing body that belongs to: private sector||0.000412<br>(0000368)|0.107***<br>(00394)|0.00334<br>(000261)|
|Percent of governing body that belongs to: government or other sector||.|.<br>0.0744*|.|

88
|Percent of governing body that belongs to: current students|0.00206***<br>(0.000635)||(0.0396)||
|---|---|---|---|---|
|General or specific knowledge test is an admission requirement|0.0339<br>(0.0260)|0.0249<br>(0.0253)|4.516*<br>(2.521)|0.163<br>(0.178)|
|Interview is an admission requirement||0.0478***<br>(0.0158)|4.977***<br>(1.477)|0.337***<br>(0.110)|
|Min. scores in HS GPA or national entry test is an admission<br>requirement|-0.0141<br>(0.0251)||||
|Student body, program, and HEI characteristics (PCA scores)|✓|✓|✓|✓|
|Noise controls, state- and field-fixed effects|✓|✓|✓|✓|
|Graduation year fixed effect||✓|✓|✓|
|Observations|22,663|7,177|6,827|7,089|
|R-squared|0.126|0.122|0.109|0.135|
|Mean dependent variable|0.303|0.707|53.52|4.791|
|Average wage (USD 2017 PPP)||||715.62|
|Adj R squared|0.124|0.117|0.103|0.131|

_Source_ : Own estimations using individual-level data from administrative sources and program-level data from the WBSCPS for Brazil. _Notes_ : This table shows coefficients from the (second stage) OLS regressions of student outcomes on the quality determinants selected by LASSO in the first stage. The unit of observation is an individual. Outcomes (dependent variables) are graduation within three years of starting the program (col. 1), and the following labor market outcomes pertaining to the 12-month period following graduation: whether the student is formally employed at least one month (col. 2); percent of months she is formally employed (col. 3); and average monthly wage (col. 4). Average monthly wage is computed over the months worked; it is zero if the individual does not work formally at all. Regressions are weighted using WBSCPS sampling weights. All specifications include noise controls as well as fixed effects for graduation year, state, and field. They also control for PCA scores for the following: student body characteristics (based on survey data), student characteristics (based on individual-level data, including proxies for previous labor market experience), program characteristics (based on survey data), HEI characteristics (based on survey data), and peer characteristics (based on individual-level data, including proxies of peers’ previous labor market experience). See Appendix 1 for the list of the variables included in each index. Standard errors clustered at program level are in parenthesis. Significance levels: * p<0.1, ** p< 0.05, *** p<0.0 89
# **Table A14. Associations between SCP Quality Determinants and Labor Market Outcomes** **_Using Individual- and Program-Level Data for Ecuador_**

||**Ever**<br>**employed the**<br>**year after**<br>**graduation**|**Percent of time**<br>**employed after**<br>**graduation**|**Average**<br>**monthly**<br>**wage (log)**|
|---|---|---|---|
||<br>(1)|(2)|(3)|
|_Infrastructure_||||
|Number of workshops or labs available for practice||2.645***<br>(0.445)||
|_Costs_||||
|Government scholarships are used by some students||7.623*<br>(4.241)||
|_Training and curriculum_||||
|Analyze student performance to solve problems more than once||-8.775**||
|<br>per year||(3.818)||
||0.137***|7.221*|0.650**|
|Collect student satisfaction data more than once per year|<br>(0.0446)|<br>(3.678)|<br>(0.277)|
|Government standards are very important to update program|0.0546<br>|21.14***<br>|0.611**<br>|
||(0.0402)|(3.658)|(0.246)|
|Enrollment trends are very important to update program||-5.478<br>(4.142)||
|Student feedback is very important to update program||10.65***<br>(3.310)||
||0.209***|<br>17.42***|1.212***|
|Teaches numerical competencies|(0.0566)|(6.273)|(0.357)|
||-0.0850*|||
|Teaches a foreign language|(0.0434)|||
||-0100**||-0682**|
|Professional association test required for graduation|.<br>(0.0443)||.<br>(0.285)|
|Time assigned to practical training (%)||-0.253**<br>||
|||(0.122)||
|Offers remediation classes before starting the program||-6.298<br>(3.821)||
|_Link with productive sector_||||
|Industry lends or provides equipment to program for student training||11.11***<br>(3.309)||
|HEI ti tdt f b iti|0.217***|<br>16.11**|1.491***|
|rans suens or jo nervews|(0.0522)|(6.982)|(0.347)|
|HEI coordinates job interviews with firms||-11.31***<br>(4048)||
||0.107**|.<br>21.23***|0.537*|
|HEI provides job market information|(0.0425)|(3.710)|(0.297)|
|HEI has agreements with private firms to hire graduates||6.911**<br>(3.341)||
||0159***|<br>7536|1131***|
|Somebody (from board or staff) is in charge of industry relations|.<br>(0.0604)|.<br>(4.980)|.<br>(0.429)|
|_Faculty_||||
|Classroom observation is very important in faculty evaluation||7.454*<br>(4.460)||
|Students and peers' informal comments are very important in faculty||<br>-9.928**||
|<br>evaluation||(4.793)||
|Research skills are very important when hiring faculty|-0.171***<br>(00488)|<br>-26.70***<br>(4485)|-1.331***<br>(0275)|
||.<br>-0.0628|.<br>-3.187|.<br>-0.363|
|Number of faculty (log)|(0.0379)|(2.836)|(0.261)|
|||<br>0.177***||
|Percent of faculty with 5yrs+ industry experience||(00431)||
||-0.00122*|.<br>-0.110**|-0.00887**|
|Percent of faculty working in the industry|(0.000631)|(0.0514)|(0.00423)|
|Percent of faculty with SCP degree||-0.212***<br>(0.0680)||
|Percent of <40 years old faculty||-0.218***<br>(0.0785)||

90
|Percent of faculty that are women|-0.000868<br>(0.000789)|-0.0655<br>(0.0965)|-0.00500<br>(0.00517)|
|---|---|---|---|
|_Other practices_||||
|Percent of governing body that belongs to current students||0.0112<br>(0.210)||
|Min. scores in HS GPA or national entry test is an admission<br>requirement|0.0902**<br>(0.0449)||0.761***<br>(0.288)|
|Student body, program, and HEI characteristics (indexes)|✓|✓|✓|
|Noise controls, and field-fixed effects|✓|✓|✓|
|Observations|1,214|1,201|1,206|
|R-squared|0.230|0.325|0.247|
|Mean of dependent variable|0.348|23.81|2.294|
|Average wage (USD 2019 PPP)|||300.6|
|AdjRsquared|0.214|0.303|0.232|

_Source_ : Own estimations using individual-level data from administrative sources and program-level data from the WBSCPS for Ecuador. _Notes_ : This table shows coefficients from the (second stage) OLS regressions of student outcomes on the quality determinants selected by LASSO in the first stage. The unit of observation is an individual. Outcomes (dependent variables) are the following labor market outcomes pertaining to the 12-month period following graduation: whether the student is formally employed at least one month (col. 1); percent of months she is formally employed (col. 2); and average monthly wage (col. 3). Average monthly wage is computed over the months worked; it is zero if the individual does not work formally at all. Regressions are weighted using WBSCPS sampling weights. All specifications include noise controls as well as fixed effects for graduation year, state, and field. They also control for PCA scores for the following: student body characteristics (based on survey data), student characteristics (based on individual-level data, including proxies for previous labor market experience), program characteristics (based on survey data), HEI characteristics (based on survey data), and peer characteristics (based on individual-level data, including proxies of peers’ previous labor market experience). See Appendix 1 for the list of the variables included in each index. Standard errors clustered at program level are in parenthesis. Significance levels: * p<0.1, ** p< 0.05, *** p<0.0 91
<!-- Start of picture text --> oOo<br>oO<br>oO<br>co<br>2<br>=]<br>Ris}<br>oO<br>>oO<br>oO oO<br>vo<br>c<br>5<br>a<br>a<br>_—<br>fo}<br>&<br>oO<br>N<br>oO<br>formal employment wage<br>HE All quality determinants [J Student characteristics<br>GE Program characteristics (HE! characteristics<br>EH Country Ms Field of study<br><!-- End of picture text --> <!-- Start of picture text --> Formal employment Wage<br>Costs<br>“Ny L |<br>Infrastructure || |<br>Link with productive sector [| [|<br>“sees | ||<br>=e LF |<br>0 5 10. 15 20-25 0 2 4 6 8<br>% of explained variation % of explained variation<br><!-- End of picture text -->
<!-- Start of picture text --> oS<br>o<br>S8=]Cc<br>Rs—<br>o<br>zo<br>momo)<br>wu<br>Cc<br>§<br>Qo<br>aT<br>Padro)<br>& oON<br>oO<br>% time employed wage<br>HE All quality determinants GE Student characteristics<br>QE Program characteristics SHE! characteristics<br>( Students' admin variables [MJ Peers' admin variables<br>MO Field of study 1) State<br>ME Graduation year<br><!-- End of picture text --> <!-- Start of picture text --> % time employed Wage<br>“Ny |__| |<br>Infrastructure [<br>“ees [ [|<br>— [| [|<br>0246802468<br>%of explained variation % of explained variation<br><!-- End of picture text -->
<!-- Start of picture text --> °<br>3<br>iaCc °<br>=<br>&—<br>a<br>zo<br>ome)<br>a<br>£&<br>Ss<br>a °<br>at<br>—<br>io)<br>&<br>°<br>N<br>°<br>% time employed wage<br>HE All quality determinants GW Student characteristics<br>WE Program characteristics QO HE! characteristics<br>MN Students' admin variables | [J Peers' admin variables<br>PY) Field of study<br><!-- End of picture text -->

<!-- Start of picture text --> % time employed Wage<br>Costs |<br>=“ [| _<br>Infrastructure |<br>eee<br>Other Practices | |<br>— _ is<br>0 5 10 15 0 2 4 6 8 10 12<br>% of explained variation % of explained variation<br><!-- End of picture text -->