metadata: authors: 0: author: Ivaschenko,Oleksiy
1: author: Naidoo,Darian
2: author: Newhouse,David Locke
3: author: Sultan,Sonya M.
count: Papua New Guinea
docna: 0: docna: Can public works programs reduce youth crime
            ? evidence from Papua New Guinea's urban youth
            employment project
docty: Policy Research Working Paper
subtopic: Labor Markets,Labor Policies,Gender and Development,Employment and Unemployment,Crime and Society,Rural Labor Markets
teratopic: Poverty Reduction,Social Protections and Labor,Conflict and Development
abstracts: cdata!: Crime rates in Papua New Guinea's
            capital city of Port Moresby are among the highest in the
            world. Few youth work, and good jobs are scarce. In 2013,
            the National Capital District Commission partnered with the
            World Bank to implement the Urban Youth Employment Project.
            The project offers out-of-school and out-of-work youth two
            months of public works employment or, for academically
            qualified candidates, six months of classroom and on-the-job
            training. This paper presents difference-in-difference
            estimates of project impacts on participants' social
            and criminal behavior, 12 to 18 months after completion. The
            control group consists of observably similar youth living in
            areas not served by the program. Project participants became
            less likely to hang out with friends at night, have a best
            friend involved in crime, and have friends involved in
            fights or robberies. The program also increased subsequent
            employment rates, and significantly reduced aggressive
            behavior and gratuitous property damage. However, there is
            little robust evidence that the program reduced
            participants' engagement in or exposure to crime.  The
            study concludes that the program had strong and healthy
            effects on participants' peer group and behavior, but
            more limited effects on the socioeconomic causes of crime.
geo_reg_key: 517191
repnme: repnme: Can public works programs reduce youth
            crime ? evidence from Papua New Guinea's urban youth
            employment project
display_title: Can public works programs reduce
            youth crime evidence from Papua New Guinea's urban
            youth employment project
listing_relative_url: /research/2017/04/27370864/can-public-works-programs-reduce-youth-crime-evidence-papua-new-guineas-urban-youth-employment-project
topicv3: Life Skills m1326038 857,Public Works m1326041 1665,Gender m1327621 615,Crime m1439217 3041,Self-Employment m1504000 2698
publishtoextweb_dt: 2017-04-18T00:00:00Z
historic_topic: Poverty Reduction,Social Protections and Labor,Conflict and Development
no_of_pages: 38
author_keys: 0: author_key: 633468
1: author_key: 1199102
2: author_key: 450079
3: author_key: 1099036
model_extractions: WPS8032 Policy Research Working Paper 8032

# Can Public Works Programs Reduce Youth Crime?

Evidence from Papua New Guinea’s Urban Youth Employment Project _Oleksiy Ivaschenko Darian Naidoo David Newhouse Sonya Sultan_ Social Protection and Labor Global Practice Group & Poverty and Equity Global Practice Group April 2017
Policy Research Working Paper 8032

# **Abstract**

Crime rates in Papua New Guinea’s capital city of Port Moresby are among the highest in the world. Few youth work, and good jobs are scarce. In 2013, the National Capital District Commission partnered with the World Bank to implement the Urban Youth Employment Project. The project offers out-of-school and out-of-work youth two months of public works employment or, for academically qualified candidates, six months of classroom and on-thejob training. This paper presents difference-in-difference estimates of project impacts on participants’ social and criminal behavior, 12 to 18 months after completion. The control group consists of observably similar youth living in areas not served by the program. Project participants became less likely to hang out with friends at night, have a best friend involved in crime, and have friends involved in fights or robberies. The program also increased subsequent employment rates, and significantly reduced aggressive behavior and gratuitous property damage. However, there is little robust evidence that the program reduced participants’ engagement in or exposure to crime. The study concludes that the program had strong and healthy effects on participants’ peer group and behavior, but more limited effects on the socioeconomic causes of crime.

This paper is a joint product of the Social Protection and Labor Global Practice Group and the Poverty and Equity Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http:// econ.worldbank.org. The authors may be contacted at oivaschenko@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Can Public Works Programs Reduce Youth Crime? Evidence from Papua New Guinea’s Urban Youth Employment Project**

_Oleksiy Ivaschenko, Darian Naidoo, David Newhouse, and Sonya Sultan_<sup>1</sup> JEL Classification Codes: J30, J68, C9 Keywords: Public works, Job Training, Crime, Active Labor Market Programs, Papua New Guinea 1 Oleksiy Ivaschenko (Corresponding author), Sr. Economist, Social Protection and Labor Global Practice, World Bank; email: <u>oivaschenko@worldbank.org; Darian Naidoo, Consultant, World Bank; email: dnaidoo@worldbank.org; David</u> Newhouse, Sr. Economist, Poverty Global Practice, World Bank; email: dnewhouse@worldbank.org; Sonya Sultan, Sr. Social Development Specialist, Social Development Practice, World Bank; email: <u>ssultan@worldbank.org. The</u> views presented in this paper are solely those of the authors, and should not be attributed to the World Bank. We would like to express thanks to Ingo Wiederhoefer, Sonya Woo, Bassam Ramadan, and Tom Gillis for their enthusiastic support for and contributions to the evaluation. We would like to also like to thank Victor Canales, Francesca Tinibar, and Serrah Nao for managing the data collection, and Dung Doan for preparing the baseline questionnaire. Juan Carlos Parra Osorio, Ravi Corea and Prathip Manuel also provided useful comments and support provided with data analysis. We gratefully acknowledge funding was provided by the Rapid Social Response Trust Fund.
# 1. **Introduction**

Active labor market programs (ALMPs) aim to increase the employment, employability and income of participants. They have a decidedly mixed track record of raising employment and earnings, and tend to have modest impacts, especially in the short run.<sup>2</sup> Particular types of programs can be effective, however, when they are well‐tailored to local labor markets and implemented effectively.<sup>3</sup> In recent years, there has been greater interest in the potential for ALMPS to offer non‐pecuniary benefits to young participants, such as a reduction in risky behavior and criminal activity. In contexts where unemployment and crime are both high, ALMPs have the potential to improve social as well as economic outcomes. However, little evidence exists regarding the impacts of ALMPS on social indicators such as crime, especially in fragile contexts.<sup>4</sup> This study seeks to fill the gap using unique data collected in Port Moresby, the capital city of Papua New Guinea (PNG).

Port Moresby is a particularly interesting context to study the social effects of active labor market programs because unemployment and crime are rampant, as the creation of formal jobs and other livelihood opportunities have failed to match rapid population growth. A recent report estimated that the homicide rate in the capital city of Port Moresby was 33 per 100,000 persons in 2010, which would rank among the top 50 cities in the world today (World Bank, 2014). This is not surprising: Among young people aged 15 to 24 in Port Moresby, 29 percent were estimated to live on less than $3.10 per day in 2009/10, and 31 percent were unemployed.

To invest in youth and promote their social inclusion, the National Capital District Commission (NCDC) has, since September 2012, implemented the Urban Youth Employment Project (UYEP) in Port Moresby.<sup>5</sup> The project seeks to improve employment opportunities, earnings potential and living standards of urban youth by providing skills trainings and short‐term placement into employment. It is one of the very few programs that invest in PNG’s marginalized youth, and is the main public intervention attempting to address the economic and social marginalization of unemployed youth. The program successfully targeted youths involved in criminal activity, as 57% of UYEP participants report being a victim of crime, and 46% report having been involved in a criminal activity, at least once in their lives.

The UYEP program consists of two main components. The first component is the Youth Job Corps (YJC), which targets less educated youth and provides short‐term (about 40 days, or two months) _public works_ employment, mostly related to maintaining/cleaning roads. The second component, the Skills Development and Employment Scheme (SDES) targets more educated youth and provides classroom Pre‐ Employment Training (PET) of 1 month followed by on‐the‐job training (OJT) /employment (of 5 months) with employers in various sectors. Both groups of youth are required to participate in a one‐week (40

> 2 Recent reviews of ALMPs include Card, Kluve, and Weber (2015), Brown and Koettle (2015), and Callendo and Schmidt (2015).

> 3 See, for example, Sarvimäki and Hämäläinen (2016), and Attanasio, et al (2011).

> 4 Ralston and Blattman, 2015.

> 5 The implementation of the Project has been taking place under technical assistance from the World Bank.

2
hours) Basic Life Skills Training (BLST) at the start of the program, which aims to improve the soft skills of project participants.

The project operates on a continuous basis, with new batches/intakes of youth (of about on average 500 per intake) participating in a project every few months, though the numbers recruited per intake are now increasing.<sup>6</sup> The new batches/intakes are usually taken from districts that have not previously participated in the project. In total, the project aims to reach approximately 10,500 disadvantaged urban youth between the ages of 16 and 35 who have limited social and economic opportunities in its first five years of operation. So far, as of December 2015, about 8,500 youth have participated in the project. The project was recently extended an additional two and a half years, and is now expected to reach 15,500 youth over a period of 7 years.

The project’s rigorous data collection system is the first of its kind in PNG. All youth who expressed interest in participation were first screened through the Eligibility Screening Survey (ESS) instrument, which, in addition to verifying the basic eligibility criteria (such as age and residency requirement), also collects information on the key socio‐economic characteristics of youth. In selected intakes, youth who were deemed eligible, and were selected for participation in the project, were then given the Eligibility Baseline Survey (EBS), which collected a richer set of socio‐economic variables compared to ESS, including data from a survey module on crime and social behavior.

As part of the impact evaluation effort, a control group was identified in districts around Port Moresby that did not participate in the project.<sup>7</sup> The baseline survey instrument was administered to both the “treatment” group (youth selected for the 4<sup>th</sup> and 5<sup>th</sup> intake of the Project), and the “control” group. This allowed the research team to establish a solid baseline for both groups. Following the participation of the 4<sup>th</sup> and 5<sup>th</sup> intakes in the Project, a follow‐up survey (FUS) was conducted on the samples of “treatment” and “control” groups in the summer of 2015 (about a year to a year and a half after these intakes participated in the Project) to evaluate the changes in the outcomes of interest over time. A detailed timeline of the study/evaluation is presented in Annex 1.

This paper summarizes estimated impacts of program participation on youth crime and anti‐social behavior, and suggests possible channels through which this impact may take place. It is the first such evaluation in the context of Port Moresby. Overall, the results indicate that the program had strong and lasting effects on participants’ social behavior. Participants, relative to controls, became 9 percentage points less likely to hang out with friends late at night, 12 percentage points less likely to have a best friend engaged in crime, and 16 percentage points less likely to have friends involved in fights or robberies. These represent reductions of 20, 50, and 45 percent respectively from their baseline values. Participants also

> 6 The 1st intake started the program in September 2012 and consisted of 240 participants. The 2nd intake started in October‐November 2012, and consisted of 354 participants. The average size of the intakes has grown over time. 7 The control group was defined during the 4th and 5th intakes of the Project, which took place in mid‐2013. In the first and second intakes, the program was oversubscribed and randomly selected participants among a large pool of applicants with the intention of conducting a randomized control trial to evaluate the program. However, controls were not clearly informed that they had to reapply to the program in subsequent intakes, and reportedly expressed their displeasure by physically attacking participants selected in later intakes. As a result, the NCDC and the World Bank decided to evaluate the program against a control group selected from neighboring villages.

3
reported a large drop in aggressive behavior. The program reduced participants’ frequency of threatening to use force by 13 percentage points and of fighting back in response to an attack by 11 percentage points, which correspond to 65 and 25 percent reductions relative to baseline.

Taken at face value, the results suggest that the effect of the program on youths’ peer group and behavior was sizeable and beneficial. These positive changes in participants’ social lives and behavior translated into limited reductions in criminal activity. Participants, perhaps because they spent less time hanging out at night, reported engaging in vandalism for fun. Prevalence fell by 6 percentage points more among treatments than controls, representing a roughly 0 percent reduction from participants’ baseline. But there is little robust evidence that the program reduced participants’ engagement in or exposure to other types of crime. Although the estimated effects are similar in magnitude for alcohol‐related crime, they are not statistically significant. Moreover, for more serious types of crime– assault, robberies, and trespassing ‐‐ the large drop among participating youth was comparable to that of the control group and therefore cannot be attributed to the program. One explanation is that the program had weaker effects on the type of youth that were least likely to engage in crime, and more investigation is necessary to better understand the relative importance of economic and social factors in explaining why the program failed to curb criminal activity.

The remainder of the paper is structured as follows. Section 2 provides a brief literature review. Section 3 details of the UYEP’s intervention design and implementation. Section 4 describes the evaluation’s objectives, methodology and data. Section 5 presents the context for the findings and discusses the key evaluation results. Section 6 presents concluding remarks.

# 2. **What is known about public works employment and crime?**

The intuitive popularity of public works and job training programs for youth as an effective way to reduce crime in high‐risk areas rests on the assumption that productive employment reduces incentives to engage in anti‐social behavior or commit crime. This is consistent with two theories of crime (Bushway and Reuter, 2003). The first is economic choice theory, in which criminals choose rationally to engage in criminal activity if the economic benefits exceed the costs, which are lower for those without decent job opportunities (Becker, 1968, Ehrlich, 1973). A second is control theory, whereby the absence of employment opportunities leads to a breakdown in social bonds and low levels of social organization in the community (Wilson, 1996). Other theories, however, suggest more limited effects of employment programs on crime. One is labeling theory, which posits that criminals are stigmatized, both to others and themselves. This can make it difficult for ex‐criminals to obtain jobs and reintegrate with society, and can also limit the effectiveness of job programs in communities that have been stigmatized. Another factor is that, in response to better employment opportunities, criminals adjust on the intensive margin rather than giving up crime altogether (Reuter et al., 1990; Berman, et al., 2011). Given the lack of a clear theoretical link between crime and employment programs, it is important to consider the limited empirical evidence that is available.

4
Traditionally, most studies of employment and crime were conducted using American data and many found conflicting or small effects of labor market outcomes on crime (Bushway, 2010). But more recent empirical evidence from the OECD on the relationship between employment and crime suggests more significant effects (Mustard, 2010). Fougere et al (2008) find that increased youth unemployment, instrumented for by changes in industry structure, increased burglaries, thefts, and drug offenses, with particularly large effects for youth. In the US, Lin (2007) estimates that a 1 percentage point increase in the average state unemployment rate raises property crime by about 4 to 6 percent. Gould, et al (2002) document that both wage levels and unemployment rates are significantly related with crime, with estimated elasticities with respect to wage levels of ‐0.55 for property crime and ‐1.08 for violent crime. Finally, a randomized evaluation of the youth job corps, a program for disadvantaged 16‐24 year olds that offers 30 weeks of training and a stipend, found that it reduced crime among all subgroups. However, the monetary benefits of the reduction in crime were estimated to be only about 6 percent of all program costs (Kluve, 2014). Heller (2014) finds that the provision of an 8 week summer job to disadvantaged youth in Chicago reduced the probability of violent‐crime arrest by 43 percent.

The few existing studies from developing countries on how employment‐related interventions affect engagement in illicit activities or violent conflict suggest more nuanced effects than in the OECD. One set of studies examines the effects of India’s National Rural Employment Guarantee Scheme (NREGS), and generally finds weak effects. For instance, Das (2015) examines at the district level, how implementation of India’s National Rural Employment Guarantee Scheme (NREGS) affected official reports of violent crime from 2001 to 2011. The results indicate that the program reduced reports of kidnapping, but most of the relationships between crime and program implementation were not robust to the inclusion of district‐ specific time trends. Other studies have examined the effect of NREGS on the prevalence on Maoist violence, and found mixed results. Dasgupta et al. (2014) find that NREGS substantially reduced the prevalence of armed conflict as proxied by local news reports. This is consistent with Feltzer (2014) who found that NREGS mediated the effect of adverse rainfall shocks on conflict. However, Khanna and Zimmerman (2015) study the same program using a different methodology, and find that the program increased insurgent activity, which they ascribe to increased cooperation between villagers and the police outweighing the increase in the opportunity cost of conflict.

A second set of studies evaluates the social effects of training programs in Latin America. Calearo and Rozo (2016)) use experimental data from the Galpao training program in Brazil. The results fail to show a discernible average effect on risky behaviors such as tobacco and alcohol use, though positive effects were apparent for a subset of participants with stronger self‐reported socio‐emotional skills. In the Dominican Republic, meanwhile, a youth training program reduced the probability of teenage pregnancy by about 8 percentage points, or 20 percent (Novella and Ripani, 2014), though no evidence is available on the effects on crime.

Finally, Blattman and Annan (2015) examine the results of a randomized evaluation of a program to rehabilitate high‐risk men in Liberia, and also find mixed results. The program provided agricultural training and capital inputs to former soldiers in Liberia. Program participants reduced hours devoted to illicit resource extraction by 20 percent, and reduced interest in fighting in a short‐lived conflict in 5
neighboring Côte d’Ivoire. But the program led to virtually no exit of illicit activity, no change in peer networks, nor any reduction in aggression or attitudes towards violence.

# 3. **The UYEP intervention’s design and implementation**

The UYEP aims to contribute to PNG’s National Youth Policy priorities to support livelihood promotion, poverty reduction, and socially‐healthy behavior for youth, by assisting the growing segment of disadvantaged youth who have dropped out of school or university and lack the basic competencies required to compete in the labor market. The program provides unemployed, out of school youth in Part Moresby with human capital/training and work experience, as well as income, from temporary employment opportunities, and to ultimately increase their employability. This is done through three main components: The Youth Job Corps (YJC), the Skills Development and Employment Scheme (SDES) and Project Management.

The YJC involves a week of Basic Life Skills Training (BLST) followed by a two‐month (40 days) public works program that serves roughly three‐quarters of the applicants. The SDES is made up of Basic Life Skills Training (BLST), followed by one month of the classroom Pre‐Employment training (PET), and five months of the on‐the‐job (OJT) training with an employer in Port Moresby. People for PET are selected on the basis of having the highest scores in the tests, and doing well in the BLST. This ensures selectivity into a limited number of PET positions. OJT is offered to roughly one‐quarter of the BLST/PET graduates who are rated highly by their trainers and pass tests with the highest marks. The final component is Project Management, which insures that all Project implementation activities are on track and are properly monitored and evaluated. Given this intervention for youth is something new in PNG, great effort was put into establishing mechanisms, systems and capacity to deliver program services efficiently to targeted youth.

Youth are accepted into the program through continuous intakes, which occur once every 3 months. The enrollment of each intake into the Project consists of several steps:

- (i) First, the Project Management Unit (PMU) of the NCDC initiated a communication and awareness building campaign in selected districts inviting youth that meet the described eligibility criteria to apply. This process usually took 2‐4 days in a given neighborhood/district of the city. The PMU also worked with community leaders to reach out to potentially interested youth.

- (ii) Second, interested applicants showed up at a screening site on a given date to undertake the Eligibility Screening Survey (ESS), which was administered by an independent survey firm. The ESS collected basic information, including the socio‐economic characteristics of youth.

- (iii) Third, the survey firm collected and processed the ESS data to establish eligibility. To be eligible for selection into the Project, youth were required to be of eligible age (16‐35),<sup>8</sup> have

> 8 It is worth noting that while initially the upper age limit was set at 25 years, it was subsequently (starting from intake 4) increased to 35 years.

6
been out of work and school for the last six months, and have been residing in the National Capital District (NCD)/Port Moresby for at least one year. To verify eligibility, representatives of the survey firm also conducted interviews with community leaders, and declared applicants ineligible if a consensus emerged that they have misrepresented any of these three criteria.<sup>9</sup> As the program has become better known over time, greater numbers of youths began to apply to the program. To balance the interests of competing groups, each neighborhood group was allocated a certain number of places, and the program strived for a 50:50 ratio of men to women applicants. Participants were selected randomly from the pool of eligible applicants, separately by neighborhood group and gender.

- (iv) Successful applicants were given the Eligibility Baseline Survey (EBS) that collects a richer set of information compared to ESS (see Annex 2 for a more detailed description of various surveys). They were then offered a 40‐hour Basic Life Skills for Employment Training (BLST) administered over the course of one week. Each BLST participant took a literacy and numeracy test, which was used to determine the quarter of each intake that is then streamed into the SDES component. Usually, many more than a quarter of the participants are literate, so only those with top scores and strong participation in the BLST were allowed to proceed to the next stage of the SDES component.

- (v) Participants were enrolled in either Component 1 (YJC) or Component 2 (OJT) of the Project. Participants in Component 1 (the YJC) were offered to undertake two months (40 days) of public works, such as cleaning the sides of roads. When youths completed the two‐month employment, they completed the program. This component of the Project was offered to approximately 12,500 eligible urban youths over the course of seven years, equivalent to approximately 80 percent of total program participants.

- (vi) Meanwhile, participants in Component 2 were provided with: i) one month (equivalent to 160 hours) of pre‐employment training (PET) that aims to prepare them for employment in the private, public, or civil society sectors, and: ii) five months of on‐the‐job training (OJT) with a private or public company. The PET has two program tracks that participants can choose from ‐‐ the first focuses on trade‐, industrial‐ and commerce‐related jobs, while the second focuses on basic book keeping, data entry, business practices, and information technology skills. Each PET program track is expected to train about 2,000 eligible youths over the life of the Project.

The UYEP enrollment process is summarized in Table 1.

> 9 In general, about 90% of youth screened through ESS are deemed eligible as the eligibility criteria are generally communicated by the Project and are known in the community before the enlisting takes place.

7
**Table 1: Outline of the UYEP process (selection and allocation of participants)**

|**Step**|**Activity**|**Participants**|
|---|---|---|
|1|Publicity campaign of the program|National Capital District Commission|
|2|Application|Interested youth|
|3|Participants selected|The screening survey firm<br>(implementation agency), under the<br>oversight of the Project Management<br>Unit (PMU) and with input from<br>community leaders, determines<br>applicants’ eligibility based on the ESS.|
|4|Basic Life Skills Training (BLST) and<br>Literacy and Numeracy test + EBS|All program participants|
|5.1|Component 1: YJC|Majority of participants. This<br>component is targeted to vulnerable<br>youth with limited education, and who<br>need short‐term employment to earn<br>some money.|
|5.2a|Component 2a: PET training for one<br>month|Participants that are rated above the<br>70th percentile on completion of the<br>basic life skills training and the basic<br>literacy and numeracy test|
|5.2b|Component 2b: OJT for 5 months|Participants that have at least 80%<br>attendance during the PET training then<br>progress to the OJT for 5 months.|

Source: Compiled by authors. Note: The project design has subsequently evolved and may change in the future In the past, selected YJC graduates, such as those who achieved perfect attendance during the 40 days of public works, were also graduated to the SDES. This incentive proved to be very effective in ensuring high attendance rates at the YJC. It was eventually phased out in 2015, however, as too many YJC graduates maintained perfect attendance. Now, a small proportion of the YJC who have demonstrated strong commitment and worked well under the YJC are provided extra days of semi‐skilled work under the YJC scheme itself. This typically consists of building concrete footpaths and drainage within the city, and has so far proved to be popular with the youth.

There is now a wider range of interventions offered to youth joining UYEP, especially for those joining the YJC stream. All the new possible pathways through the UYEP project are shown in Table 2 below. The inputs provided by UYEP now range from the minimum interventions of BLST + 40 days of work on public works, to about 6 months of various inputs (the most intensive set being BLST + YJC + PET + OJT). The additions to the YJC stream have been introduced to incentivize those on YJC to keep working, and offer them a chance to graduate to further training, and work options. The project will need to continue to evaluate which of the pathways is suitable for different groups of youth (e.g. for youth with more education versus those who are illiterate) and what combination of interventions has the most impact.

8
**Table 2: UYEP Streams**

|**1. Basic YJC Stream**||**Additional YJC Activities**|
|---|---|---|
|Basic Life Skills Training|+ YJC Work Experience|+<br>Community Based Project<br>**Or**<br>+<br>2<sup>nd</sup>YJC work<br>experience/additional semi‐skilled<br>work<br>**Or**<br>+<br>Pre‐Employment Trainingand OJT|
|**2. Basic OJT Stream**||**Additional OJT Activities**|
|Basic Life Skills Training|+ Pre–<br>Employment<br>training<br>+<br>OJT Work<br>Placement|+<br>Mentoring during OJT|

Source: Compiled by authors.

# 4. **Impact Evaluation, Data, and Sample Balance**

# _4.1 Objectives and Methodology_

The main objective of the impact evaluation of the UYEP was to assess the impacts of the program participation on the participants’ social behavior and criminal activity. To do so, we utilize a Difference‐in‐ Differences (DD) evaluation strategy to estimate the effect of program participation on a wide variety of indicators (Abadie, 2005; Imbens et al., 2009) This strategy compares the average change over time in the outcome variable for the treatment group to the average change over time for the control group. The key identifying assumption is that trends, in the absence of the program, would be identical for treatment and control households. A wide range of outcomes are considered, including social (or anti‐social) behavior and social environment, (self‐reported) engagement in a bad/criminal activity, and the reported incidence of being a victim of such an activity. We also explore, to the extent possible given data limitations, potential channels underlying the results.

# _4.2 Data_

As part of the impact evaluation design of the UYEP, the Eligibility Baseline Survey (EBS) was administered during the 4<sup>th</sup> and 5<sup>th</sup> intakes to both treatment group and control groups. The treatment group consisted of youth who were selected for the 4<sup>th</sup> and 5<sup>th</sup> intake of the project, while the control group consisted of eligible youth from neighboring districts who had not participated in program at the time of the follow‐ up survey (FUS). This survey was conducted in the summer of 2015, about a year after participants’ completion of the program, to evaluate the changes in the variables of interest over time. The primary purpose of the FUS was to collect information on post‐intervention outcomes of treatments and controls to estimate various impacts of the program. A more detailed description of the ESS, EBS and FUS is provided in Annex 1.

9
The FUS was conducted on all youth from treatment and control groups who could be successfully re‐ contacted, resulting in sample sizes as presented in Table 3. Only about 45% of treatments and controls were successfully re‐contacted, due to the high rates of mobility among the youth, and a very challenging security environment in which this survey work has taken place. To assess the extent of attrition bias, we regressed our main outcome of interest variables at baseline on an indicator for attrition. This analysis, which is further elaborated below, reveals no significant overall difference in baseline outcomes between attrition and non‐attrition individuals. The section bellows describes in detail how various samples used for evaluation were created, and how these samples compare to each other in terms of the basic characteristics at the baseline.

**Table 3: Sample sizes for various groups used in evaluation**

||Eligible|Treatments|Re‐interviewed<br>Treatments|Controls|Re‐interviewed<br>controls|
|---|---|---|---|---|---|
|Intake 4|1,199|381|168|270|110|
|Intake 5|1,014|362|170|299|140|
|Total|2,213|743|338|569|250|

Source: Authors’ calculations based on survey data.

# _4.3 Sample Balance_

## 4.3.1 Sample Balance (participants vs. eligible non‐participants)

The youths who were selected into the program were randomly drawn, with stratification by gender, from a larger sample of eligible applicants. As noted in Table 3, 743 individuals were accepted into the program (381 in intake 4 and 362 in intake 5) out of a total of 2,213. Since the baseline screening survey was administered to all applicants, a logical first step is to verify that the baseline characteristics of program participants look similar to all eligible applicants, as would be expected if selection were indeed random.

We compare the means of several basic baseline characteristics between participating youth and other eligible youth who did not participate, and test whether the difference in means is statistically significant. These results are reported in Table 4 (A). Column 1 shows the means and standard deviations for eligible non‐treatments (or “other eligible”), while Column 2 shows the corresponding means and standard deviations for treatment youth. Column 3 indicates the difference in means (and standard errors underneath), and if the difference is statistically significant at conventional levels. The results indicate that, as expected, the two groups are very similar to each other. The only characteristic that is somewhat different (at a 5% level) is never having a formal (wage) job before, with respectively 79% of treatments and 75% of eligible non‐treatments being in this category. In terms of the levels of basic characteristics, treatments are on average 24.7 years old, 63% are males, 37% are married, and the average educational attainment is 9.3 years (see Column 2, Table 4A). 36% of them report having some earnings from an informal job/self‐employment last month, and 32% report looking for a job during the same reference period. Overall, the similarity of the two groups is consistent with the random selection of participants from the pool of eligible candidates.

10
_Table 4A: Balance between Treatments and other eligible youth_

|Variable of interest|**Other eligible**<br>(mean/sd)<br>(1)|**Treatments**<br>(mean/sd)<br>(2)|**Difference**<br>(mean/se)<br>(3)|
|---|---|---|---|
|_Individual's basic characteristics:_||||
|Age|24.65<br>|24.70<br>|‐0.05<br>|
||_(4.92)_|_(4.85)_|_(0.22)_|
|Male (1 if male, 0 if female)|0.65|0.63|0.02|
||_(0.48)_|_(0.48)_|_(0.02)_|
|Married|0.34|0.37|‐0.03|
||_(0.47)_|_(0.48)_|_(0.02)_|
|Years (grades) of education completed|9.13|9.32|‐0.19|
||_(2.79)_|_(2.81)_|_(0.13)_|
|_Individual's household characteristics:_||||
|Household size|9.01|9.29|‐0.28|
||_(4.59)_|_(4.46)_|_(0.21)_|
|N of females in a household|3.91|4.09|‐0.18|
||_(2.38)_|_(2.64)_|_(0.11)_|
|N of males (16+) in paid work|1.11|1.18|‐0.06|
||_(1.10)_|_(1.20)_|_(0.05)_|
|Dwelling has floors made of concrete|0.05|0.06|‐0.01|
||_(0.23)_|_(0.24)_|_(0.01)_|
|Dwelling has floors made of wood|0.92|0.90|0.02|
||_(0.27)_|_(0.30)_|_(0.01)_|
|_Individual's employment characteristics:_||||
|Never had a wage (formal) job|0.75|0.79|‐0.04|
||_(0.43)_|_(0.41)_|_(0.02)_|
|If earned money through informal job/self‐<br>employment last month|0.35|0.36|‐0.02|
||_(0.48)_|_(0.48)_|_(0.02)_|
|If searched for a paid job last month|0.33|0.32|0.00|
||_(0.47)_|_(0.47)_|_(0.02)_|
|N of observations|**1470**|**743**||

Notes: Table reports means and standard deviation (error) for the two groups, and their difference. *** p<0.01; ** p<0.05; * p<0.1 11
# 4.3.2 Sample Balance (re‐interviewed vs. lost treatments)

As noted above, attrition rates in the follow‐up survey were quite high. Of the 743 individuals in the treatment group, only 338 individuals (or about 45%) were successfully re‐contacted and interviewed. This will bias the estimate impacts of the program if changes in youths’ socioeconomic conditions have different effects on the probability that treatments and controls were re‐interviewed. For example, if successful graduates of the program were more likely than successful controls to respond to an interview request, because they felt gratitude towards the program, the benefits of the program could be overstated. Similarly, the benefits of the program could also be overstated if less successful program graduates were less likely to be successfully re‐interviewed because they are ashamed of their current status, or are less favorably disposed to the program.

While the extent of this attrition bias is unobservable, one can gain a measure of insight into the potential for bias by comparing the mean baseline characteristics of participants who were re‐interviewed with those who were not. Similar characteristics among these groups would provide a degree of confidence that observable baseline characteristics, at least, are not systematically related to the probability that participants were successfully re‐interviewed in the follow‐up survey.

The results of this comparison are reported in Table 4B. Column 1 shows results for participants who were not re‐interviewed, Column 2 shows results for participants who were, and FUS, and Column 3 shows the difference. Again, we find that these two groups are very similar in their basic baseline characteristics. They are only slightly different in the mean age, with re‐interviewed treatments being on average 9 months (0.75 years) older, at the time of the baseline, than non‐interviewed treatments. Furthermore, regressions of the main outcomes of interest, measured at baseline, on a dummy for attrition show no significant difference in outcomes at the baseline between the two groups.

To summarize, our results indicate that, in terms of observables, our sample of treatments is a good representation of all youth eligible for the project, and that the treatment group that was successfully re‐ interviewed is very similar to those that were not, in terms of both the baseline basic characteristics and baseline outcomes.

# 4.3.3 Sample Balance (re‐interviewed vs. lost controls)

We next perform the same exercise for controls, comparing the observable baseline characteristics of those who were re‐interviewed with those who were not. The sample of controls consists of 569 individuals, of which 250 (or about 44% of the total sample) were successfully re‐interviewed.<sup>10</sup> The results of this analysis are reported in Table 4 (C). A couple of differences emerge between controls who were re‐contacted and those who were not. In particular, the re‐contacted group contains a statistically

> 10 Note that the original sample of controls is 595 individuals. However, we have missing data for 27 individuals, which results in a sample of 569 individuals for which we report results.

12
_Table 4B: Balance between treatments that were re‐interviewed and those that were not._

|Variable of interest|**Treatments,**<br>**non‐FUS**<br>(mean/sd)<br>(1)|**Treatments,**<br>**FUS**(mean/sd)<br>(2)|**Difference**<br>(mean/se)<br>(3)|
|---|---|---|---|
|_Individual's basic characteristics:_||||
|Age|24.36|25.11|‐0.75|
||_(4.90)_|_(4.77)_|_(0.36)_|
|Male (1 if male, 0 if female)|0.64|0.62|0.02|
||_(0.48)_|_(0.49)_|_(0.04)_|
|Married|0.35|0.39|‐0.05|
||_(0.48)_|_(0.49)_|_(0.04)_|
|Years (grades) of education completed|9.44|9.18|0.26|
||_(2.81)_|_(2.81)_|_(0.21)_|
|_Individual's household characteristics:_||||
|Household size|9.25|9.34|‐0.09|
||_(4.41)_|_(4.53)_|_(0.33)_|
|N of females in a household|4.02|4.18|‐0.16|
||_(2.64)_|_(2.64)_|_(0.19)_|
|N of males (16+) in paid work|1.20|1.15|0.04|
||_(1.30)_|_(1.07)_|_(0.09)_|
|Dwelling has floors made of concrete|0.06|0.06|0.01|
||_(0.25)_|_(0.23)_|_(0.02)_|
|Dwelling has floors made of wood|0.89|0.91|‐0.02|
||_(0.32)_|_(0.28)_|_(0.02)_|
|_Individual's employment characteristics:_||||
|Never had a wage (formal) job|0.81|0.78|0.03|
||_(0.40)_|_(0.42)_|_(0.03)_|
|If earned money through informal job/self‐<br>employment last month|0.34|0.39|‐0.04|
||_(0.48)_|_(0.49)_|_(0.04)_|
|If searched for a paid job last month|0.31|0.33|‐0.02|
||_(0.47)_|_(0.47)_|_(0.04)_|
|Number of observations|**405**|**338**||

Notes: Table reports means and standard deviation (error) for the two groups, and their difference. *** p<0.01; ** p<0.05; * p<0.1 13
_Table 4C: Balance between controls that were re‐interviewed and those that were not._

|Variable of interest|**Controls, not re‐**<br>**interviewed**<br>(mean/sd)<br>(1)|**Controls, re‐**<br>**interviewed**<br>(mean/sd)<br>(2)|**Difference**<br>(mean/se)<br>(3)|
|---|---|---|---|
|_Individual's basic characteristics:_||||
|Age|24.15|23.45|0.70|
||_(5.16)_|_(4.81)_|_(0.42)_|
|Male (1 if male, 0 if female)|0.52|0.65|‐0.13|
||_(0.50)_|_(0.48)_|_(0.04)_|
|Married|0.40|0.39|0.02|
||_(0.49)_|_(0.49)_|_(0.04)_|
|Years (grades) of education completed|8.71|8.44|0.27|
||_(2.96)_|_(3.06)_|_(0.26)_|
|_Individual's household characteristics:_||||
|Household size|8.58|8.13|0.44|
||_(4.27)_|_(4.06)_|_(0.35)_|
|N of females in a household|3.88|3.66|0.22|
||_(2.35)_|_(2.22)_|_(0.19)_|
|N of males (16+) in paid work|1.23|1.06|0.17|
||_(1.18)_|_(1.02)_|_(0.09)_|
|Dwelling has floors made of concrete|0.15|0.04|0.11|
||_(0.35)_|_(0.19)_|_(0.02)_|
|Dwelling has floors made of wood|0.82|0.93|‐0.11|
||_(0.39)_|_(0.26)_|_(0.03)_|
|_Individual's employment characteristics:_||||
|Never had a wage (formal) job|0.74|0.64|0.10|
||_(0.44)_|_(0.48)_|_(0.04)_|
|If earned money through informal job/self‐<br>employment last month|0.25|0.32|‐0.07|
||_(0.44)_|_(0.47)_|_(0.04)_|
|If searched for a paid job last month|0.26|0.25|0.01|
||_(0.44)_|_(0.43)_|_(0.04)_|
|Number of observations|**319**|**250**||

Notes: Table reports means and standard deviation (error) for the two groups, and their difference. *** p<0.01; ** p<0.05; * p<0.1 14
significantly higher share of males, and also contains a lower share of youth ever having a formal (wage) job. In addition, the sample of re‐interviewed controls seems to have somewhat worse quality of dwelling characteristics, as measured by the material of flooring, which could reflect the different district/neighborhood mix for the two samples.<sup>11</sup> In general, the majority of the characteristics seem to be very similar for the two samples, and the few differences that are found do not appear to warrant any major concern.

As was the case for treatments, we also investigate the presence of potential attrition bias for controls by regressing of the key outcomes of interest, measured at baseline, on an attrition dummy. We find that in most cases the attrition dummy is either not significant, or becomes so once location effects are included as controls. Out of about 10 outcomes of interest, we find only two for which some differences remain even after controlling for the location effects, at 5% and 10% levels, respectively. In general, these results indicate no systematic bias between controls who were re‐interviewed and those who were not.

4.3.4 Sample Balance (re‐interviewed controls vs. re‐interviewed treatments) The differences‐in‐differences evaluation strategy does not require the treatment and control group have to have the same mean of the outcome of interest, as long as the trends would be the same absent the program. However, it remains instructive to compare the baseline characteristics of the re=interviewed treatment and re‐interviewed control groups, since these are the two groups that are used to estimate the effects of the program. The results of this analysis are reported in Table 4 (D). In terms of individual characteristics, the treatment group is on average 1.67 years older, has 0.75 extra years of education, and is less likely to report having a formal job in the past. These differences are significant at a 1% level. The treatment group youth also seem to be coming from somewhat larger (by about one person) households. We do not find significant differences in other characteristics, including gender, marital status and propensity to have earnings from informal jobs. The treatment group is somewhat more likely to look for a paid job over the past month, with a difference in this indicator between groups being significant at a 5% level.

To summarize, the results presented above indicate how various samples are constructed, how they relate to each other, and what the basic baseline characteristics of youth are, which provides the context for the discussion on the impacts (changes in the outcomes of interest over time) that follows. Overall, the differences in baseline characteristics due to attrition appear to be relatively minor.

> 11 When regressing floor characteristics on the FUS/non‐FUS dummy and district/location fixed effects, the difference in the quality of dwelling between the two samples becomes less pronounced.

15
_Table 4D: Balance between treatments that were re‐interviewed and controls that were re‐interviewed._

|Variable of interest|**Re‐interviewed**<br>**Controls**<br>(mean/sd)|**Re‐interviewed**<br>**Treatments**<br>(mean/sd)|**Difference**<br>(mean/se)|
|---|---|---|---|
||(1)|(2)|(3)|
|_Individual's basic characteristics:_||||
|Age|23.45|25.11|‐1.66|
||_(4.81)_|_(4.77)_|_(0.40)_|
|Male (1 if male, 0 if female)|0.65|0.62|0.04|
||_(0.48)_|_(0.49)_|_(0.04)_|
|Married|0.39|0.39|‐0.01|
||_(0.49)_|_(0.49)_|_(0.04)_|
|Years (grades) of education completed|8.44|9.18|‐0.74|
||_(3.06)_|_(2.81)_|_(0.25)_|
|_Individual's household characteristics:_||||
|Household size|8.13|9.34|‐1.21|
||_(4.06)_|_(4.53)_|_0.3615181_|
|N of females in a household|3.66|4.18|‐0.52|
||_(2.23)_|_(2.63)_|_(0.21)_|
|N of males (16+) in paid work|1.06|1.15|‐0.10|
||_(1.02)_|_(1.07)_|_(0.09)_|
|Dwelling has floors made of concrete|0.04|0.06|0.02|
||_(0.19)_|_(0.23)_|_(0.02)_|
|Dwelling has floors made of wood|0.93|0.91|0.01|
||_(0.26)_|_(0.28)_|_(0.02)_|
|_Individual's employment characteristics:_||||
|Never had a wage (formal) job|0.64|0.78|‐0.14|
||_(0.48)_|_(0.42)_|_(0.04)_|
|If earned money through informal job/self‐||||
|employment last month|0.32|0.39|‐0.06|
||_(0.47)_|_(0.49)_|_(0.04)_|
|If searched for a paid job last month|0.25|0.33|‐0.09|
||_(0.43)_|_(0.47)_|_(0.04)_|
|Number of observations|**250**|**338**||

Notes: Table reports means and standard deviation (error) for the two groups, and their difference.

- *** p<0.01; ** p<0.05; * p<0.1

16
# 5. **Evaluation results**

This section focuses on the presentation and the discussion of the key evaluation results, with a focus on the outcomes of interest related to social behavior and crime. Before discussing the main results, the first subsection describes the socioeconomic context of Port Moresby, and presents descriptive evidence on the extent of crime and anti‐social behavior from the baseline sample.

# _5.1 Context_

PNG is disproportionately young, as 67% of the national population is below 35 years of age and 25% are aged 16 to 35.<sup>12</sup> Urban youth face several challenges. For example, in the 2009/10 Household Income and Expenditure Survey, 58% of urban youth had dropped out of school or tertiary education. 60% of urban youth were unemployed, and less than 7% of those employed are in formal wage jobs5. Furthermore, PNG’s rapid GDP growth has had a limited effect on the well‐being of PNG’s youth. Despite real GDP annual growth of almost 8% in the last decade, urban wage employment decreased by 11 percentage points between 1996 and 2009/10. Rapid increase in rural‐urban migration, high rates of poverty and inequality, and high levels of unemployment create community stress that contributes to crime and violence, which is highly prevalent in the main urban centers. For example, Port Moresby and Lae City reported 33 and 66 homicides per 100,000 persons, respectively, between 2010 and 2013. According to surveys in Port Moresby, Youth aged 19 to 29 are the most active in crime; in fact, criminal activity is so pervasive that 54% of youth report to have been a victim of crime, and 35% report to have been engaged in crime at some point.<sup>13</sup> The national government is well aware of these challenges, and established “Human Capital Development, Gender, Youth, and People Empowerment” as one of the 7 pillars in its long‐term ‘Vision 2050’ plan. This plan states that “programs and systems must be improved and expanded to encourage greater participation from youths in all organizations and institutions, and in development programs”. In addition, the Development Strategic Plan (DSP) 2010‐ 2030 sets ambitious targets to provide universal education for all, ensure that all school graduates enjoy employment opportunities, and also, to halve the rate of youth crime.

Given the emphasis on youth well‐being and crime, the evaluation was collected on a variety of behavioral and criminal characteristics, which are divided into three groups. The first set is anti‐social behavior associated with either being a victim of or engaging in criminal activity. These include hanging out with friends in the streets late at night, having friends who have engaged in a criminal activity, or frequently getting drunk. The second set is self‐reported engagement in criminal activity, such as stealing, assault (physical or verbal), alcohol or drug‐related activity (e.g., fights), trespassing and domestic violence. The final group of indicators is the reported incidence of being a victim of a criminal activity.

> 12 Derived from 2011 Census Report.

> 13 National Youth Policy 2007‐2017, National Youth Commission, 2007.

17
In most contexts, concerns about negative stigma might lead few youth to self‐report being involved in anti‐social behavior or criminal activity. Since criminal activity is so common in Port Moresby, however, rates of self‐reported victimization and involvement are high in both the baseline and follow‐up survey. Other factors may also have contributed to respondents freely sharing their experience with crime. During the interview process, respondents were explicitly informed that their responses to these sensitive questions would not affect their chances of being selected for the project. The survey was also managed by an independent survey firm not associated with the Project Management Unit. Finally, participants were informed that the project had a specific objective of targeting disadvantaged youth, which in the context of NCD has included many youth who may have been involved in anti‐social behavior or criminal activities in the past.

As a result, a large share of eligible youth from intakes 4 and 5 report that they have engaged in, or been a victim of, anti‐social behavior or criminal activity at least once in their life. Applicants were somewhat more likely to report being victims of crime, but reported rates of victimization and perpetration are both remarkably high. For example, 53.3% of all project eligible youth, including 57.3% of all males and 47.7% of all females, reported being victimized by criminal activity in the past (see Figure 1). Moreover, 36.7% of eligible youth report being involved in the past in at least one criminal activity. While the survey is not a representative sample of youth, the rates of self‐reported crime are still remarkably high.

Figure 2 presents the results on engagement in anti‐social behavior or criminal activity or being a victim of it by the type of activity. This includes stealing, assault, alcohol and drug related activity such as fights, and trespassing.<sup>14</sup> For each of these activities, we report the incidence of self‐reported perpetration and victimhood, disaggregated by gender (see Figure 2). The results indicate that 37% of youth report being a victim of stealing, with rather similar levels for males and females (38.3% and 34.7%, respectively). Not surprisingly, significant differences emerge by gender, as 24.2% for males and 3.0% for females reporting previous involvement (see Panel A, Figure 2).

The reported incidence of assault is also very high, both for perpetration and victimhood. The average reported incidence of being an assault victim is 34.7%; 39.1% for males and 26.7% for females. Strikingly, 31.1% of males also report having engaged in assault at least once. The reported incidence for females is much lower, but still quite significant, at 16.6% (see Panel B, Figure 3). It is worth noting that the definition of assault includes both physical and verbal abuse. With regard to domestic violence, 19.8% of sample respondents indicate being a victim at some point in life, with 17% and 24.7% for males and females, respectively. 14.2% also report being involved in such behavior (see Panel B, Figure 2).

> 14 We have also looked at the category “other criminal activity” in the survey, but since we have found only a few

> (4) observations in this category, the results are not reported here.

18
**Figure 1: Reported incidence of being engaged in or a victim of criminal activity**

<!-- Start of picture text --> Panel A: Any crime ‐ activity/victim, %<br>70.0<br>60.0 57.3<br>53.9<br>45.6 47.7<br>50.0<br>40.0 36.7<br>30.0<br>20.5<br>20.0<br>10.0<br>0.0<br>Total Males Females<br>Activity Victim<br><!-- End of picture text --> Source: Authors’ calculations.

**Figure 2: Reported incidence of being involved in a bad/criminal activity or being a victim of it, by type of activity**

<!-- Start of picture text --> Panel A: Stealing ‐ activity/victim, % Panel C: Alchol‐related (fights) ‐ activity/victim, %<br>45.0 45.0<br>40.0 40.0<br>37.0 38.3 34.7<br>35.0 35.0<br>28.1 30.8<br>30.0 30.0<br>24.2 24.6<br>25.0 25.0<br>20.1<br>20.0 20.0<br>16.6 13.5<br>15.0 15.0<br>10.0 10.0<br>5.6<br>5.0 3.0 5.0<br>0.0 0.0<br>Total Males Females Total Males Females<br>Activity Victim Activity Victim<br>Panel B: Assault ‐ activity/victim, % Panel D: Trespassing ‐‐ activity/victim, %<br>45.0 15.0<br>39.1 13.9 14.1<br>40.0<br>34.7 12.5<br>35.0 11.6<br>31.1 9.6<br>30.0 10.0<br>25. 9 26.7<br>25.0 7.1<br>7.5<br>20.0<br>16.6<br>15.0 5.0<br>10.0<br>2.5 1.9<br>5.0<br>0.0 0.0<br>Total Males Females Total Males Females<br>Activity Victim Activity Victim<br><!-- End of picture text --> Source: Authors’ calculations.

19
Alcohol and drugs are also prevalent, though less so for young women. Being a victim of alcohol or drug‐ related activity is reported by 24.6% of the sample overall, and by 30.8% of young men and 13.5% of young women. This gender imbalance is slightly greater for those that abuse alcohol and drugs, as the incidence is 28.1% for males, and only 5.6% for females (see Panel C, Figure 2). As one might expect, this type of activity is closely related to other types of criminal activities. Of all the youth reporting to have been engaged in alcohol‐related activity, 50.7% also report engaging in stealing, and 61.5% report engaging in assault.

Compared with assault, Involvement in trespassing is less common. In the baseline sample, only 11.6% of youth report ever being a victim of trespassing, with 14.1% and 7.1% for men and women, respectively. When it comes to (self‐reported) incidence of involvement in such an activity, the numbers are 13.9% and 1.9% for males and females, respectively (see Panel D, Figure 2).

In summary, these results confirm that the sample is capturing the high prevalence of self‐reported anti‐ social behavior and criminal activity among youth in Port Moresby. Strong gender differences also emerge, with men much more likely to abuse alcohol, and engage in robbery or assault. In this context, we next provide evidence on whether the program succeeded in reducing participants’ exposure to and involvement in anti‐social behavior and crime.

# _5.2 Impacts on social behavior, violence and crime_

This section reports the estimated impacts of the intervention on a variety of outcomes of interest related to social behavior, violence and crime. Each of these outcomes of interest is observed for treatment and control groups both at baseline, prior to the intervention, and in the follow‐up survey (FUS) after the intervention. For both treatment and control groups, we evaluate the changes over time, test the statistical significance of these changes, and calculate the difference‐in‐differences (DD) estimates.

# 5.2.1 Impacts on social behavior/environment

Here we report the results from looking at the social behavior and environment indicators. The program may have improved outcomes for participants by altering the people they interact with. One indicator of this is whether youth report hanging out with friends late at night, which is a strong correlate of anti‐social behavior and crime. We find that for the treatment group, the reported incidence declined from 36.8% at baseline to 29.6% post‐intervention), or 18.7 percentage points (significant at a 5% level). For the control group, the incidence rose slightly from 42.2% to 44.2%, which is not statistically significant. Hence, the Difference in Difference estimate is 8.8 percentage points, significant at the 5% level, which is a decline of 24.2 percentage points relative to the baseline value for the treatment group. These results are reported in Table 5. They are also presented graphically in Figure 4 (Panel A). At baseline, those in treatment group 20
# **Table 5: Estimated Impacts on crime‐related social behavior**

|Variable of interest|**Before**<br>(mean/sd)<br>(1)|**After**<br>(mean/sd)<br>(2)|**Difference**<br>(mean/se)<br>(3)|
|---|---|---|---|
|Hang out with friends late at night(treatments)|0.364|0.296|‐0.068 **|
||_0.482_|_0.457_|_0.030_|
|Hang out with friends late at night(controls)|0.422|0.442|0.020|
||_0.495_|_0.498_|_0.035_|
|Difference‐in‐Differences (DD)|||**_‐0.088_****|
||||_0.036_|
|Best friend involved in crime last 3 months||||
|(treatments)|0.237|0.157|‐0.080 ***|
||_0.426_|_0.364_|_0.027_|
|Best friend involved in crime last 3 months||||
|(controls)|0.269|0.309|0.040|
||_0.444_|_0.463_|_0.034_|
|Difference‐in‐Differences (DD)|||**_‐0.120_*****|
||||_0.050_|
|Gets drunk often (1 or more times per week)||||
|(treatments)|0.442|0.341|‐0.101 **|
||_0.498_|_0.475_|_0.046_|
|Gets drunk often (1 or more times per week)||||
|(controls)|0.476|0.398|‐0.078|
||_0.501_|_0.491_|_0.049_|
|Difference‐in‐Differences (DD)|||**_‐0.023_**|
||||_0.072_|
|Chews bettel nut(treatments)|0.808|0.822|0.015|
||_0.395_|_0.383_|_0.022_|
|Chews bettel nut (controls)|0.823|0.867|0.044|
||_0.382_|_0.360_|_0.027_|
|Difference‐in‐Differences (DD)|||**_‐0.029_**|
||||_0.044_|

Notes: Table reports means and standard deviation (error) for two groups and its difference.

*** p<0.01; ** p<0.05; * p<0.1. The sample size is 338 for treatments and 249 for controls.

21
were slightly more likely to report hanging out at night with their friends, but the difference is not statistically significant. The difference becomes significant (at a 1% level) following the intervention. Although participants report hanging out less with friends at night, it is not clear whether this reflects a change in behavior, a change in friends, or both. To gain a small amount of insight into this, we examine whether participants report that their best friend was involved in crime during the last 3 months. This reflects the aphorism “tell me who your friend is, and I will tell you who you are.” We find that for the treatment group the reported incidence of having a best friend involved in crime last 3 months declined from 23.7% prior to the intervention to 15.7% after intervention, which is statistically significant at the 1% level. For the control group, as was the case for hanging out with friends, the incidence rose slightly from 26.9% to 30.9%, which is not statistically significant. These differences result in a D in D estimate of 12 percentage points, significant at the 1% level. These results are reported in Table 5. They are also presented graphically in Figure 3 (Panel B). We also note that, as for the previous indicator, the means of this variable are not statistically different between groups at baseline, but become different (at a 1% level) following the intervention.

**Figure 3: Impacts on social behavior/environment associated with crime**

<!-- Start of picture text --> Panel A: Hangs out with friends late night, %Panel A: Hang out with friends late night, %<br>65 0.0%<br>42.2% 44.2%<br>50.0%40.0%<br>36.4%42.2% 44.2%<br>40.0% 36.4% 29.6%<br>30.0% 29.6%<br>30.0%<br>20.0%<br>20.0%<br>10.0%<br>10.0% 2.0%<br>2.0%<br>0.0%0.0%<br>Before Before After After ‐6.8% D D<br>‐10.0% ‐6.8%<br>Treatments Controls<br>Panel B: Best friend involved in crime last 3 m., %<br>40.0%<br>30.9%<br>26.9% 30.9%<br>30.0%<br>30.0% 23.7%23.7% 26.9%<br>20.0% 15.7%<br>20.0% 15.7%<br>10.0%<br>4.0%<br>10.0%<br>4.0%<br>0.0%<br>0.0% Before After ‐8.0% D<br>‐10.0%<br>Before After D<br>‐ 12 0.0%<br>‐8.0%<br>Treatments Controls<br><!-- End of picture text --> Source: Authors’ calculations.

22
We next look at the reported incidence of drinking, and, specifically, if a person reports getting drunk often, defined as once or more per week. This binary variable was constructed from a categorical variable with multiple categories indicating how often the person gets drunk. For the treatment group, this indicator declined 10 percentage points, from 44.2% pre‐intervention to 34.1% post‐intervention, which is statistically significant at the 5% level. For the control group, this indicator has also declined, from 47.6% to 39.8% (significant at a 10% level). As there was a decline in incidence for both groups, the D in D value (of 2 percentage points) is not significant (see Table 5). The difference in the mean values of this indicator between the treatments and controls is not significant at either baseline or follow‐up. Similarly, no evidence emerged that the program reduced expenditure on alcohol during the past week, as the change was not statistically significant for both the treatment and control group.<sup>15</sup> Nor was there any estimated impact of the program on betel nut, which is a popular mild stimulant, or the use of marijuana.<sup>16</sup> In sum, the program is associated with reductions in friends hanging out at night and their best friends getting drunk, rather than a reduction in actual drinking and drug use.

# 5.2.2 Impacts on anti‐social behavior/violence

We next look at a number of variables related to anti‐social behavior and violence. The first indicator is whether youth report that their friends have been involved in fights or robbery recently (last 3 months). The results indicate that for the treatment group the incidence has declined from 35.2% pre‐intervention to 22.5% post‐intervention, which is statistically significant at the 1% level. In the control group, for which the baseline level of this indicator is similar, the rate increased slightly from 35.3% to 38.6%, which is not a statistically insignificant increase. The DD value is thus equal to 15.9 percentage points, which is a decline of 45.3% relative to the baseline level for the treatment group. The results are presented in Table 6, and graphically in Figure 4 (Panel A).

We next look at the reported incidence of using threat or force with somebody. We find that in the treatment group this indicator declines from 16% to 7.7% (statistically significant at a 1% level). In the control group, the incidence has increased from 10.8% to 15.3%. Hence, the DD value is 12.7 percentage points (significant at a 1% level). The results are presented in Table 6, and graphically in Figure 4 (Panel B).

> 15 These results are not reported here, but are available from the authors.

> 16 In PNG, chewing a betel nut is a very common habit. Betel nuts are chewed with betel leaf for their effects as a mild stimulant. Chewing the mixture of betel nut and betel leaf is a tradition, custom or ritual which dates back thousands of years in much of the geographical areas from South Asia eastward to the Pacific. It constitutes an important and popular cultural activity in many Asian and Oceanic countries. While in itself it is not considered a bad behavior, there are some negative effects associated with it, such as spitting in public places (for that reason, the Government of PNG has recently imposed a ban on chewing betel nut in public places). More on this can be found at: <u>https://en.wikipedia.org/wiki/Areca_nut.</u>

23
**Table 6: Estimated Impacts on crime‐related social behavior**

|Variable of interest|**Before**<br>(mean/sd)<br>(1)|**After**<br>(mean/sd)<br>(2)|**Difference**<br>(mean/se)<br>(3)||
|---|---|---|---|---|
|Friends involved in fights or robbery(treatments)|0.352|0.225|‐0.127|***|
||0.478|0.418|0.029||
|Friends involved in fights or robbery (controls)|0.353|0.386|0.032||
||0.479|0.488|0.038||
|Difference‐in‐Differences (DD)|||‐0.159|***|
||||0.055||
|Used threat or force with somebody(treatments)|0.160|0.077|‐0.083|***|
||0.367|0.267|0.023||
|Used threat or force with somebody(controls)|0.108|0.153|0.044||
||0.312|0.360|0.024||
|Difference‐in‐Differences (DD)|||‐0.127<br>0.038|***|
|Have been attacked and fought back(treatments)|0.438|0.314|‐0.124|***|
||0.497|0.465|0.032||
|Have been attacked and fought back(controls)|0.426|0.450|0.024||
||0.495|0.498|0.040||
|Difference‐in‐Differences (DD)|||‐0.148|***|
||||0.057||
|Damaged somebody's property for fun/joke|||||
|(treatments)|0.086|0.050|‐0.036|**|
||0.280|0.219|0.018||
|Damaged somebody's property for fun/joke|||||
|(controls)|0.056|0.080|0.024||
||0.231|0.272|0.020||
||||‐0.060<br>0.030|**|

Notes: Table reports means and standard deviation (error) for two groups and its difference.

*** p<0.01; ** p<0.05; * p<0.1. The sample size is 338 for treatments and 249 for controls.

24
**Figure 4: Impacts on aggressive behavior and violence**

<!-- Start of picture text --> Panel A: Friends involved in fights or robbery, %<br>50.0%<br>38.6%<br>40.0% 35.2% 35.3%<br>30.0%<br>22.5%<br>20.0%<br>10.0%<br>3.2%<br>0.0%<br>‐10.0% Before After D<br>‐12.7%<br>‐20.0%<br>Treatments Controls<br>Panel B: Used threat or force with somebody, %<br>25.0%<br>20.0%<br>16.0% 15.3%<br>15.0%<br>10.8%<br>10.0%<br>7.7%<br>4.4%<br>5.0%<br>0.0%<br>Before After D<br>‐5.0%<br>‐10.0%<br>‐8.3%<br>Treatments Controls<br><!-- End of picture text --> Source: Authors’ calculations.

A third indicator related to violence is the reported incidence of being attacked and fighting back. In the treatment group, this indicator declined from 43.8% to 31.4%, which is statistically significant at the 1% level. In the control group, which had a similar baseline level, we find no statistically significant change. The DD value is thus 14.7 percentage points, or a decline of 33.9 percent, significant at a 1% level, relative to the baseline value for the treatment group (see Table 6).

Finally, we also look at the reported incidence of doing damage to somebody’s property for fun or as a joke. In the treatment group, the incidence declined from 8.6% to 5%, which is a decline of 3.6 percentage points, or 41.4 percent (significant at a 5% level). We find no statistically significant change among the controls. The DD value is 6 percentage points, which is a statistically significant (at a 5% level) decline (see Table 6). Overall, unlike the results on social behavior reported in the previous section, the program is associated with reductions in anti‐social behavior and violence both for participants and their friends.

25
# 5.2.3 Impacts on criminal activity

In this section, we look at the reported incidence of involvement in a specific type of criminal activity over the reference period of the last 6 months. We investigate the impact of program participation on various types of criminal activity, which include theft/stealing, both physical and verbal assault, alcohol‐related activity (e.g., fights) and trespassing. These types of activities could be under‐reported by youth, even though the respondents were specifically instructed that their responses to these question have no bearing on the chances of program participation, and that all responses are kept in a strict confidence. If the degree of under‐reporting is independent of program participation, however, differences between the treatment and controls over time can be measured reliably.

The results indicate little association between the program and the incidence of theft or assault. The incidence of theft/stealing among participants declined from 19.5% to 11%, which was significant at the 1% level (Table 7 and Figure 5, Panel A). Meanwhile, the corresponding incidence declined less for the controls, from 24% to 18.8%, which was not statistically significant. The D in D estimate is therefore a 3.2 percentage point effect, which is not statistically significant. The results for physical and verbal assault, which are reported in Table 7, and graphically in Figure 6 (Panel B), were similar. Participants saw a statistically significant decline from 29.7 to 23.5%, but the control group experienced a similar decline from 40 to 33.6 percent. Therefore, the resulting D in D estimate is close to zero (0.3 percentage points) and not statistically significant.

## F **igure 5: Impacts on specific criminal activities**

<!-- Start of picture text --> Panel A: Involved in stealing, %<br>40.0%<br>30.0%<br>24.0%<br>19.5% 18.8%<br>20.0%<br>11.0%<br>10.0%<br>0.0%<br>Before After D<br>‐10.0% ‐5.2%<br>‐8.4%<br>‐20.0%<br>Treatments Controls<br>Panel B: Involved in assault (verbal or physical), %<br>50.0%<br>40.0%<br>40.0%<br>29.7% 33.6%<br>30.0%<br>23.5%<br>20.0%<br>10.0%<br>0.0%<br>Before After D<br>‐10.0% ‐6.1% ‐6.4%<br>Treatments Controls<br><!-- End of picture text --> Source: Authors’ calculations.

26
# F **igure 5 (continued): Impacts on specific criminal activities**

<!-- Start of picture text --> Panel C: Involved in alcohol‐related crime, %<br>40.0%<br>30.0% 26.0%<br>20.9% 22.4%<br>20.0%<br>11.3%<br>10.0%<br>0.0%<br>Before After D ‐3.6%<br>‐10.0%<br>‐9.6%<br>‐20.0%<br>Treatments Controls<br><!-- End of picture text --> <!-- Start of picture text --> Panel D: Involved in trespassing, %<br>30.0%<br>25.0%<br>18.8%<br>20.0%<br>15.0%<br>11.6% 8.4%<br>10.0%<br>5.2%<br>5.0%<br>0.0%<br>‐5.0% Before After D<br>‐10.0% ‐6.4%<br>‐15.0% ‐10.4%<br>Treatments Controls<br><!-- End of picture text --> Source: Authors’ calculations.

The estimated effects on alcohol‐related crime, which potentially includes stealing and assault as well as fights and damage to property, are larger but remain statistically insignificant (Table 7, and Figure 5 Panel C). For the treatment group, incidence declined substantially – from 20.9% to 11.3%, which is statistically significant at the 1% level. For the control group, the incidence declined from 26% to 22.4%, which is not statistically significant. The difference in difference estimate is therefore 6 percentage points, but due to the small sample size is not statistically significant. Nonetheless, the larger effect on alcohol related crime is in line with the large estimated effect of the program on the incidence of getting drunk discussed above.

When it comes to trespassing, we find a statistically significant decline for both participants and the control group. The results are presented in Table 7, and graphically in Figure 5 (Panel D). The control group saw a slightly greater decline than program participants, by about 4 percentage points, but the difference is not statistically significant. Nonetheless, the estimates are sufficiently precise to rule out large beneficial program effect on this indicator.

27
**<mark>Table 7: Impacts on specific criminal activities</mark>**

|Variable of interest|**Before**<br>(mean/sd)<br>(1)|**After**<br>(mean/sd)<br>(2)|**Difference**<br>(mean/se)<br>(3)||
|---|---|---|---|---|
|Involved in stealing last 6 months(treatments)|0.195|0.110|‐0.084|***|
||_0.397_|_0.314_|_0.023_||
|Involved in stealing last 6 months(controls)|0.240|0.188|‐0.052||
||_0.428_|_0.391_|_0.028_||
|Difference‐in‐Differences (DD)|||**_‐0.032_**<br>_0.045_||
|Involved in assault (physical or verbal) last 6 months|||||
|(treatments)|0.297|0.235|‐0.061|**|
||_0.457_|_0.425_|_0.029_||
|Involved in assault (physical or verbal) last 6 months|||||
|(controls)|0.400|0.336|‐0.064||
||_0.491_|_0.473_|_0.037_||
|Difference‐in‐Differences (DD)|||**_0.003_**||
||||_0.054_||
|Involved in alcohol‐related crime last 6 months|||||
|(treatments)|0.209|0.113|‐0.096|***|
||_0.407_|_0.318_|_0.027_||
|Involved in alcohol‐related crime last 6 months|||||
|(controls)|0.260|0.224|‐0.036||
||_0.440_|_0.418_|_0.033_||
|Difference‐in‐Differences (DD)|||**_‐0.060_**||
||||_0.046_||
|Involved in trespassing last 6 months(treatments)|0.116|0.052|‐0.064|***|
||_0.321_|_0.223_|_0.019_||
|Involved in trespassing last 6 months(controls)|0.188|0.084|‐0.104|***|
||_0.391_|_0.278_|_0.027_||
|Difference‐in‐Differences (DD)|||**_0.040_**||
||||_0.036_||

Notes: Table reports means and standard deviation (error) for two groups and its difference.

*** p<0.01; ** p<0.05; * p<0.1. The sample size is 338 for treatments and 249 for controls.

28
Finally, to complement the evidence on the self‐reported incidence of engagement in anti‐social behavior and/or criminal activity, we examine the reported incidence of being a crime victim. The respondent was asked if he/she was a victim of stealing, assault, alcohol‐related crimes, or trespassing during the last 6 months. The only statistically significant result is for trespassing, in which the program is estimated to increase the probability of being a victim of trespassing, by about 7 percentage points. While this result is somewhat difficult to explain, it could result from participants moving to areas where trespassing is more likely to occur. With regard to the other three crimes, the estimated difference in difference effects are all negative but not statistically insignificant. However, for alcohol and assault, the 95 percent confidence intervals range from approximately a 16 percentage point reduction to no reduction, meaning that we cannot rule out substantial effects.

Overall, in contrast to the estimated effects on anti‐social behavior, there is little robust evidence that the program reduced participants’ in or exposure to crime. Besides the estimated effect on vandalism reported above, in no case was the estimated effect on crime negative and statistically significant at the 5 percent level. Furthermore, in many cases, the point estimates tend to be close to zero. However, the estimates are sufficiently imprecise that we cannot rule out the possibility that the program had substantial effects on assault or stealing.

# 5.2.4 Effects on Employment and Savings

To better understand these results, it is useful to examine the potential intermediary role of labor market outcomes. In the Follow‐up Survey, the treatment group was asked if they had a wage employment after the program participation. We find that 43.6% of program participants report having a job after the program participation, with 49% and 39.2% for OJT (internship) participants and YJC (public works) participants, respectively. Unfortunately, the same question cannot be asked for the control group. However, the control group has been asked about having a wage employment in the last 6 months, and we find that only 15.1% report to have been employed. This provides some indication that the rate of wage employment in the treatment group is substantially higher. Interestingly, 40.8% of employed OJT participants and 32.4% of employed YJC participants report staying with the same employer after graduation from the program. Furthermore, employed treatments and controls report similar average working hours per week, of 43.6 and 44 hours, respectively. Hence, it appears that the main difference between treatments and controls may be coming from the extensive margin through employment rates, rather than through the intensive margin for those employed.

Besides raising employment rates, the program also appears to have had beneficial effects on job search behavior. To look at job search behavior, the sample is restricted to youth that report to being unemployed in both periods. In the treatment group, the share of youth seeking wage employment in the last 3 months increased from 21.1% to 32.6%, or 11.5 percentage points (this change is significant at a 1% level). In the control group, the share of job searching unemployed declined from 21.3% to 18.9%, which statistically insignificant.<sup>17</sup> Program participants are much more likely than controls to report that they

> 17 The results on job search are not reported here for brevity, but are available from the authors upon request.

29
have been helped by other in their job search ‐‐ about 50% of treatments vs. 28% of controls, respectively. Finally, the increased job search activity among unemployed graduates reflects their own belief that the program made them more employable. 76% of OJT graduates and 64% of YJC graduates reported that they believed that participation in a program has increase their employability. Focus group discussions with program participants highlighted the beneficial effects of program participation on expanding their social networks and exposure to potential jobs.

In addition to improving employment outcomes, the project also provided access to the formal financial system, which could lessen the demand for economically motivated crime. In particular, the program set up a bank account for each participant to receive stipends. Not surprisingly, this resulted in much higher share of youth with bank accounts in the treatment group versus the control group. While both groups started with similar levels of bank account ownership, at the time of the follow‐up survey 70.9% of treatments had a bank account versus 23.1% of controls. During the program, access to bank accounts was almost universal among the treatment group, but some youth decided to close their bank accounts after graduation from the program due to the associated bank fees.

Overall, the program appears to have had strong effects on employment, job search, and access to bank accounts. Employment rates appear to be substantially higher for the treatment then for the control groups, although the follow‐up question on employment was not directly comparable. Program graduates that were not employed were much more likely to be actively searching for jobs and working with others to do so, and were much more likely to have a bank account. Employment, or even greater expectation that searching for work could pay off, both reduces the economic need and raises the opportunity cost of crime. This evidence is consistent with a program that reduced anti‐social behavior for participants that were less likely to commit crimes, while having smaller effects on the behavior of those engaged in crime.

The results reported up to now have been based on simple difference‐in‐difference estimates that compare treatments and controls. As a robustness check, we also examine estimates obtained using a slightly different method, in which outcomes during the final period are regressed on a set of baseline variables. The results are reported in Table 8. In general, controlling for baseline characteristics has minor effects on the estimates, and make the estimates only slightly more precise. However, the estimated effect of the program on gratuitous property damage (damaging someone’s property for fun or joke) declined slightly, making the estimate no longer statistically significant at the 5 percent level. This further reinforces the notion that the program’s strong effects on anti‐social behavior did not necessarily translate into as strong a reduction in criminal activity.

30
_Table 8: Estimated impacts with and without baseline controls_

|Variable of interest|**Impact (DD),**<br>**with no baseline**<br>**controls**<br>(mean/se)|**Impact (DD),**<br>**with baseline**<br>**controls**<br>(mean/se)|
|---|---|---|
||(1)|(2)|
|_A: Social behavior associated with crime_|||
|Hang out with friends late at night|‐0.088 **|‐0.102 **|
||_0.036_|_0.052_|
|Best friend involved in crime last 3 months|‐0.120 ***|‐0.124 ***|
||_0.050_|_0.047_|
|Gets drunk often (1 or more times per week)|‐0.023|‐0.034|
||_0.072_|_0.071_|
|Chews betel nut|‐0.030|‐0.036|
||_0.044_|_0.045_|
|_B: Aggressive behavior/violence_|||
|Friends involved in fights or robbery|‐0.159 ***<br>_0.055_|‐0.169 ***<br>_0.051_|
|Used threat or force with somebody|‐0.127 ***|‐0.122 ***|
||_0.038_|_0.038_|
|Have been attacked and fought back|‐0.148 ***|‐0.142 ***|
||_0.057_|_0.055_|
|Damaged somebody's property for fun/joke|‐0.060 **|‐0.056 *|
||_0.030_|_0.030_|
|_C: Specific criminal activities (last 6 months)_|||
|Involved in stealing|‐0.032|‐0.020|
||_0.045_|_0.042_|
|Involved in assault (physical or verbal)|0.003|0.020|
||_0.054_|_0.053_|
|Involved in alcohol‐related crime|‐0.060|‐0.048|
||_0.046_|_0.045_|
|Involved in trespassing|0.040|0.040|
||_0.036_|_0.035_|
|_D: Being a victim of specific criminal_<br>_activities (last 6 months)_|||
|Victim of stealing|‐0.020|‐0.028|
||_0.057_|_0.057_|

31
|Victim of assault (physical or verbal)|‐0.063|‐0.062|
|---|---|---|
||_0.057_|_0.056_|
|Victim of alcohol‐related crime|‐0.064|‐0.049|
||_0.048_|_0.047_|
|Victim of trespassing|0.069|*<br>0.083 **|
||_0.039_|_0.039_|

Notes: Baseline controls include age, gender, ethnicity, marital status, education, HH size; living conditions; individual employment status (if ever had a wage job, if informally employed). *** p<0.01; ** p<0.05; * p<0.1.The sample size is 338 for treatments and 249 for controls.

# 6. **Conclusions**

The main objective of this analysis is to evaluate the extent to which the program achieved the government’s goals of reducing youth crime and anti‐social behavior, and to suggest possible channels through which impacts may have occurred. This is the first such evaluation in the context of Port Moresby, Papua New Guinea, which is characterized by extremely high levels of crime and violence. In addition to providing empirical evidence in this unique context, this paper also contributes to the very scant evidence base on the impact of training and temporary employment schemes on crime and social behavior in a context of a developing country.

The evidence is mixed. Participation in UYEP has sizable impacts on several outcomes related to social behavior and criminal activity. Program participants became much less likely to hang out with friends late at night (a reduction from 36.8% to 29.6%, or 18.7 percent), and more selective in terms of their social circle/friends. The reported incidence of having a best friend involved in crime last 3 months declined from 23.7% to 15.7%, or 33.8 percent. Furthermore, the reported incidence of having friends engaged in robberies fell from 35 percent to 23 percent among participants, as compared with a small rise from 35 to 39 percent among controls.

Reflecting this change in social behavior, we also find moderate declines in the probability engaged in that participants engaged in gratuitous property damage, was involved in alcohol‐related crime, or was a crime victim. These estimated effects are not precise enough to be statistically significant. Nonetheless, they are consistent with program participants being less likely to be hanging around criminal gangs or in the wrong place at the wrong time. The results also indicate a decline in the incidence of specific illegal activities such as assault (physical or verbal) and theft/stealing. However, since we also find a comparable decline in these categories for the control group, the decline may result from youth growing older rather than participating in the program.

What explains these positive findings? The most consistent finding is the project’s large effect on participants’ social networks and circle of friends, which could either result from both direct friendships made through the program itself, or an indirect effect of higher aspirations due to being exposed to work.

32
Consistent with this, the program appears to have led to a strong increase in the probability of employment, which both raises the opportunity cost of anti‐social behavior, and may alter participants’ shared sense of mission with gangs that hang out at night and engage in criminal activity. At the same time, there is no robust evidence that the perpetration of most forms of crime – assault, robbery, and trespassing – were affected by the program. More work is needed to tease out why this may be the case, but the results are consistent with heterogeneous program impacts. The program appears to have helped youth who were least likely to commit crime stay out of trouble. Meanwhile, it may have had weaker effects on youth who were most prone to commit serious crimes, perhaps because they have stronger social ties to gangs. Either economic incentives, or a strong personal identification with the criminal lifestyle, could mitigate the effect of the program among violent offenders.

More research, with a larger data set, would help disentangle these heterogeneous impacts. Now that the program is well‐known and oversubscribed within communities, a future evaluation could revisit the question of whether participation can be randomized. Even with existing data, however, more analysis can also shed light on the design of the program. Data were collected on additional outcomes, such as participants’ mental well‐being and future aspirations, which have yet to be examined. It would also be important to better document heterogeneity in program impacts based on youths’ observable characteristics. This would shed light both on the differential effects of public works and job training on outcomes, and on how effects varied across different segments of the youth population. Nonetheless, the initial results provide important evidence that the program has led to a range of positive social and behavioral outcomes for participating youth, while underlying the need for additional interventions to address many of the root causes of violent crime.

33