metadata: authors: 0: author: Balavac-Orlic,Merima
1: author: Giles,John T.
2: author: Hari,Siddharth
3: author: Ovadiya,Mirey
count: Bosnia and Herzegovina
docna: 0: docna: Private Provisioning of Employment Services
            : Experimental Evidence from Bosnia and Herzegovina
docty: Policy Research Working Paper
theme: Labor Market Institutions,Job Creation,Human Development and Gender,Private Sector Development,Labor Market Policy and Programs,Active Labor Market Programs,Jobs
abstracts: cdata!: This paper examines the effect of
            compensating a private sector provider of employment
            services for successful placement of jobseekers with
            employers.  Within this program, active job matching,
            relative to only job counseling, contributes to a
            significantly higher rate of employment, and these effects
            are more pronounced for subgroups that are less likely to
            find employment without assistance, including youth and
            low-skilled workers. Among the jobseekers randomly allocated
            to matching, those who were judged by counselors to be
            &#x201c;employable&#x201d; were far more likely to
            be matched to employers. To understand the decisions made to
            rate jobseekers as employable, the paper makes use of both
            information directly observable to job counselors and
            characteristics and personality traits from a separate
            survey of the jobseekers. Evidence suggests that under
            incentivized matching, counselor assessments of
            employability and designation for more intensive effort are
            consistent with cream-skimming.
display_title: Private Provisioning of Employment
            Services : Experimental Evidence from Bosnia and Herzegovina
listing_relative_url: /research/2024/06/34350878/private-provisioning-employment-services-experimental-evidence-bosnia-herzegovina
topicv3: Job Placement m1326264 1652,Active Labor Market Policies m1326265 1654,Labor Intermediation m1326948 1651,Training Services m1327430 1653
publishtoextweb_dt: 2024-06-26T00:00:00Z
dois: 10.1596/1813-9450-10826
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
no_of_pages: 37
author_keys: 0: author_key: NA
1: author_key: 450080
2: author_key: 1399396
3: author_key: 124805
model_extractions: Policy Research Working Paper 10826 Private Provisioning of Employment Services Experimental Evidence from Bosnia and Herzegovina _Merima Balavac-Orlic John Giles Siddharth Hari Mirey Ovadiya_ Development Research Group & Social Protection and Jobs Global Practice June 2024
Policy Research Working Paper 10826

# **Abstract**

This paper examines the effect of compensating a private sector provider of employment services for successful placement of jobseekers with employers. Within this program, active job matching, relative to only job counseling, contributes to a significantly higher rate of employment, and these effects are more pronounced for subgroups that are less likely to find employment without assistance, including youth and low-skilled workers. Among the jobseekers randomly allocated to matching, those who were judged by counselors to be “employable” were far more likely to be matched to employers. To understand the decisions made to rate jobseekers as employable, the paper makes use of both information directly observable to job counselors and characteristics and personality traits from a separate survey of the jobseekers. Evidence suggests that under incentivized matching, counselor assessments of employability and designation for more intensive effort are consistent with cream-skimming.

This paper is a product of the Development Research Group, Development Economics and the Social Protection and Jobs Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at merima.balavac@efsa.unsa.ba, jgiles@worldbank.org, shari@worldbank.org, and movadiya@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Private Provisioning of Employment Services: Experimental Evidence from Bosnia and Herzegovina**<sup>*</sup>

Merima Balavac-Orlic,<sup>+</sup> John Giles,<sup>++</sup> Siddharth Hari<sup>⊥</sup> and Mirey Ovadiya<sup>**</sup> Keywords: Active Labor Market Programs, Private Employment Services JEL Classification Codes: **J08, J64, J68** This paper is a product of cooperation between the Development Research Group and the Europe and Central Asia Social Protection and Jobs team at the World Bank. The paper is part of a larger effort by the World Bank to provide open access to its research and to make contributions to development policy discussions around the world. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.

*The intervention and research were supported by grants from the Swedish International Development Cooperation Agency (SIDA) and the Knowledge for Change Program at the World Bank. The authors are grateful for helpful comments from John Earle, Maurice Kugler, Alicia Marguerie, Peter Orazem, Josefina Posadas, Jonathan Stoterau, Francesca Truffa and participants at the DIME KDI Conference and the SoLE session at the 2024 meetings of the Midwest Economics Association conference.

> +University of Sarajevo, Faculty of Economics (merima.balavac@efsa.unsa.ba).

> ++ The World Bank (Research Group) and Institute for the Study of Labor (IZA) (jgiles@worldbank.org).

> ⊥ The World Bank (Social Protection and Jobs, Europe and Central Asia) (shari@worldbank.org).

> ** The World Bank (Social Protection and Jobs, Europe and Central Asia) (movadiya@worldbank.org).
# **1. Introduction**

In several countries across Europe in general, and the Western Balkans in particular, Active Labor Market Programs (ALMP’s) and other forms of support are provided by government-operated public employment services (PES). Across these countries, the PES are under fiscal pressure, even as they are tasked with doing more to assist the long-term unemployed, youth and those with low skills. Outsourcing of employment services to private providers has potential to help address this challenge. Indeed, over the last three decades, OECD countries including Australia, Denmark, France, the Netherlands, the UK and the United States have moved in varying degrees to public funding of privately provided employment services. This paper provides experimental evidence from a pilot program under which employment services were outsourced to a private provider in Bosnia and Herzegovina (BiH).

Supporters of private contracting argue that private providers might plausibly tailor counseling and other assistance to the needs of the more vulnerable and less employable among the unemployed without expanding the civil service workforce. Several studies examine the question of whether outsourcing employment services to the private sector results in improved employment outcomes for jobseekers and whether such an approach is cost-effective. Overall, the literature paints a mixed picture, with some programs, such as Ireland’s _JobPath_ and UK’s _Work Programme_ , showing positive effects on employment outcomes and reduced fiscal burden (OECD, 2022). A review of recent impact evaluations suggests, however, that moving to private sector provisioning may not lead to improved access to employment or savings on public spending (Crepon, 2018).

The nature of the contract offered to private providers also has considerable impact on their performance. Contracts emphasizing recruitment targets for participants may lead to “parking” of jobseekers, where the private provider enrolls many candidates to meet targets and then parks those who are difficult to place (Bredgaard and Larsen, 2008; Davidson, 2010). Alternatively, “cream-skimming,” the selective enrollment or favoring of higher ability job seekers, may occur under contracts that reward successful placement of a target number of jobseekers into employment for a spell lasting at least a specified minimum duration (Koning and Heinrich, 2013). In a careful review of public procurement of employment services, Finn (2011) notes that public purchasers often find themselves redesigning contract parameters to manage both tendencies, as well as other gaming behavior of private providers.

While the existing literature has focused on comparing the relative efficacy of private and public employment service provision in improving employment outcomes and costs, this paper examines the behavior of private employment service providers and the relative impact of different services they may 2
provide, highlighting policy questions which have received less attention. Private employment firms often provide a range of services related to job intermediation, including worker profiling, labor market information, CV and interview preparation, job search assistance, and job matching. When outsourcing intermediation to the private sector, both the relative impact of different services, and in particular, the efforts made in providing these services across a heterogeneous pool of job seekers are important to consider when designing incentive schemes.

This paper concentrates on a pilot program in BiH that was narrow in scope and facilitated comparison of employment outcomes across two levels of service provided by a private provider. Further, access to information on counseling sessions and job counselor assessments make it possible to examine both the classification of jobseekers and counselor decisions. Under the pilot, which was rolled out in three calls between August 2020 and April 2021, self-selected participants were randomly allocated to one of two schemes administered by a private sector provider. A base scheme includes many features of interventions highlighted in recent research, including assistance with CVs, providing information about potential occupations (e.g., Abebe et al, 2021; Beam, 2016; Jensen, 2012), and developing job search plans (e.g., Abel, 2019). The second, more intensive scheme, included all features of the base scheme, with additional effort to match jobseekers with potential employers.

Access to job matching relative to only job counseling led to significantly higher rates of employment in the short-run, with smaller effects in the medium-run. Analysis of administrative data reveals that 3 to 6 months after the program, jobseekers assigned to both counseling and job matching services were 5.8 percentage points, or 25.2 percent, more likely to be formally employed than those assigned to only counseling services. In the medium term, 21 to 24 months after the program, this effect is 4.1 percentage points or a 22.6 percent greater likelihood of being formally employed. Moreover, from examining heterogeneity in the impact of assignment to matching, we find that the impact of matching is driven by a larger magnitude effect for those jobseekers who are less likely to find employment without assistance or may be less capable of signaling their ability without a match from the employment services firm, which acts essentially as a referral.<sup>1</sup> Specifically, we find that treatment effects are much larger for younger workers (under the age of 30) and workers with a secondary school degree or less. However, we find no

> 1One of the main methods used by jobseekers to find work across a host of countries continues to entail either references or introductions from family or friends (a social network), or through direct contact with employers (Carranza and McKenzie, 2024). Jobseekers are left at a disadvantage in job search if they have less access to a referral network of skilled peers or lack other means of finding potential employers and signaling their ability.

3
effects for the long-term unemployed, suggesting that a job-matching intervention, as implemented in this program, may not be sufficient to improve their employment outcomes.

Turning to the role of job counselors working for the private provider, those jobseekers randomly assigned to job-matching services were provided with one of two distinct forms of matching. Some jobseekers were allocated to “specific matching,” in which they were offered a unique introduction to a firm for a position that was available. Alternatively, other job seekers were assigned to “general matching,” in which their CVs were included among a stack of CVs provided to employers. As designed, the project envisioned that assignment to matching would be characterized by “specific matching,” but the private provider was granted considerable flexibility in the implementation of job matching. A review of the job counselors’ notes suggests that the firm’s response to both the contract design and time constraints helped to inform the counselor’s decision over how to allocate workers across these two approaches to matching. Under their contract, the firm was given a target of helping 500 jobseekers to find sustained employment, with part of the payment contingent on program beneficiaries either finding or being placed in formal sector jobs and still employed after three months.

The job counselor’s assessment of a jobseeker’s _employability_ was an important determinant of the modality of matching provided to jobseekers and provides suggestive evidence of how cream skimming behavior is introduced into the job-matching process. Based on an initial interaction with the jobseeker, counselors assessed their employability: each jobseeker was assigned to one of three categories – _less employable, employable_ and _highly employable_ . We find that after controlling for skill, age, gender, and geographic location, jobseekers judged to be _“employable”_ by counselors were 6 percentage points more likely to be provided specific matching services (where they were referred to specific positions with employers) relative to jobseekers assessed as being _less employable_ . Not surprisingly, the overall positive effect of the job matching treatment is driven by those jobseekers assigned to specific matching. Similarly, jobseekers rated _highly employable_ were also more likely to receive matching services.

Examining heterogeneity of treatment effects across employability assessments, the analysis highlights one potential promise of using private sector providers. Treatment effects were positive for jobseekers rated _less employable_ even though they were significantly less likely to be assigned to specific matching, whereas they were zero and, in some cases, negative for those rated _highly employable._ One plausible explanation lies with the possibility that those initially rated as _less employable_ displayed characteristics unobservable to the econometrician during subsequent counseling activities and such information was used to assign jobseekers to specific matching, yielding favorable employment outcomes.

4
Further, the counselors have information on specific needs (demand) of firms and a fit may have arisen for a jobseeker of low skill who was otherwise judged to be less employable. For those rated _highly employable_ , and who had a higher likelihood of being referred to specific matching, treatment effects were close to zero as even in the control group these individuals managed to find employment.

Given their importance for assignment to specific matching, the paper next examines the correlations of worker characteristics with employability ratings from job counselors. Among characteristics observable by jobs counselors, jobseekers over age 30 are more likely to be judged employable, which may be a proxy for prior work experience, and those with less than high school education and those unemployed for more than a year, are judged to be less employable. Gender and characteristics including household size and income are uncorrelated with employability assessments. Exploiting data from a separate survey highlights the importance that personality traits may play in the counseling process. Jobseekers who are extraverted and open to new experiences are viewed to be more employable. While some types of jobs, particularly those in the service sector, may require workers who are more extraverted, attention to how personality traits of jobseekers influence the ratings of job counselors may be useful when considering contracts with the private sector in the future. Not all positions involve facing customers, even in the service sector, and thus will not necessarily require workers to exhibit high levels of extraversion and openness to new experiences.

This paper makes two main contributions to the literature on outsourcing private employment services. First, it examines the relative impact of the services provided by a private employment service provider and finds that job matching in addition to counseling services can have significant positive impacts on employment outcomes, particularly for youth and low-skilled workers who may struggle to otherwise signal their skills in the labor market. Second, it opens the “black box” of the jobseekercounselor relationship to better understand the behavior and incentives of a private employment service provider. Counselors act upon their assessments of jobseeker employability, and jobseekers who are assessed to have limited employment prospects are less likely to receive costly matching services, in linewith a cream-skimming behavior on the part of the service provider.

The rest of the paper is organized as follows. Section 2 provides an overview of the labor market in Bosnia and Herzegovina, as well as a description of the program and evaluation design. Section 3 describes the data sources used for the evaluation. Section 4 lays out the empirical approach used in the paper, and section 5 describes the results of the analysis. Section 6 concludes with a discussion of key findings and their policy implications.

5
# **2. Context, Program Details and Evaluation Design**

## **2.1 Labor Markets in Bosnia and Herzegovina**

Over the last twenty years, the labor market in BiH has been characterized by high rates of unemployment, though overall unemployment saw a sharp reduction since 2015, including in the aftermath of the COVID crisis. However, vulnerable groups including the long-term unemployed, youth and women continue to see higher rates of unemployment, driven in part by lack of relevant skills, limited work experience and difficulties in searching for jobs and signaling their abilities.<sup>2</sup> Youth in BiH face poor prospects in the labor market: young people aged 15-24 had an unemployment rate of 36% in 2022 (and even higher rates earlier in the 2010s). Even among the employed, job opportunities are poor, and a high share of youth work in part-time and temporary positions concentrated in sectors like tourism and trade, which are vulnerable to shocks, leading to periodic spikes in youth unemployment. The percentage of young adults “not in employment, education or training,” or NEETs, in 2021 was almost twice as high as in the EU. Unlike the EU, BiH youth seldom work while in school, only 1% as compared to over 15% in the EU, which also inhibits the transition to work.

The gender employment gap in the country is also pronounced, with unemployment rates much higher among women than men. In terms of the gender employment gap, BiH is classified as a "high gap" country. In 2021, the registered unemployment rate among women in the Federation of Bosnia and Herzegovina (FB&H) was 17 percentage points higher than that among men. Additionally, even women who are highly educated are unemployed at higher rates: in 2021, 63% of highly educated unemployed individuals were women.

## **2.2 Program Details**

The objective of the program was to provide access to a private employment service provider to the vulnerable among the unemployed population. The target group included women, low-skilled jobseekers and the long-term unemployed. The onset of COVID-19 in March of 2020 also created a pool of newly unemployed workers, and there was a strong public interest in helping this group as well. Therefore, the program focused on four groups of unemployed jobseekers: (i) women; (ii) the long-term unemployed; (iii) youth; and (iv) those newly unemployed due to COVID-19.

> 2Unemployment early in one’s adult life is associated with financial distress and skills depreciation in the short term (Bell et al, 1999), and reduced lifetime earnings prospects and fewer opportunities over the long-term (Bell and Blanchflower, 2011). Further, long periods of unemployment adversely affect health and wellbeing outcomes (Lindo, 2011; Rege et al., 2011), and scarring effects on mental health often show significant persistence over the life course (Strandh et al, 2014).

6
The program was implemented by an accredited local private intermediation agency (hereafter referred to as the private provider) which is an affiliate of the largest jobs portal in BiH. In operation since early 2000, the portal hosts vacancies and provides recruitment and other HR services to employers. The company was chosen to implement the pilot project through a competitive procurement process, and its selection was based on its experience, capacity, proven track record in provision of employment services, a good understanding of the BiH labor market and access to an extensive network of employers. Even though the private provider did not have prior experience working with the less skilled and socially vulnerable populations targeted by the program, the capacity of its staff and its knowledge of the labor market were assessed to be critical assets and ingredients for program success.

Participants were screened for eligibility based on their response to public calls issued by the private service provider. Three rounds of public calls for participation were issued between August 2020 and April 2021, informing potential jobseekers about the program, and setting out the eligibility criteria. These included a requirement to be registered with the PES and to belong to one of the project’s target groups, as described above. A total 4,209 applications were received across the three rounds of which 2,592 (62 percent) were eligible to participate in the program. The program covered 11 cantons of the Federation of Bosnia and Herzegovina (BiH), with only 6 percent of beneficiaries coming from Republika Srpska.

Beneficiaries received a range of services from the service provider. First, counselors conducted a profiling interview with each jobseeker to understand their education levels, skills, employment history, job preferences and constraints. A total of 1,576 eligible beneficiaries were profiled.<sup>3</sup> Based on interviews with counselors, an Individual Action Plan (IAP) was developed for each jobseeker outlining the kinds of support that would be most useful for them. One type of support was job counseling services, which included help with preparing a jobseeker’s CV and interview preparation, job search assistance (such as information on potential sectors and employers based on the jobseeker’s profile), and work readiness training. In some cases, where there was pre-identified employer demand and interest, jobseekers were also referred to on-the-job training. Finally, the service provider also delivered dedicated job matching services, activities which, in practice, included both “general” and “specific” matching. Notably this distinction of matching services to be provided was developed by the private provider. The main distinction between the two is as follows: for general matching the service provider shared a list of all

> 3Jobs counselors indicated that most eligible beneficiaries who were not profiled lost interest in the program for one of two reasons: (1) they found employment on their own between the time that they expressed interest in the service and the scheduled interview with a job counselor; or (2) they did not follow through and schedule a session to develop an individual action plan.

7
suitable jobseekers with firms they had business relations (with or without a specific request from the firm). Specific matching involved pre-selecting candidates and matching them with employers in response to specific needs of a firm. As part of the matching services, the private employment service provider also recorded interactions with employers including whether the employer requested CVs of certain jobseekers and whether they followed-up with the jobseeker based on submission of a CV. Table 1 provides an overview of the services provided under the program.

Payments to the private employment service provider were based on performance. The firm received a payment for every IAP developed based on the profiling interview. An additional payment was made for each beneficiary who secured a formal job, whether through their own search or through a match secured by the provider. To independently verify employment of the beneficiary, administrative data from the Federal Employment Institute (FEI) was used. For the purposes of payment, a jobseeker is considered formally employed if s/he appeared in the FEI employment database for at least three consecutive months (for details see section 4 describing the data sources below).

# **2.3 Impact Evaluation Design**

The impact evaluation focused on studying the relative effectiveness of the different services offered by the private employment service using a randomized design. A secondary objective was to open the “black box” to examine decisions of job counselors **.** While most of the existing literature focuses on comparing private providers to the PES, the present study evaluates the relative efficacy of different services provided by the same (private) provider, and to better understand the (typically unobservable) priorities and decisions made by the private provider. To do this, the research team first stratified program beneficiaries by gender and the length of unemployment.<sup>4</sup> Each stratum was then randomly divided into two arms. The private provider was given the names of jobseekers to allocate to a treatment arm which received more intensive services that included both job counseling (CV and interview preparation, work readiness training and information about potential employers) as well as job matching services. It is important to note that the firm was free to design the specific forms of matching services and to whom they were provided (and indeed, as will be seen only one-fifth of jobseekers in treatment ended up receiving the more intensive specific-matching). Further, matching services were not required to be

> 4 There are four strata based on gender and whether an individual is long-term unemployed or not. Jobseekers who had been unemployed for more than 12 months were defined as long-term unemployed.

8
identical across all jobseekers assigned to the treatment group, and the categorization of jobseekers into specific and general matching was one result of this flexibility.

Beneficiaries in the control group received job counseling services but did not have access to job matching. These services are similar in content to those provided by the public employment services. The sample for the analysis included a total of 1,541 jobseekers including 765 jobseekers in the treatment group receiving job matching services and 776 in the control group receiving only basic counseling services. A downside to this design, it is important to note, is that this study does not have a “pure” control group – i.e., a group receiving services only from the PES. As a result of this, the paper cannot compare the efficacy of a private employment service provider relative to the PES.

# **3. Data Sources**

The analysis made use of three sources of data which are described below.

## **3.1 Administrative Data from Federal Employment Institute (FEI)**

Data from FEI provided two strands of information. The first was information on the individual characteristics of program participants including gender, age, education level and length of unemployment. An individual is classified as unemployed if they are registered as such with the FEI, an approach followed by Balavac-Orlic and Posadas (2023). In some sense, this is a less strict definition relative to ILOs criteria, which additionally requires active job-search efforts and readiness to start a job to be classified as unemployed. As benefits for the unemployed are only available upon registration with the FEI, we view it as unlikely that large shares of those out of work are not registering. The FEI also provided information on the formal employment status of jobseekers after their engagement with the private provider. To do this, the FEI drew on a database maintained by the Tax Administration. As described above, payment to the private employment service provider was based on independent verification of a jobseekers’ formal employment status. Whenever the private employment service submitted a payment request for a successful placement, FEI would independently verify the employment status of the jobseeker. This typically happened 3-6 months after the jobseeker started receiving program services and is used in this analysis as an indicator of employment in the short-term. In March 2023 (2125 months after program services were first delivered), another round of employment status verification was conducted which is used as a measure of medium-term employment outcomes. These data are available for the full sample of beneficiaries and includes 1,541 observations.

9
# **3.2 Data from the Private Employment Service Provider**

The private employment service provider collects two sets of data on program participants. The first set contains information regarding the participants’ work experience, skills, and potential barriers to employment. These data are gathered during the profiling interview whereby details on formal education, main and alternative occupations, and self-identified obstacles related to these occupations are recorded. Additionally, information on other skills, such as foreign language proficiency and possession of a driving license, is also collected. Based on the information gathered during the profiling interview, the private provider’s counselors categorize beneficiaries as "less employable," "employable," or "highly employable", respectively. This categorization is based on the counselor’s assessment of the relevance of jobseekers’ skills (technical and soft skills), the relevance of their education to their preferred occupation, and prevailing labor market conditions.

The second set of data includes complete administrative records of the services provided by the counselors to each participant. These activities include conducting a profiling interview, developing an individual action plan, counseling, monitoring, and (depending on the jobseeker) matching. Data on services provided is available for 1,496 jobseekers. However, data on employability assessments is available for 1,399 individuals (or 93.5 percent of the sample). There is no differential attrition between treatment and control groups.

# **3.3 Survey Data**

In addition to the two sources of administrative data, a baseline and end-line survey of program participants was conducted. These surveys collected detailed information on the jobseeker and their household and included modules on: (i) Household demographics and socioeconomic status; (ii) jobseeker’s employment history; (iii) experience with the PES and private employment service providers; and (iv) a battery of questions to characterize each of the jobseeker’s “Big 5” personality traits.<sup>5</sup> The baseline survey had a response rate of 59.3 percent (914 out of 1,541 beneficiaries participated in the survey). There was no difference in response rates across the two treatment groups, likely because the survey was administered before any services were provided by the private employment service provider. Table 2 below shows the timelines of the program and impact evaluation related activities.

> 5Given the relatively high rates of refusal to participate in the end-line survey, we have not used these data for analysis of the impact evaluation. As there were no systematic differences across treatment and control groups, analysis of the endline survey could be included as an additional robustness check.

10
# **4. Empirical Strategy**

Given the random assignment of jobseekers, a comparison of employment outcomes between the treatment and control groups provides a causal estimate of the impact of allocation to matching relative to counseling services alone. To further improve the comparability, the analysis also controls for demographic characteristics of jobseekers (age, education, length of unemployment), location, and time (the public call in which the jobseeker participated).

Specifically, regressions of the following form are estimated: where i i refers to an outcome of interest (for example, formal employment) for individual _i_ living in location _l_ participating in public call _c_ and belonging to strata _s._ In some models, additional individual level YY controls, , may be included. Depending on data used (administrative exclusively, or inclusion of survey data) these additional controls include age and education, but in some models, also household XX demographics, and indicators of behavioral skills. Fixed effects for location ( ) and public call ( ) control for fixed features of the local labor market and macroeconomic conditions at the time of the call.<sup>6</sup> is an LL CC idiosyncratic error term. Given individual level randomization, we use robust standard errors without ii εε clustering (Abadie et al 2023).

We examine treatment effect heterogeneity along five dimensions – gender, age, education, length of unemployment and jobseeker employability (as assessed by the counselor). To do this we include an interaction term between the dimension of interest and the binary treatment variable.

To better understand the behavior of the private employment service provider, we estimate two additional sets of regressions. First, we regress access to specific matching on jobseekers’ employability levels through a linear probability model. Second, we estimate an ordered logit model with the employability assessment as the dependent variable, regressed on a large set of jobseeker demographic and household characteristics as well as “Big-5” personality traits.

> 6 The 11 cantons were grouped into 6 labor markets based on geographic proximity.

11
# **5. Results**

## **5.1 Summary Statistics and Balance**

We start by providing an overview of the study sample and examining balance across the treatment and control groups in table 3. Panel A of Table 3 focuses on demographic information on jobseekers from the FEI database. Overall, in line with the objectives of the program, nearly 80 percent of beneficiaries were women, more than half were above the age of 30, more than 70 percent had a secondary school education or less, and 43 percent had been unemployed for over a year. As can be seen from the last column of this table, there are no differences in the average characteristics across control and treatment groups.

Panel B examines jobseeker characteristics collected through the baseline survey. Roughly half the sample is married, and the average household size is 3.36. About 60 percent of beneficiaries live in households with monthly income below 900 KM. For comparison, the average household monthly consumption in the country in 2015/16 was 1,419 KM and in 2021/22 was 1,778 showing that program beneficiaries came from relatively poor households. Of the beneficiaries 29 percent had some vocational training. Once again, there are no significant differences between the control and treatment groups for any of these characteristics. Finally, the baseline survey included a standard battery of questions to characterize each of the jobseeker’s “Big 5” personality traits - openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism. The table reports standardized averages of each of these traits and again the treatment and control groups are (on average) identical. Panel C shows the distribution of the job counselor’s assessment of applicant employability. Of the jobseekers, 78 percent are categorized as “ _employable_ ”, whereas 11 percent each are categorized as “ _highly employable_ ” and “ _less employable_ ”. Again, these distributions are identical across the treatment and control arms.

## **5.2 Differences in Services Provided to Treatment and Control Groups**

Next, we establish that the experimental design was adhered to by looking at differences in services provided across the treatment and control groups. By design, all jobseekers should have had access to similar counseling services, and as can be seen in Table 4 this was indeed the case with their being no differences between the treatment and control groups **.** Profiling was completed and IAP’s developed for more than 96 percent of our sample (columns 1 and 2). Column 3 shows that there were no differences in counseling services received across the two groups.

12
Table 4 also shows that matching services were made available almost exclusively to jobseekers in the treatment group, in line with the experimental design. More than 80% of jobseekers in treatment benefited from general matching. The corresponding number for the control group was 7.5 percent (column 4). The reason for this spillover was that during the design it was agreed that while matching services would be restricted to the treatment group, if any jobseekers from the control group became aware about and requested these services, they would be provided to them as well. Specific matching (which involved pre-selecting candidates and matching them with employers in response to specific needs of the firm) was provided to 23 percent of jobseekers from the treatment group and only a single jobseeker from the control group (column 4).

# **5.3. Main Treatment Effects and Heterogeneity**

The overall intention-to-treat effects of assignment to matching services on short- and medium-run employment outcomes are presented in Table 5. The first two columns include controls from administrative data, whereas the last two include additional control variables collected using the baseline survey (and therefore only include individuals who responded to the survey). Matching services increased the likelihood of formal employment by 5.8-8.8 percentage points (pp) in the short run. The effects are smaller over the medium run but remain positive and significant at 4.1-6.9pp. Given that 22.9 percent and 17.9 percent of jobseekers in the control group were employed in the short and medium run respectively, these effects translate to an approximately 25 percent increase in employment probability in the shortrun, and 22.6 percent over the medium-run.

Next, we examine heterogeneity of the treatment effects along five dimensions – gender, age, level of education, length of unemployment and _“employability”_ level (as determined by the job counselor). Table 6 shows that the treatment effects in the short and medium run are large, positive, and significant for men at around 12.9pp. For women, the treatment effects, while positive are much smaller in magnitude and statistically indistinguishable from zero.

Turning to heterogeneity by age in Table 7, we find that access to matching had large effects for jobseekers under the age of 30 with treatment effects of 9.3pp and 7.8pp in the short and medium run respectively. For older jobseekers, the effects were much smaller and statistically indistinguishable from zero. Similarly, Table 8 shows that access to additional matching services increased the likelihood of 13
formal employment by 6.9 pp (4.5 pp in the medium term) for jobseekers with lower levels of education. The effect for more educated jobseekers is small and statistically insignificant. These results are consistent with the recent literature which has documented that younger jobseekers and those with low levels of education find it harder to signal their skills in the labor market. Matching addresses this constraint by screening and providing direct connections with employers. We also find that the treatment effects are stronger at 7.3 pp in the short-term (5.3 pp in the medium term) for jobseekers who have been unemployed for shorter periods of time (see Table 9). Matching has a statistically null effect on the longterm unemployed (defined as having been unemployed for over a year).

As described above, counselors also rated jobseekers on their employability with each jobseeker rated as either “less employable”, _“employable”_ or _“highly employable”_ . In Table 10, we explore heterogeneity of treatment effects along this dimension. Interestingly, treatment effects are positive and significant for jobseekers rated _“employable”_ (6.4pp in the short-run and 5.1pp in the medium-run). On the other hand, treatment effects are positive (thought statistically indistinguishable from zero given the small sample size) for individuals rated _“less employable”,_ and zero or even negative for individuals rated _“highly employable”_ . In both cases, however, the coefficients are statistically indistinguishable from zero, which is likely driven by high standard errors due to the small number of individuals who were rated either _less_ or _highly employable._ A plausible explanation for these observed results is as follows: for those rated _less employable_ , there are unobservable characteristics unobservable to the econometrician but observed by the counselors which was used to assign jobseekers to specific matching resulting in positive effects. Further, the counselors have information on specific needs (demand) of firms and a fit may have arisen for a jobseeker of low skill who was otherwise judged to be less employable. For those rated _highly employable_ , even though they had a higher likelihood of being referred to specific matching (see Table 11), treatment effects were close to zero since even in the control group these individuals managed to find employment.

# **5.4 Determinants of Specific Matching and Employability Assessments**

The private employment service provider was free to choose which jobseekers from the treatment group received specific matching services and only 23.3 percent ended up receiving these services. We next analyze the role of the counselor’s rating of a jobseeker’s employability in determining what services were provided. Table 11 presents results from this analysis, with the first column not including any 14
controls (except for location and time fixed effects) and the second column including controls for age, gender, education, and length of unemployment. As can be seen from this table, employability ratings are an important determinant of access to matching services. Being rated _employable_ (relative to _less employable)_ increases the likelihood of receiving matching by 5-6pp. Similarly, being rated _highly employable_ increases the likelihood of receiving matching 5-7.4pp (though given the small sample size, this effect is not statistically significant). Providing matching services is costly for the private employment service provider. Unsurprisingly, therefore, they are more likely to provide these to jobseekers who, in their assessment, are most likely to be formally employed as a result of these services. As was shown previously, despite a lower likelihood of receiving matching, treatment effects are positive for _less employable_ jobseekers, suggesting that the counselors make use of additional information (for example what they learn during the profiling interviews and demand for the specific skills of the jobseekers) to identify those most likely to benefit from matching.

Given the importance of jobseeker employability rating we analyze correlates with the rating in Table 12, which marginal effects predicted from an ordered logit model (coefficient estimates are presented in Appendix Table A.1). The first set of marginal effects (in the first three columns) are based on a model including basic demographic characteristics from administrative data. The second set (middle three columns) adds household size, income and previous employment status as additional variables, and the third set (last three columns) includes the “Big 5” personality traits (measured in the baseline survey). First, we note that, consistent with random assignment, treatment is uncorrelated with employability assessments. Second, while gender has no impact on employability assessments, younger jobseekers, those with lower levels of education and the long-term unemployed are likely to have lower employability assessments. In terms of marginal effects, relative to younger jobseekers, jobseekers above the age of 30 are 5.6pp less likely to be rated _less employable,_ and 5.1pp more likely to be rated _highly employable_ . Similarly, those with less than secondary school education are 7.6pp more likely to be rated _less employable_ and 6.9pp less likely to be rated _highly employable_ relative to jobseekers with more than a secondary school degree. The long term unemployed are 11.7pp more likely to be rated _less employable_ and 10.6pp less likely to be rated highly employable relative to jobseekers who have been unemployed for less than a year. Household characteristics including size, number of children and income are uncorrelated with employability assessments.

15
Turning to personality characteristics, we find that of the big-5 traits only two – extraversion and openness – are positively correlated with employability assessments. A one standard deviation increase in extraversion reduces the likelihood of being rated _less employable_ by 1.8pp and increases the likelihood of being rated highly employable by 1.6pp. Similarly, a one standard deviation increase in openness reduces the likelihood of being rated _less employable_ by 2.4pp and increases the likelihood of being rated highly employable by 2.2pp. It is important to highlight that personality traits were measured using an independent survey, and not by the counselors themselves.

# **6. Conclusion and Policy Implications**

Outsourcing employment intermediation and matching to the private sector offers greater flexibility and can play an important role in improving employment outcomes for vulnerable groups. This paper studied the effects of an intervention where jobseekers were randomly assigned to receive less intensive (profiling and counseling) or more intensive (profiling, counseling, and matching) employment services. The overall results from the evaluation point to significant positive effects of the matching treatment. Twenty-three percent of jobseekers in the control group who received less-intensive services found employment, and access to matching services in addition to counseling resulted in 5.8 percentage point (25.2 percent) higher likelihood of formal employment. These results were strong for harder to employ jobseekers including youth and jobseekers with low skills, were driven by specific matching, where jobseekers were directly connected with employers and benefitted from an introduction to the employer. This suggests that matching can play an important role in removing certain information asymmetries and related constraints for employers and jobseekers, and improving employment outcomes for vulnerable groups.

This paper also analyzed the incentives and behaviors of the private employment service provider. Assessment of the jobseekers’ employability by the counselor was a key determinant of services provided to them. Consistent with cream skimming behavior, those assessed to have low employment prospects were significantly less likely to be provided matching services. Interestingly, despite the lower likelihood of being allocated to matching services, treatment effects were positive (though statistically indistinguishable from 0 owing to small sample size) for _less employable_ individuals. For those rated less employable, there are characteristics unobservable to the econometrician but observed by the counselors which was used to assign jobseekers to specific matching resulting in positive effects. Further, the 16
counselors have information on specific needs (demand) of firms and a fit may have arisen for a jobseeker of low skill who was otherwise judged to be less employable. For those rated highly employable, even though they had a higher likelihood of being referred to specific matching, treatment effects were close to zero as even in the control group these individuals managed to find employment.

Finally, the paper also examined the determinants of employability assessments. Among characteristics observable by jobs counselors, jobseekers over age 30 were more likely to be judged employable, which may be a proxy for prior work experience, and those with less than high school education and those unemployed for more than a year, were judged to be less employable. Gender and household characteristics including size and income are uncorrelated with employability assessments. Two of the “Big-5” personality traits of openness and extraversion are positively associated with employability assessments.

The implementation and evaluation of this program yielded several insights relevant for policy. Design features and payment arrangements play a critical role in determining how successful such programs can be. Contracting provision of employment services may potentially offer multiple benefits, including increased flexibility to expand coverage in line with unemployment, the possibility to offer more costeffective services, and to better tailor services in terms of content and through specialized service providers. In the context of BiH, outsourcing intermediation to the private sector seems to have had positive effects on employment outcomes, driven largely by matching. However, it is important to pay close attention to the design of such outsourcing arrangements. In the treatment arm with counseling and matching, only 23 percent of jobseekers received specific matching services, and there is evidence that the counselor’s assessment of employability was a key determinant of who received these services and that those jobseekers who were classified as “highly employable” or “employable” had a higher probability of receiving specific matching. Therefore, if the payments to the private firm are largely contingent on job placement without a premium on less-employable jobseekers, there is a risk of creamskimming behavior on their part, by focusing efforts on those who are relatively easy to employ.

Several factors need to be considered when examining the potential of scaling-up from the pilot program. First, successful scale-up would require carrying out a detailed assessment of the capabilities of existing private sector providers. Given the size of the pilot program, the private employment service provider could rely on existing connections with employers to provide matching services. Successful 17
scaling-up of the program would require working with private employment services who can cater to a wide array of sectors and occupations. It would be important, therefore, to carry out an assessment of the landscape of private employment service providers prior to expanding such programs. Second, the institutional capacity of the public employment services to administer outsourcing is an important factor in its success. There are sizeable transactions costs involved in contracting-out employment services, and the contracting authority will need to devote sufficient resources to administration and monitoring. Third, the target groups for the pilot program (women, youth and the long-term unemployed) are not a typical clientele for the private provider, so there may be limited opportunities for cost effective scale-up (conversations with the implementing agency suggest this may be the case). This suggests that there is a need to devise a tiered payment scheme to encourage and reward the private provider to deliver additional counseling, training, and intermediation for the “less employable” and minimize creamskimming. A fourth consideration turns on strategies to minimize the potential displacement effects of a larger scale program. A program targeting the “less employable” may displace more productive jobseekers who might otherwise find work on their own. A final consideration is on external validity of such program evaluations. The pool of applicants in the present study were likely to have been very motivated and driven (they responded to advertisements for employment services during a pandemic), and possibly different than the average PES registered jobseeker. Therefore, the same effects may not have been possible to obtain with a less motivated cohort of unemployed workers.

It is also important to remember that matching addresses specific constraints and may not always work and may not work for everyone. The results suggest that matching can help rectify the limited ability of certain jobseekers to signal their skills and abilities to employers. In a labor market where information asymmetries are large, matching can therefore play an important role. However, it is important to keep in mind that there are several other constraints that matching cannot address. For example, if a labor market is characterized by low demand and productivity, matching-based interventions will have very limited effectiveness. Similarly, if the skills of the workforce are not suited to labor demand (for example, due to changing technology or outdated education systems), matching can have at best marginal impacts on employment outcomes.

18