metadata: authors: 0: author: Yamauchi,Futoshi
1: author: Nomura,Shinsaku
2: author: Imaizumi,Saori
3: author: Areias,Ana Carolina
4: author: Chowdhury,Afra Rahman
count: India
docna: 0: docna: Asymmetric information on noncognitive
            skills in the Indian labor market :  an experiment in online
            job portal
docty: Policy Research Working Paper
subtopic: Labor Markets,Education For All,Educational Populations,Education for Development (superceded),Gender and Development,Rural Labor Markets
teratopic: Social Protections and Labor,Gender,Education,Rural Development
abstracts: cdata!: This paper examines the impact of
            noncognitive (socio-emotional) skills on job market
            outcomes, using a randomized control trial implemented in an
            online job portal in India. Job seekers who registered in
            the portal were asked to take a Big-Five type personality
            test and, for a random subsample of the test takers, the
            results were displayed to potential employers. The outcomes
            are measured by whether a potential employer shortlisted a
            job seeker by opening (unlocking) his/her application and
            background information. The results show that the treatment
            group for whom test results were shown generally enjoyed a
            higher probability of unlock. That is, employers are more
            interested in those for whom they can see personality test
            results. Such a relationship was not seen in the pre-test
            period, which confirms that the results are unlikely to be
            spurious. The study also finds a significant impact among
            organized, calm, imaginative, and/or quiet applicants (no
            effect is detected among easy-going, sensitive, realistic,
            and/or outgoing applicants), which seems to display
            employers' preference.
geo_reg_key: 517198,517191,517212
repnme: repnme: Asymmetric information on noncognitive
            skills in the Indian labor market :  an experiment in online
            job portal
display_title: Asymmetric information on
            noncognitive skills in the Indian labor market : an
            experiment in online job portal
listing_relative_url: /research/2018/03/29751646/asymmetric-information-noncognitive-skills-indian-labor-market-experiment-online-job-portal
publishtoextweb_dt: 2018-03-26T00:00:00Z
historic_topic: Social Protections and Labor,Gender,Education,Rural Development
no_of_pages: 27
author_keys: 0: author_key: 1209647
1: author_key: 782579
2: author_key: 1107003
3: author_key: 1075816
4: author_key: 1112576
model_extractions: WPS8378 Policy Research Working Paper 8378 Asymmetric Information on Noncognitive Skills in the Indian Labor Market An Experiment in Online Job Portal _Futoshi Yamauchi Shinsaku Nomura Saori Imaizumi Ana Areias Afra Chowdhury_ Education Global Practice March 2018
Policy Research Working Paper 8378

# **Abstract**

This paper examines the impact of noncognitive (socio-emotional) skills on job market outcomes, using a randomized control trial implemented in an online job portal in India. Job seekers who registered in the portal were asked to take a Big-Five type personality test and, for a random subsample of the test takers, the results were displayed to potential employers. The outcomes are measured by whether a potential employer shortlisted a job seeker by opening (unlocking) his/her application and background information. The results show that the treatment group for whom test results were shown generally enjoyed a higher probability of unlock. That is, employers are more interested in those for whom they can see personality test results. Such a relationship was not seen in the pre-test period, which confirms that the results are unlikely to be spurious. The study also finds a significant impact among organized, calm, imaginative, and/or quiet applicants (no effect is detected among easy-going, sensitive, realistic, and/or outgoing applicants), which seems to display employers’ preference.

This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at snomura@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
Asymmetric Information on Noncognitive Skills in the Indian Labor Market An Experiment in Online Job Portal<sup>1</sup> Futoshi Yamauchi<sup>2</sup> , Shinsaku Nomura, Saori Imaizumi, Ana Areias, Afra Chowdhury JEL Classifications: J23, J24, C81, C93 Key words: Noncognitive skills, Job portal, Randomized control trial, India 1 We would like to thank John Gibbons, Sean Blagsbets, and Vir Kashyap (formerly Babajob, which was merged into Quikr Jobs in June 2017) for implementing the proposed experiment and participants at the KDI School of Public Policy and Management 20<sup>th</sup> anniversary commemorative workshop for useful comments and suggestions. Financial supports are received from the World Bank’s Jobs Trust Fund and the Strategic Research Program. We are grateful to Pankhuri Shrivastava (Quikr Jobs) for continued support to this study. The opinions and conclusions expressed in this paper are ours and do not necessarily reflect positions of the World Bank and its member governments. Any remaining errors are ours.

2 Corresponding author. International Food Policy Research Institute, 1201 Eye Street, NW, Washington DC 20005; Email: f.yamauchi@cgiar.org; Phone: +1-202-862-6469.
# 1. Introduction

Asymmetric information on both workers and employers is one important source of frictions in the labor market. Employers often have to rely on observables to infer unobservable characteristics of workers such as innate ability. In a classic example, educational attainment is taken as an observable to signal ability under some circumstances, which leads to an inefficient equilibrium in the labor market (Spencer,1973; Rothchild and Stiglitz, 1976; Wilson, 1977). In this setting, workers (students) are encouraged to invest in education simply to prove their ability rather than to augment their labor productivity, in contrast to the conventional human capital theory (Becker, 1962). Though it has been increasingly recognized that roles played by non-cognitive (socio-emotional) skills are quite important in the workplace (Heckman and Kautz, 2002; Barrick and Mount, 1991), such skills of job applicants are typically unobservable to employers. In this paper, we examine the impact of non-cognitive skills on job market outcomes by introducing a unique randomized control trial into an online job portal in India.

The recent literature highlights the importance of non-cognitive skills, including soft skills, personality traits, abilities, character skills, and socio-emotional skills, as a determinant of life outcomes (Almlud, et al., 2011; Kautz, et al., 2014). Studies show that non-cognitive skills are equally as significant as cognitive ability to explain labor market outcomes (e.g., reviewed in Kautz, et al., 2014). Non-cognitive skills can be also formed and malleable until later ages. Our study uses a relatively well-accepted taxonomy of non-cognitive skills called the Big Five (OCEAN: Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism).<sup>3</sup> For example, Conscientiousness of the Big Five - the tendency to be organized, responsible, and hardworking - is associated with job performance and wages (Barrick and Mount, 1991; Hogan and Holland, 2003; Nyhus and Pons, 2005; Salgado, 1997). The experiment we propose below aims to break a typical setting in the labor market that, to a large extent, non-cognitive skills of job seekers are unknown to potential employers. In this attempt, the online job portal proves very useful.

Currently, global online job portals such as LinkedIn, Indeed, Monster and Career Builder connect employers and job seekers in many countries. A growing number of local online job portals have also emerged, including formerly Babajob (merged into Quikr Jobs in June 2017; all Babajob users were carried

> 3 Other measurements are also proposed to capture a certain aspect of non-cognitive skills, such as Grit (Duchworth, et al., 2007). The Big-Five measurements depend on self-reported answers, but researchers also proposed to use actual behavioral decisions to measure non-cognitive skills, such as risky and reckless behaviors measured in the adolescent years (Heckman, et al., 2014) and tenth grade participation in sports, academic clubs, and fine arts activities (Lleras, 2008). As discussed in Section 3, the answers by job seekers might be biased as those who decided to take the test might have an intention to look good. However, the sample of test takers is randomly split into treatment and control, so our empirical results are unbiased under the above-mentioned conditionality.

2
to Quikr Jobs).<sup>4</sup> The job portals provide information on job vacancies and help expand access to job information for those who have internet connectivity. These online platforms, global or local, provide realtime streams of labor market data that have remained largely unused by researchers as well as policy makers.<sup>5</sup> A rapid expansion of job portals is a manifestation of their usefulness for both job seekers and employers. A great advantage of online job portals is low marginal cost to acquire information. In online job portals, job seekers and employers can easily access each other. Kroft and Pope (2014) and Mang (2012) discuss that there has been a large-scale shift from posting job listings in newspapers and other print media toward websites and that this process has lowered the cost of acquiring employment information. Moreover, several studies from developed countries have found that the use of employment websites has reduced unemployment rates (Beard et al., 2012; Kuhn and Mansour, 2014), though an impact on wages remains unclear (Kuhn and Mansour, 2014; Shahiri and Osman, 2014). However, as Shahiri and Osman (2014) point out, it is important to keep in mind that a large number of employers and job seekers still cannot fully utilize employment websites due to limited internet infrastructure, high user fees, and/or a lack of knowledge about information technology. Some studies also found that a greater use of cell phones improves employment outcomes in developing countries (Klonner and Nolen, 2010 for South Africa; Aker, 2011 for Niger; Burga and Barreto, 2014 for Peru), implying that information constraints are a significant source of inefficiency in the labor market.

In this paper, we use an online job portal in India (Babajob, described in the next section) to experiment on information asymmetry of job seekers’ non-cognitive skills. Job seekers who registered in the portal were asked to take a Big-Five type personality test and, for a random sub-sample of the test takers, the results were displayed to potential employers. In this experiment, outcomes are measured by whether a potential employer assesses a seeker by opening (unlocking) his/her application and background information. Though whether or not to take the test is a voluntary decision (thus creating a selectivity issue), whether or not the results are displayed to potential employers in Babajob is random. Therefore, we can compare outcomes between those for whom the results were shown and not shown to analyze the impact of non-

> 4 Currently the platform developed in formerly Babajob was part of the Quikr Jobs platform. In this paper, we refer to Babajob since the randomized control trial was introduced into Babajob prior to their transition to Quikr Jobs (June 2017).

> 5 For example, LinkedIn, a social networking service for businesses and professionals launched in 2003, had data on 433 million individuals as of Q1 2016. The site is available in over 200 countries worldwide and in 20 different languages. The data obtained through LinkedIn have been primarily used for business purposes, rather than for policy formulation, though various industry- and skills-focused analyses have been published on LinkedIn’s official blog. Tambe (2014) used skills data from LinkedIn to measure employers’ investment in big-data-related human resources management.

3
cognitive skills. The execution of our experiment was fast once it was programmed and tested in the system, as the number of participants in Babajob is large (i.e., big data environment).

The paper is organized as follows. Section 2 describes the Indian labor market and Babajob. Section 3 describes in detail the experiment on non-cognitive skills. Section 4 displays empirical results. The results show that the treatment group for whom test results were shown generally enjoyed a higher probability of unlock (one step before being shortlisted). That is, employers are more interested in those for whom they can see personality test results. Such a relationship was not seen in the pre-test period, which confirms that the above results are unlikely to be spurious. We also found a significant impact among organized, calm, imaginative and/or quiet applicants (no effect was detected among easy-going, sensitive, realistic and/or out-going applicants), which seems to display employers’ preference. Implications are discussed in the concluding section.

# 2. Empirical Setting

India has made remarkable progress in economic growth as well as poverty reduction over the past few decades. India’s gross domestic product (GDP) grew at an average rate of 7.3 percent per year between 2007 and 2012. This contributed to a substantial decline in the incidence of poverty, and an estimated 138 million people rose above the poverty line during the period. Despite its robust growth, India still faces major challenges to improve labor productivity and match the supply and demand of workforce skills.<sup>6</sup> In India, online job portals emerged in the late 1990s, but only began to flourish in the past decade as mobile phone and internet use became more widespread and social networks expanded. The share of mobile phone subscribers more than tripled from 20 percent of the population in 2007 to 70 percent in 2014, and the number of fixed broadband subscriptions increased eightfold during the same period. There are now about 20 job search portals, many of them focusing solely on the Indian labor market.<sup>7</sup> Babajob, once established

> 6 The Indian labor force is large, mostly informal and relatively young. India’s population reached 1.295 billion in 2014, including 497 million workers.  The size of the labor force has been expanding at an annual net growth rate of 4.2 million for the past 10 years. The labor force participation rate is 54 percent, with a relatively high participation rate among men (80 percent) and low rate among women (26 percent). Only 16 percent of the labor force is engaged in wage employment (18 percent of male workers and 12 percent of female workers), and a large majority work in the informal sector. Moreover, 54 percent of the country’s population is under 25 years of age. While the country’s relatively young population has the potential to yield significant demographic dividends, ensuring sufficient employment opportunities for a young workforce also presents a critical challenge for the government. India faces a lack of highly trained workers and a large share of unskilled youth. As a result, job creation and skills development are critical priorities. To address these challenges, the government launched the National Policy for Skill Development and Entrepreneurship in 2015, and its 12th Five-Year Plan set a goal of training 400 million workers by 2022.

> 7 Though not exhaustive, 22 firms were identified by compiling a list from government agencies and search engines. 77 percent of those firms provide job-search services, while a few provide job-matching services. Some platforms

4
in 2007, became one of the leading job-matching websites in the country. Between July 3, 2007 and May 24, 2017 there were 1,286,812 ads posted by 524,672 employers and 529,023 employers and 8,218,720 job seekers were registered with Babajob. Babajob matched workers to potential employers in both formal and informal sectors. In order to reach disadvantaged populations, Babajob provides a variety of access options including standard websites, mobile sites, interactive voice response (IVR), text messaging and web applications.<sup>8</sup> A large number of the jobs listed in Babajob are at the entry level, with the largest share of listings in 2015 classified as clerical support. 70 percent of the jobs advertised were in the 10 most populous cities: Bangalore, Delhi, Mumbai, Chennai, Hyderabad, Pune, Kolkata, Thane, Patna, and Lucknow. In 2015, the average offered salary was 13,182 rupees (Rs.) per month.<sup>9</sup> The average offered salary for professionallevel jobs is Rs. 14,900, 17 percent higher than that for non-professional jobs, Rs. 12,739. By city, the average salary for professional jobs ranged from Rs. 16,970 in Mumbai to Rs. 12,757 in Patna, while the average for non-professional jobs ranged from Rs. 14,184 in Delhi to Rs. 10,742 in Patna.

# 3. Experiment

In mid-February 2017, a shorter version of the Big-Five personality test (Big Five Inventory-10 in Rammstedt and John, 2007)<sup>10</sup> was introduced to job seekers who were relatively active in Babajob in the past several months. Test takers were then randomly split into two groups: those for whom the results were displayed to potential employers and those for whom the results were kept confidential. Those who decided to take the test were informed that the results may be displayed in their profiles as additional information when they apply for a position.

# - <u>Big Five Personality Test</u>

focus on entry-level jobs, while others mainly advertise technology or senior management-level jobs. Job-matching platforms use different techniques to match workers with job opportunities, including leveraging social networks, providing curated job information based on a individual profiles and connecting local recruiters with candidates. 68 percent of the platforms focus on jobs located in India, with some portals concentrating on specific cities.

8 The Babajob platform works to connect job seekers and employers. Job seekers can create a profile in Babajob, search and apply for jobs online or offline, all for free. Employers can create profiles in Babajob and post their hiring requirements for free in a service that resembles online job classifieds, or they can opt for the paid, premium service ( _RapidHire_ ) that offers a facilitated hiring experience. All job posts are alive for 90 days. _RapidHire_ jobs are promoted more heavily on the site for a certain period depending on the plan opted for (e.g. the basic plan promotes jobs for 15 days). _RapidHire_ also offers other services, such as additional screening, executive recruitment support, unlocking more information about candidates, and SMS promotion of the posted job to relevant job seekers within a certain radius.

> 9 Using an exchange rate of US$1=Rs. 67 (the average rate during the first half of 2016) this amount is equivalent to about US$197.

> 10 We used BFI 10 plus 1 additional item on Agreeableness suggested by Rammstedt and John (2007).

5
The Big-Five personality test aims to measure non-cognitive (socio-emotional) skills through the following five components: (i) openness (to experience), (ii) conscientiousness, (iii) extraversion, (iv) agreeableness and (v) neuroticism. Details on each component are below (Table 4.2 of Jonh, Naumann and Soto, 2008): _Openness to experience_ describes the breath, depth, originality and complexity of an individual’s mental and experiential life.

_Conscientiousness_ describes socially prescribed impulse control that facilitates task- and goal-directed behavior, such as thinking before acting, delaying gratification, following norms and rules, and planning, organizing, and prioritizing tasks.

_Extraversion_ implies an energetic approach toward the social and material world and includes traits such as sociability, activity, assertiveness and positive emotionality.

_Agreeableness_ contrasts a prosocial and communal orientation toward others with antagonism and includes traits such as altruism, tender-mindedness, trust and modesty.

_Neuroticism_ contrasts emotional stability and even-temperedness with negative emotionality, such as feeling anxious nervous, sad and tense.

# <u>Pilots</u>

The experiment was first developed through pilots. Two different pilots were introduced to test both the content of the assessment and the survey tools planned to be used. The first pilot was composed of a telephone survey of 100 job seekers in our target population, using the Big Five Personality Test (24 question version adopted in the World Bank’s Skills Toward Employment and Productivity (STEP) Measurement Survey (Pierre et al. 2014)). For this survey, Babajob’s Consumer Insights Team was instructed to call job seekers and give the questions exactly as they were written without any explanation, even if the respondent requested it. The survey was administered from June 22 to June 24, 2016. Half of the job seekers were told that the results of this survey would be shared with potential employers, while the other half were told that the survey was purely for research purposes and the results would not be shared. The goals of this pilot were to ensure that job seekers understood the questions of the assessment and how to answer them, and to investigate whether job seekers were likely to change their answers if they believed the results would be shown to employers. Overall, the job seekers understood the questions being asked (only 2.7% of all responses given were the “Respondent couldn’t answer” option), and there was not significant deviation between the two groups.

6
For the second pilot, the team tested our planned data collection methods. As described later, the team decided to pursue recruitment to the RCT primarily through SMS messages to job seekers, sending a link to the survey instrument in the text. This pilot aimed to test the effectiveness of the recruitment, as well as gave the opportunity to experiment with the combinations of messaging and SMS timing to determine how to increase the number of respondents.

From November 29 to December 1, 2016, a series of SMS blasts were sent to job seekers over a week with a link to a survey tool that allowed the users to fill out the Big Five Personality Test (11 question version). Overall, we sent approximately 124,000 SMS to 44,000 job seekers. They clicked the link in the SMS 3,865 times, with 1,010 completing the survey. The team experimented with different variations in the content and frequency of the messages, starting with sending three SMS to each seeker spread over three hour intervals. In our final SMS blasts, sending one SMS and waiting at least 24 hours before subsequent blasts was found as the most efficient conversions.

# <u>Technology Design</u>

A variety of iterations were tested when determining how to run the main intervention/data collection (i.e. getting job seekers to take the personality test). In the original design, the team was planning to build the functionality into the Android app, as users on this platform have data-rich profiles and high application per applicant numbers. Due to sample size requirements and the smaller numbers seen in app downloads and usage, the team decided to focus on job seekers using Babajob’s mobile web platform. Building the assessment into the registration process was also considered, but this would not allow for proper sampling and threatened Babajob’s registration/application numbers, so the team switched to a direct recruitment method, prompting users through SMS to take the test.

The team planned to use a survey software outside of the Babajob platform to collect the required data (one stage of the RCT pilot), but this approach provided several technical difficulties and meant that updates to users’ profiles with the personality test results would only happen periodically. Instead, the team decided to have job seekers take the test through Babajob’s chatbot. Through this, job seekers would be able to click a link in the recruitment SMS, enter their registered mobile number and begin the personality assessment. Below are screenshots of the test introduction, the process of answering the questions, and the job seeker’s view of the results: 7
<!-- Start of picture text --> x<br>a bab H b Ml ask you 17 queshons and then tell 2.Often<br>these eth employers uncreases the 4 Almott Never<br>Odds youl be hired. ys ee<br>1 Rew yew tees great nearty joo m3 | Rimember there ane no right or &<br>ouetors wrong arswers<br>Seteads ~ hatees baie tes Genre 111: [Perionalitytrad, whach rl thow<br><!-- End of picture text -->

<!-- Start of picture text --> Rekajetsdiene- Betnes lg.gifat tami“ee ae4 Alrmost- Never tmaginative os<br>Teter lee eed Percemek Copeine Cuncul, et by few Ings Lary<br>arirret entice Se 6<br>] Take the Quc Share with Freends! @ Mew<br>2/11;other Avepeople?you generally(Reply withtrustinga number) of<br><!-- End of picture text -->
<!-- Start of picture text --> x 920 KM“away UNLOCK. CANDIDATE—<br> SCORE:O% ut Feb le 2007<br>PERSONALITY [manner on | corres |[ceormceron ||camecmctn|| ie<br>Show More +<br>;<br>x T17000 95 KM awaya ——<br>1 SCORE: 0m ui am<br>Show More +<br><!-- End of picture text -->
<!-- Start of picture text --> Sow Mure +<br>— ol<br>a ¥ Scone om | Sa<br>Show More 4<br>x nowan aney frTh e ee<br>1 SCORE:0% fed= 4 2077<br>PERSONALITY [ow ATE Sm| cr res | | cooprmmren ars || capr-come ors [| caus so |<br><!-- End of picture text -->

<!-- Start of picture text -->
* Male ? Bangalore, 560091 =<br>© 19 years old ih TenPlusPUCOr12th<br>@ English,Kannada, VIEW PROFILE<br>PERSONALITY TEST RESULTS<br>CET Fanciful and curious, likesto try new things. Creative and in touch with<br>thes feelings Usuatly Lideral.<br>CETalone ratherPrefersthan in togroups concentrate and doing on asolitary single activityactivities.at a time, Likes to spendtime<br>Seas Sensitive and warm-hearted, gets along wellwith others. Tends to<br><!-- End of picture text -->

<!-- Start of picture text --> CREEomented and TL]lessMore driverflexibleby success.and spontaneous. Tendsto be more laid back,tess goal-<br>= Tends to be calm and unemotional. Handles stressful situations well. Does<br>hot get eathy upset.<br>= <=<br>Hirtng Tips: interested iin hiring Shera) ?<br>1 Schedule a4 interview here - wel tend intervers remingert,$0 That the Candicate turns wp on time<br>2. Offer here - we'll make sure that the casdi¢ate Goes not receive other job nothcatioat/oters,<br><!-- End of picture text -->
The team was able to track the results of the RCT without additional development based on Babajob’s existing data tracking methods. Results tracking was performed through May 24, 2017.<sup>11</sup> Babajob tracked every application made by a user, storing a record of the same, as well as anytime an application is “Unlocked” (the primary result we are tracking, which signifies an employer is placing value on the applicant). In addition to this, Babajob has tracked which job seekers have completed the personality test and their answers.

# <u>Descriptive Analysis</u>

In this sub-section we show descriptive results to characterize the experiment. Personality traits were distributed as follows (including both treatment and control).

There is potential bias in the distributions due to voluntary decisions made by seekers to take the test. Therefore, the distribution is not representative of personality traits in the general population. For the same reason, there can be differences between this result and the first pilot done by phone as the pilot was conducted under strict confidentiality. Except in extraversion, the traits are one-sided: cooperative in agreeableness, organized in conscientiousness, imaginative in openness, and calm in emotional stability. Those who decided to take the test might have an intention to make them look attractive to potential employers by choosing particular answers.

Next, the treatment and control groups are compared in personality test scores as well as traits. In our experiment, 51.5% of the sample is in treatment (Table 2). Since the treatment is a random sub-sample of test takers, it is designed that the treatment and control groups are probabilistically ex-ante identical.

Table 3 shows that personality traits (defined in this study) are balanced between the treatment and control groups. Two-sided t tests could not differentiate the two groups (the equality of means is not rejected at the conventional level).

Table 4 compares gender and education distributions by treatment status. Though we do not perform a test to compare the two groups, it is evident that the two distributions extraordinarily resemble. Similarly, age distributions are compared between the treatment and control groups (Figure 1), which confirms that the two groups are balanced in age. The above descriptive results appear to provide a good justification to identify the impact of non-cognitive skills (additional information on profiles) on job market outcomes.

> 11 There are variations in the tracking period. The last test taker was recorded on May 16.

11
# 4. Results

We have confirmed that the treatment and control groups are similar in individual characteristics and personality test scores. The only difference between the two groups is whether the test scores were displayed to potential employers (treatment). Data were collected in the portal even before the test, so we have information on their job search behavior and outcomes prior to the experiment. Therefore, this setting offers us four different situations: (a) treatment after experiment, (b) control after experiment, (c) treatment before experiment and (d) control before experiment. In (c) and (d), the distinction between treatment and control is trivial because they have not taken the test at that stage, thus whether or not the test results were displayed in the system has no significance. The situations in (a) and (b) are in our primary interest and we conjecture that the difference between treatment and control after experiment should not hold before experiment if that is attributed to the intervention defined above.

The system records an incident that an employer opens (unlocks) an application. As described earlier, we use the unlock incidence as the outcome measure to detect the impact of displaying non-cognitive skill information on potential employers’ reactions. In Babajob, there are two types of job advertisements: paid and unpaid. In paid advertisements, employers who post those advertisements pay fees to have additional services from Babajob such as a preliminary screening to improve matching. In unpaid advertisements, such a service is not attached, so employers have to check applications by themselves. To unlock is the first action by employers in order to see details on the job applicant for short-listing. In the analysis, we use unpaid job advertisements because the number of unlocks in paid job advertisements was very small during the period under our experiment. Apparently, the number of unlocks is (positively) correlated with that of applications, so we include the number of applications to control on the right hand side. The intervention dummy is interacted with the number of applications to capture two possible effects: (a) direct effect and (b) indirect effect through the rate of unlock per application.

Estimation is also executed in a sub-sample to know differentiated effects by job seekers’ characteristics. We look at their personality traits (revealed in the test), gender and job category. For example, results in the conscientiousness part of the test provide an indication as to whether the person is likely to be organized or easy-going and it is possible that the impact of personality traits information display could differ between organized applicants and easy-going applicants. Similarly, we can compare cooperative and competitive applicants; imaginative and realistic; calm and sensitive; quiet and out-going. Possible gender difference is also an interesting issue. If female and male applicants to a job vacancy are not treated equally for some cultural reasons (e.g., the job is traditionally handled by males), the employer may want to know more details of the female applicant to assess her suitability to the job. Alternatively, if female characteristics, 12
both observable and unobservable, are in small variations, employers may want to gather more information from male applicants. In some jobs, workers are required to face customers face to face and soft skills may be more important in those jobs.

Table 5 presents the benchmark results. The estimation used the sample of job seekers if they had been in the Babajob platform more than 30 days prior to the test and the number of applications is less than 30. As hypothesized earlier, we confirm that the disclosure of personality test results (treatment) has a positive and significant impact on the number of unlocks after controlling the number of applications and its interaction with the treatment indicator only if the post-intervention period is used. In the pre-intervention period, the treatment indicator is insignificant, which indicates that the above result is unlikely to be spurious. Even in the pre-test period, the number of applications is positive and significant, which confirms that the positive relationship between the numbers of applications and unlocks remains stable before and after the intervention. The qualitatively same results were confirmed even when gender, education and age are controlled (though the sample size is reduced to missing data for some observations).

In Table 6, we relax one of the assumptions to define the estimation sample. The number of days staying in the system prior to the test is 60 days. The results remain qualitatively the same. Only after the intervention, the treatment indicator has a positive and significant effect on the number of unlocks.

Employers may be looking for job seekers who possess a certain set of personality traits. Our experimental design enables us to look deeply into this issue. Since we saw earlier that personality trait distributions are identical between the treatment and control groups, we can examine the treatment effect separately for different personality trait groups, e.g., the organized versus the easy-going. Table 7 shows the results by component-wise personality trait. Interestingly, we found a significant impact among organized (high conscientiousness), calm (low neuroticism), imaginative (high openness) and/or quiet (low extraversion) applicants (no effect was detected among easy-going, sensitive, realistic and/or out-going applicants), which seems to display employers’ preference. Cooperative and competitive are not easily differentiated in the above results (t values are 1.81 and 1.76, respectively). The results imply that employers generally prefer job seekers who are organized, clam, imaginative and/or quiet.

Next, we split the sample by gender to check gender-specific treatment effects. Table 8 presents the results for each gender group. Interestingly, we observe a significant and positive (direct) effect of the intervention among males, but such an effect was not detected among females. Instead, the estimation shows that the marginal effect of the number of applications significantly increases with the intervention, i.e., the unlock rate becomes higher among females. It is possible that insignificance of the direct effect among females 13
maybe due to a relatively small sample size (power). To cross check potential difference of personality trait patterns between males and females, Table 9 shows the percentage of test takers (by gender) who are labeled as cooperative, organized, imaginative, calm and/or quiet. Though there are some differences by gender, we do not see any large gaps between the two groups.

A similar sub-sample analysis is performed on job categories. Here we pick up categories that have relatively large numbers of observations in the sample. They are Office Clerk, BPO, Sales, Receptionist and Driver. Table 10 reports the results by job category group. The treatment (direct) effect is significant and positive in Office Clerk, Sales and Receptionist, in which workers normally need to physically face customers. In contrast, the effect is not significant in BPO and Driver. In these categories, they face customers by phone (BPO) or while in transportation if they drive a taxi or bus (truck drivers do not face customers). The results seem to imply that non-cognitive skills are considered to be valued in jobs for which physical customer interface is important. However, it is also important to be reminded that some job categories have relatively small sample sizes, likely not enough to render sufficient confidence in the estimates.

# 5. Conclusions

The innovative randomized control trial introduced in an online job portal has shown that (i) non-cognitive (socio-emotional) skills play potentially important roles in the hiring process and (ii) employers seem to have certain (stereo-typical) preference when looking for employees. In other words, information on noncognitive skills is usually quite private, thus employers generally do not have such knowledge on job applicants and may have to imperfectly infer it from observable (easily verifiable) data. In this sense, asymmetric information on non-cognitive skills can be a significant source of inefficiency prevailed in the labor market.

Our study stands very uniquely in the current big data research. While the majority of analyses on big data apply different methodologies to such rich data, we successfully introduced an experiment into a job portal platform (big data). We are aware that the recruitment of test takers was on a voluntary basis, but differentiation of the takers into treatment and control was random, which offers an ideal setting to test roles played by non-cognitive skill asymmetric information in the job market.

References 14
Aker, J.C., 2011. Dial “A” for agriculture: a review of information and communication technologies for agricultural extension in developing countries, _Agricultural Economics_ , 42(6): 631-647.

Almlund, M., A. Duckworth, J. J. Heckman, and T. Kautz (2011). Personality psychology and economics. In E. A. Hanushek, S. Machin, and L. W ̈oßmann (Eds.), _Handbook of the Economics of Education_ , Volume 4, pp. 1–181. Amsterdam: Elsevier.

Barrick, M. R. and M. K. Mount, 1991, The Big Five personality dimensions and job performance: A meta-analysis, _Personnel Psychology_ 44 (1), 1–26.

Beard, T. R., Ford, G. S., Saba, R. P., & Seals, R. A., 2012, Internet Use and Job Search. _Telecommunications Policy_ , 36(4), 260-273.

Becker, G., 1962, Investment in human capital: A theoretical analysis, _Journal of Political Economy_ 70 (9).

Burga, P. I. R. and M. E. G. Barreto, 2014, The effect of Internet and cell phones on employment and agricultural production in rural villages in Peru, Manuscript, Harris School of Public Policy, University of Chicago.

Duckworth, A.L., C. Peterson, M.D. Matthews, D.R. Kelly, 2007, Grit: Perseverance and passion for long-term goals, _Journal of Personality and Social Psychology_ , 92 (6), pp. 1087-1101.

Heckman, J.J. and T. Kautz, 2012, Hard evidence on soft skills, _Labour Economics_ , 19 (4), August 2012 Heckman, J. J. and S. Mosso, 2014, The economics of human development and social mobility. _Annual Review of Economics_ 6 (1), 689–733.

Hogan, J. and B. Holland, 2003, Using theory to evaluate personality and job-performance relations: A socioanalytic perspective. _Journal of Applied Psychology_ 88 (1), 100–112.

John, Naumann and Soto, 2008, Paradigm shift to the integrative Big Five trait taxonomy: History, measurement and conceptual issues, Chapter4, In O. P. John, R. W. Robins, & L. A. Pervin eds., _Handbook of personality: Theory and research_ , New York, NY: Guilford Press.

Kautz, T., J.J. Heckman, R. Diris, B. Weel, and L. Borghans, (2014), Fostering and measuring skills: Improving cognitive and non-cognitive skills to promote lifetime success, NBER Working Paper 20749 Klonner, S., & Nolen, P. J., 2010, Cell Phones and Rural Labor Markets: Evidence from South Africa. Proceedings of the German Development Economics Conference, Hannover 2010 56, Verein für Socialpolitik, Research Committee Development Economics.

Kroft, K., & Pope, D. G., 2014, Does Online Search Crowd out Traditional Search and Improve Matching Efficiency? Evidence from Craigslist, _Journal of Labor Economics_ , 32(2), 259-303.

15
Kuhn, P., & Mansour, H., 2014, Is Internet Job Search Still Ineffective? _Economic Journal_ , 124(581), 1213-1233.

Lleras, C., 2008, Do skills and behaviors in high school matter? The contribution of noncognitive factors in explaining differences in educational attainment and earnings. _Social Science Research_ 37 (3), 888– 902.

Mang, C., 2012, Online job search and matching quality, Ifo Working Paper No.147, Ifo Institute – Leibniz Institute for Economic Research at the University of Munich.

Nyhus, E. K. and E. Pons, 2005, The effects of personality on earnings, _Journal of Economic Psychology_ 26 (3), 363–384.

Pierre, Puerta, Valerio and Rajadel, 2014, STEP skills measurement surveys: Innovative tools for assessing skills, Social Protection & Labor Discussion Paper No.1421, World Bank.

Rammstedt and John, 2007, Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German, _Journal of Research in Personality_ 41: 203-212.

Rothschild, M. and Stiglitz, J., 1976, Equilibrium in competitive insurance markets: An essay on the economics of imperfect information, _Quarterly Journal of Economics_ 80: 629-649.

Salgado, J. F., 1997, The five factor model of personality and job performance in the European Community, _Journal of Applied Psychology_ 82 (1), 30–43.

Shahiri, H., and Z., Osman, 2014, Internet Job Search and Labor Market Outcome, _International Economic Journal_ , 1-13.

Spencer, M., 1973, Job market signaling, _Quarterly Journal of Economics_ 87: 355–374.

Tambe, P, 2014. Big data investment, skills, and firm value, _Management Science_ 60 (6): 1452-1469.

Wilson, C.A., 1977, A model of insurance markets with incomplete information, _Journal of Economic Theory_ 16: 167-207.

16
Table 1 Personality traits

|Agreeableness Cooperative: 2017 Competitive: 853|
|---|
|Conscientiousness Organized: 2001 Easy-going: 869|
|Openness Realistic: 1022 Imaginative: 1848|
|Emotional stability Calm: 2058 Sensitive: 812|
|Extraversion Quiet: 1461 Outgoing: 1409|

Table 2 Sample composition by treatment status

||# obs|%|
|---|---|---|
|Control|1392|48.50|
|Treatment|1478|51.50|
|Total|2870|100.00|

17
Table 3 Personality traits by treatment status and t-test results

||Trea|tment|Co|ntrol||
|---|---|---|---|---|---|
||# obs|Mean|# obs|Mean|p value|
|Agreeableness|1478|61.857|1392|60.944|0.1322|
|Conscientiousness|1478|67.896|1392|67.787|0.8736|
|Openness|1478|62.465|1392|62.098|0.6462|
|Emotional stability|1478|69.133|1392|68.671|0.5298|
|Extraversion|1478|57.410|1392|56.983|0.5959|

Mean equality is tested under two side and unequal standard deviation assumptions.

Table 4 Characteristics by treatment status (number of observations)

|Gender|Treatment|Control|
|---|---|---|
|Uninformed|2|4|
|Male|1099|1024|
|Female|329|329|
|Education|Treatment|Control|
|Uninformed|558|522|
|Primary|1|3|
|Secondary|334|302|
|Post-Secondary/Tertiary|585|565|

Gender reported had a large number of uninformed cases: 179 in treatment and 147 in control. The results shown above are gender imputed from their given names in the case of being uninformed.

18
Table 5 Benchmark results

||After intervention Before intervention|
|---|---|
|Treatment|0.0353**0.0058|
||(2.44) (0.51)|
|# applications|0.0317***0.0178***|
||(4.16) (5.01)|
|Treatment * # applications|-0.0132 -0.0068|
||(1.49) (1.42)|
|R squared|0.1296 0.0751|
|N obs|1795 1791|

*** 1%, ** 5%, * 10% significance. Numbers in parentheses are absolute t values using robust standard errors. The sample consists of job seekers who were in the system more than 30 days at the time of taking the test and applied for less than 30 positions during the tracking period.

Table 6 Robustness

||After intervention Before intervention|
|---|---|
|Treatment|0.0408**0.0010|
||(2.29) (0.08)|
|# applications|0.0269***0.0170***|
||(3.57) (4.14)|
|Treatment * # applications|-0.0128 -0.0058|
||(1.43) (1.02)|
|R squared|0.1079 0.0670|
|N obs|1281 1278|

*** 1%, ** 5%, * 10% significance. Numbers in parentheses are absolute t values using robust standard errors. The sample consists of job seekers who were in the system more than 60 days at the time of taking the test and applied for less than 30 positions during the tracking period.

19
Table 7 Personality [agree/con/open/emo/extra, after/before]

|CooperativeAfter intervention Before intervention|
|---|
|Treatment 0.0343 (1.81)*0.0144 (1.18)|
|# applications 0.0352 (3.34)***0.0187 (4.37)***|
|Treatment * # applications -0.0150 (1.25) -0.0114 (2.22)**|
|CompetitiveAfter intervention Before intervention|
|Treatment 0.0403 (1.76)*-0.0085 (0.38)|
|# applications 0.0246 (2.69)**0.0155 (2.41)**|
|Treatment * # applications -0.0102 (0.89) 0.0012 (0.13)|
|OrganizedAfter intervention Before intervention|
|Treatment 0.0374 (1.97)*0.0051 (0.39)|
|# applications 0.0294 (3.06)***0.0174 (3.82)***|
|Treatment * # applications -0.0095 (0.86) -0.0092 (1.72)*|
|Easy-goingAfter intervention Before intervention|
|Treatment 0.0313 (1.57) 0.0040 (0.17)|
|# applications 0.0360 (2.94)**0.0186 (3.27)***|
|Treatment * # applications -0.0221 (1.60) 0.0010 (0.10)|
|ImaginativeAfter intervention Before intervention|
|Treatment 0.0405 (2.08)**0.0038 (0.25)|
|# applications 0.0330 (3.32)***0.0192 (4.46)***|
|Treatment * # applications -0.0167 (1.46) -0.0031 (0.49)|
|RealisticAfter intervention Before intervention|
|Treatment 0.0262 (1.26) 0.0105 (0.68)|
|# applications 0.0296 (2.51)**0.0151 (2.49)**|
|Treatment * # applications -0.0065 (0.47) -0.0133 (2.15)**|

20
|CalmAfter intervention Before intervention|
|---|
|Treatment 0.0343 (2.19)**0.0057 (0.41)|
|# applications 0.0308 (3.46)***0.0184 (4.39)***|
|Treatment * # applications -0.0154 (1.53) -0.0060 (1.03)|
|SensitiveAfter intervention Before intervention|
|Treatment 0.0299 (0.94) 0.0078 (0.42)|
|# applications 0.0339 (2.34)**0.0163 (2.46)**|
|Treatment * # applications -0.0047 (0.27) -0.0099 (1.24)|
|QuietAfter intervention Before intervention|
|Treatment 0.0419 (2.14)**0.0009 (0.05)|
|# applications 0.0375 (3.24)***0.0190 (3.86)***|
|Treatment * # applications -0.0238 (1.87)*-0.0047 (0.67)|
|OutgoingAfter intervention Before intervention|
|Treatment 0.0266 (1.24) 0.0120 (0.84)|
|# applications 0.0258 (2.66)**0.0162 (3.17)***|
|Treatment * # applications 0.0005 (0.04) -0.0094 (1.52)|

*** 1%, ** 5%, * 10% significance. Numbers in parentheses are absolute t values using robust standard errors. The sample consists of job seekers who were in the system more than 30 days at the time of taking the test and applied for less than 30 positions during the tracking period

21
Table 8 Gender

|Males Females|
|---|
|Treatment 0.0384**-0.0031|
|(2.39) (0.14)|
|# applications 0.0357***0.0060|
|(4.16) (1.41)|
|Treatment * # applications -0.0197*0.0228**|
|(1.99) (2.22)|
|R squared 0.1513 0.1135|
|N obs 1381 395|

*** 1%, ** 5%, * 10% significance. Numbers in parentheses are absolute t values using robust standard errors. The sample consists of job seekers who were in the system more than 30 days at the time of taking the test and applied for less than 30 positions during the tracking period.

Table 9 Personality traits by gender

|Male Female|
|---|
|Agreeableness (% Cooperative) 69.38 73.14|
|Conscientiousness (% Organized) 69.24 70.93|
|Openness to experience (% Imaginative) 65.66 60.51|
|Emotional stability (% Calm) 72.35 69.35|
|Extraversion (% Quiet) 51.95 49.29|

22
Table 10 Job category

|Office clerk|Sales Receptionist BPO Driver|
|---|---|
|Treatment 0.0550*|0.0756**0.1290*0.0483 0.0006|
|(1.92)|(2.07) (1.91) (0.91) (0.02)|
|# applications 0.0487**|0.0439**0.0828**0.0442 0.0066|
|(2.62)|(2.62) (2.05) (1.45) (1.21)|
|Treatment * # applications -0.0207|-0.0356*-0.0590 -0.0268 0.0010|
|(0.90)|(1.89) (1.34) (0.81) (0.11)|
|R squared 0.1722|0.1471 0.3007 0.2038 0.0326|
|N obs 367|175 143 225 133|

*** 1%, ** 5%, * 10% significance. Numbers in parentheses are absolute t values using robust standard errors. The sample consists of job seekers who were in the system more than 30 days at the time of taking the test and applied for less than 30 positions during the tracking period.

23
# <u><mark>ne</mark></u>