Evaluating the Self-selection Targeting Methodof the Oportunidades Program in Mexico

Tomohito Hata	
 
	
 
	
 
	
 
	
 
	
 
	
 
December 2013	 	
 Evaluating the Self-selection Targeting Method of theOportunidades Program in Mexico

A Thesis Submitted to the
Graduate School of Public Policy,
The University of Tokyo

byTomohito Hata
The Graduate School of Public Policy,
Economic Policy Division,
The University of Tokyo

Supervised by:
Carlos Chiapa (El Colegio de México)Hidehiko Ichimura (The University of Tokyo)

2

Abstract
The present paper evaluates the effectiveness in reducing poverty of thetargeting mechanism carried out by the Mexican social program
Oportunidades. The main objective of this paper is to find whether the
newly introduced targeting method in the urban areas in Mexico in 2002
(the self-selection method) is effective or not compared with the former
targeting method used in the rural areas (the census method). The
performances in terms of reducing poverty index are compared for the
self-selection method, the census method, a hypothetical perfecttargeting and the uniform transfer, by assuming a situation in which atransfer is given to the households classified as eligible for the program
under each method. The budget is set equal for all the targeting methods,
assuming that the amount subtracted the targeting costs of each method
would be the total transfer amount. The household welfare is measured
in terms of consumption, while the criterion applied by Oportunidades isbasically based on income. It is found that the self-selection method
reduces poverty index almost equally to the census method, and the two
methods are more effective than only executing the geographic targeting.
Concerning the way of distributing the transfer, distributing uniformly
to the targeted households slightly reduces poverty more than the way of
distributing

appliedby

Oportunidades.

Also,

by

comparingtheconsumption amount of the households targeted by the self-selection
method and census method, it is found that the consumption amount of
the households targeted by the self-selection method is likely to be less.
Taking into account that the self-selection method can be carried out

3with less budget amount, the Oportunidades program’s decision of
having applied this to the urban population can be supported.

4

Acknowledgements
I would like to express my deepest gratitude to Professor Carlos Chiapa
and Professor Hidehiko Ichimura, my research supervisors, for theirpatient guidance, encouragement and valuable advices. Also I would liketo appreciate Professor Yasuyuki Sawada and Professor Michio Suzuki
for their valuable comments at the oral examination. My grateful thanks
are also extended to Professor Hiroyuki Ukeda, who provided me with
the instructive discussion and valuable comments and to Professor
Takeshi Wada, who guided me from the basics of researches as asupervisor for my undergraduate thesis. I would also like to thank Ms.
Sarah Thomason for her tremendous support in revising my English
writing.

I would also like to extend my thanks to the exchange student program
between

Japan

and

Mexico(ProgramadeCooperación

para

la

Formación de Recursos Humanos en la Asociación Estratégica Global
entre México-Japón) and to all the persons concerned for giving me theopportunity to study in Mexico from 2011 to 2012.

Finally, I wish to thank my family and friends for their support andencouragement throughout my study.

5

Table of Contents	
 1. Introduction .............................................................................. 8
2. The Program and Targeting Methods ........................................ 13
2.1 Program Description ............................................................ 13
2.2 Targeting Methods ............................................................... 15
2.2.1. Geographic targeting ....................................................... 16
2.2.2. Targeting at the household level ...................................... 17
2.3. Outlook of the Analysis of This Paper ................................. 18
3. Data Description ....................................................................... 20
3.1. Data Contents ..................................................................... 20
3.2. Sampling Method ................................................................ 22
3.2.1. Sampling method for intervention zones ........................... 23
3.2.2. Sampling method for non-intervention zones ..................... 23
3.2.3. Identifying households .................................................... 24
3.3. Remarks Implied by the Data .............................................. 25
4. Evaluation Framework .............................................................. 28
4.1. Outline of the Framework ................................................... 28
4.2. Constructing the Consumption-based Criterion ................... 29
4.2.1. Consumption as a measure of welfare ............................... 29
4.2.2. Calculating the total consumption of the households .......... 30
4.2.3. Per-capita consumption versus adult equivalent consumption.............................................................................................. 31
4.3. Constructing the Cut-off Point ............................................ 34
4.4. Targeting Costs and Transfer Amount Assumed ................... 35
4.5. Comparison of Targeting Methods ....................................... 38
4.6. FGT Measures .................................................................... 42
4.6.1. Description of FGT measures ........................................... 42
4.6.2. The advantage of using FGT measures .............................. 43
5. Results ..................................................................................... 45
5.1. Results from the Budget with Average Transfer Amount ...... 45
5.2. Results from the Budget with Estimated Transfer Amount ... 52
5.3. Implications from the Results .............................................. 58
5.3.1. Necessity for the targeting at the household level .............. 59
5.3.2. Distribution of the transfer .............................................. 59
6. Further Analysis Regarding the Performance of Self-selection
Method and Census Method .......................................................... 616

6.1. The Rate of Targeting Errors .............................................. 61
6.2. Average Distance of the Targeted Households’ Consumption
Amount from the Cut-off Point Consumption Amount ................ 63
6.2.1. Average distance of consumption amount for poor households
targeted .................................................................................. 64
6.2.2. Average distance of consumption amount for non-poorhouseholds targeted .................................................................. 66
6.2.3. Implications from the analysis ......................................... 68
6.3. Density Estimation of the Consumption Amount .................. 69
6.4. Overall Discussions on the Self-selection Method versus the
Census Method .......................................................................... 74
7. Conclusion ............................................................................... 78
References .................................................................................... 83

7

1. Introduction

For social programs in developing countries, it is essential to carrylimited resources to the people who need them the most. In the context ofpoverty alleviation programs, the selection of beneficiaries (targeting) is
made to achieve the maximum result from a given budget. The term
targeting refers to a method in which the government or policy makers
choose to focus on certain groups of populations as objectives for the
program. This allows for scarce resources to be delivered to those whoneed them the most. Evaluating the mechanism of targeting methods andimproving it would help deliver the scarce resources to those who needsupport the most, and hence would help improve the poverty alleviation
programs.
This paper evaluates the targeting mechanism of Mexico’s poverty
alleviation program Oportunidades in urban areas of Mexico (theself-selection method). 1 The main objective of this paper is to compare
the effectiveness in reducing poverty of the self-selection method withthe alternative methods, especially the method which has been used by
Oportunidades in rural areas (the census method). 2

Oportunidades used the self-selection method when it started covering

1

Oportunidades was initiated under the name of Progresa in 1997, andthe name was changed to Oportunidades in 2002. In this paper, the name
Oportunidades is used to refer to the program throughout all the periods
from 1997, as well as to refer to the period after 2002. If the discussion
is focused on only the period before 2002, the name Progresa is used.
2 The terms “self-selection method” and “census method” are used forsimplicity in this paper, representing the targeting method applied forurban areas and rural areas, respectively.
8

urban areas in 2002. The self-selection method is increasingly being
used in the developing world (Alatas et al 2012).
Under the self-selection method, those who desire to participate in the
program need to go to the program module first. This means that under
the self-selection method some parts of the program procedures are left
to the potential beneficiaries, while under the census method, as analternative, the program officers find all the potential beneficiaries byvisiting each household in the targeted area. Therefore, under the
self-selection method less targeting costs are needed, but there is a riskthat some households cannot finish the procedure for some reason. As aresult, the self-selection method is predicted to target the poor
households less precisely than the census method. What should beconcerned is whether or not this imprecision of the self-selection method
overwhelms its advantage of lower targeting costs, compared with thecensus method.

Although Oportunidades put much importance on the evaluation process,and there are many studies evaluating the impacts of Oportunidades, it
seems that there has been relatively less discussion of its targetingmechanisms, especially the self-selection method. As the existingevaluations for the targeting of Oportunidades in urban areas, theinvestigations by Gutierrez et al (2003) and Coady and Parker (2005b)
are raised.
The former evaluated the targeting mechanism in semi-urban and urban
areas. They first calculated savings obtained by using the targeting9within the localities after the geographic targeting was done. 3 Then they
compared the effectiveness and costs of the method of taking surveys foreach household in the locality that was taken in 2001, and the method ofself-selection process that was taken in 2002, using the percentage of
eligiblehouseholds

that

were

detected(i.e.

incorporated).

The

shortcoming of this study is that they used the same criteria with
Oportunidadesforjudgingtheeligibility

of

the

householdsformeasuring the effectiveness of the targeting mechanism. The criteria thatjudges if the household is eligible or not should be compared withanother one, such as the consumption based measure that I will use here.
For analyzing the effectiveness of the self-selection process, they
comparedincremental

costs

for

each

locality.

Insteadofusing

incremental costs, I will compare the reduction in the poverty measuresusing several targeting methods, under a given budget. Applying thismethod, it becomes possible to compare more than two targeting methods,
and to see how much poverty would be reduced using each. Moreover,this method enables to take into account the extent of poverty ofhouseholds, not merely looking at if the household is poor or not.
The latter study focuses on analyzing the participation process of thehouseholds under the self-selection method. They analyze which factors
were significant for poor households to participate in the program under
the self-selection method, using regression analysis. Although this work
explains the mechanism of the self-selection well, we cannot know how
effective the targeting was in urban areas, compared to other methods.
3

The geographic targeting indicates the selection of eligible zones forthe program. In the case of Oportunidades, the eligible areas is selected
first by the geographic targeting, and then the targeting at the household
level is executed.
10Hence, this paper is expected to shed a light on some aspects of theself-selection method that these studies could not analyze. If theefficiency of the self-selection method turns out to be good enoughrelative to other targeting methods, it would result in a reduction in thetargeting costs for social programs. Although I focus on Mexico,
generallythedeveloping

self-selection

countries

andmethodisdeveloped

applicablecountries,

for

both

other

too.

This

paper

contributes to improving the self-selection method and reducing thetargeting costs for social programs worldwide.

For the analysis of this paper, I follow the methodology used by Skoufias
et al (1999) and its updated version Skoufias et al (2001), which
evaluated the targeting performance of Progresa in rural areas. The
methodology used in this paper is a comparison of poverty measures
after giving a transfer to the households classified as eligible for the
program under several targeting methods. The household welfare ismeasured in terms of consumption, while the criterion applied by
Oportunidades is mainly based on income. The budget is set equal for all
thetargetingmethods.

Thetargetingmethods

comparedaretheself-selection method, the census method, the simulated “perfect”
targeting based on consumption, and the uniform transfer. For thepoverty measures, I use the measures developed by Foster et al (1984),
or the FGT measures. The data used in the analysis comes from the
ENCELURB

2002

collected

byOportunidades

in

2002,

when

Oportunidades started covering the urban areas. 4 In order to test if there
4

ENCELURB is the abbreviation of the Spanish name of the survey:“Encuesta de Evaluación de Hogares Urbanos (Urban Household11are significant differences in the effectiveness between the different
targeting methods, the statistical significance of the differences is tested,which was not done in Skoufias et al (1999) nor in Skoufias et al (2001).

The main finding of this paper is that in terms of reducing poverty
indices, the effectiveness of the self-selection method is almost equal tothe effectiveness of the census method. However, as the lower targetingcosts of the self-selection method would allow for a smaller budget,
applying this method in urban areas seems meaningful, especially fordeveloping countries that have limited financial resources.

The structure of this paper is as follows. The following section 2 briefly
describes the Oportunidades program and its targeting methods. In
section 3, the dataset I use is described. Section 4 explains themethodology for the analysis, and section 5 shows its results. Given the
results in section 5, in section 6 further analyses are given about thetargeting performance of the self-selection method and census method.
Finally, section 7 gives concluding remarks.

Evaluation Survey)”.
12

2. The Program and Targeting Methods 5

In this part a brief description of the program Oportunidades and its
targeting methods is given. The following part 2.1 explains the programin general, and part 2.2 focuses especially on the description of targeting
method taken by Oportunidades. In part 2.3 a brief description of theanalysis of this paper is given.

2.1 Program Description

In August 1997, the Government of Mexico launched its flagship social
safety-net program, Progresa, in rural areas. The program has expanded
its coverage gradually, and in 2002 it entered small and medium urban
localities under its new name, Oportunidades. Oportunidades is a
Conditional Cash Transfer (CCT) program, where the transfer is given in
cash, conditioned on a household’s behavior, such as children’s
enrollment in school. The main goal of the program is to foster thehuman capital of households in a state of poverty, and hence to cut thecycle of poverty passed on to future generations, that is, the situationthat those who are born in a poor household are likely to be poor in the
future, too.

Oportunidades consists of three components: education, health andnutrition. The education component gives the household a monthly
scholarship, as long as their children satisfy a certain percent of school
attendance. The scholarship is given to students from the third grade of5

See Levy (2006) for more detailed description of the program design.
13

primary school to the third grade of high school up to 21 years of age. 6
In addition to the monthly scholarship, a transfer support to buy schoolsupplies is given, twice a year for students in primary school, and once a
year for students in secondary or high school. The amount of thescholarship increases as the grade of the student rises. Because girlstudents are more likely to drop out of school than boys, the amount of
the scholarship is higher for girls than boys in secondary and high school.
In order to prevent the households from having incentives to have morechildren so that the transfer amount would increase, a monthly limit of
the scholarship for one household is set. The amount of the transfer in2002 is presented in table 2.1, along with the transfer for nutritional
support described below.

The

healthandnutrition

components

areintegrated.

The

healthcomponent provides an opportunity for a monthly health check at the
public clinic. The nutrition component gives the households a monthly
cashtransferasasupport

for

their

nutrition,

conditionedonaccomplishing the monthly health check. The nutrition component also
provides a nutritional supplement to the households. The monthlytransfer amount in 2002 for the nutrition component is shown in table
2.1.
Table 2.1. The amount of cash transfer given by Oportunidades in
6

To be precise, the eligibility is limited to primary or secondary school
students under 18 years old. For high school students (including otherprofessional schools), the eligibility is limited to students between 14
and 21 years old. The estimated transfer amount calculated in theanalysis (described in section 4) follows the restrictions of these age
ranges.
14

July-December 2002
Components
Transfer amount
Nutritional
150
support
Educational
Monthly transfer
Support for school supplies
scholarship
First
SecondPrimary schooltransfertransferrd
3 grade100	
 th
4 grade
115
135
65
th
5 grade150
6 t h grade
200
Secondary school
Boys
Girls
st
1 grade290
310
nd
2 grade
310
340
250
rd
3 grade325
375
High school
Boys
Girls
st
1 grade490
565
nd
2 grade
525
600
250
rd
3 grade555
635
The transfer amount is presented in Mexican pesos. The scholarship forprimary and secondary school students is limited up to 915 Mexican
pesos monthly for one household. The scholarship for high school
students is limited up to 1550 Mexican pesos monthly, including thescholarship for primary or secondary students. These limits do notinclude the support for school supplies.
Source: Oportunidades website, Monto de los Apoyos, Histórico,
Oportunidades.
(http://www.oportunidades.gob.mx/Portal/wb/Web/oportunidades_historico) accessed on June 10, 2012.
2.2 Targeting Methods 7

Here I describe how the beneficiaries were selected under Oportunidades
7

For more information about the targeting method in urban areas, see,for example, Gutierrez et al (2003), Orozco and Hubert (2005) or Orozco(2007).
15

in urban areas. The process of the targeting is as follows.

2.2.1. Geographic targetingFirst, the geographic targeting which determines the beneficiary zones
was carried out. In order to estimate how many poor households wereconcentrated in the zone, a “marginality index” was constructed. This
index shows the aggregated socioeconomic living condition in thelocality and is calculated using the Geographic Information System(Sistema de Información Geográfica; SIG) provided by INEGI (Instituto
Nacional de Estadística, Geografía e Informática). 8
Originally, in rural areas the geographic targeting was done at the levelof localities. But as in urban areas, there is a large population and thepoor households are less concentrated than in rural areas, they set asmaller unit than the localities called Basic Geostatistical Areas (Áreas
Geoestadísticas Básicas; AGEB). These AGEBs were ranked by aconcentration of poor households and accessibility of social services thatare necessary for the program such as schools and public clinics, making
use of the marginality index. However, the efficiency for detecting thepoor households in this method turned out to be very low, and it wasnecessary to look for another way.
Thus, the geographic targeting was carried out at the level of street
blocks. As in the case of AGEBs, the street blocks were ranked
8

Marginality index consists of the following data: percentage ofilliterate population, percentage of dwellings without access to water,
percentage of dwellings without drainage, percentage of dwellings
without electricity, average of occupants per room, percentage of
dwellings with dirt floor and percentage of primary sector population.
16

according to the concentration of poor households, using the marginality
index.

2.2.2. Targeting at the household level

For the selected street blocks, the selection of the eligible households
was carried out. The eligibility status was decided according to theestimated income of the households, as a result of a proxy means test.
Originally in rural areas, this targeting was done by visiting eachhousehold in the selected locality one by one and taking a survey of thehousehold’s socioeconomic situation. According to the scores of theproxy means test, which was carried out as a part of the survey, theeligibility status of the household was decided. For convenience, I callthis method “census method”.
However, in urban areas where the poor households are less concentrated,taking this method was considered to be costly given its effectiveness.
Alternatively, the following “self-selection method” was used. In thismethod, those households in the selected street blocks who wish toparticipate in the program need to go to a program module, where asimple survey about their socioeconomic situation is completed. If ahousehold is considered possibly eligible for the program, a program
official will visit the household to take a more detailed survey on
another day. If the household passes this survey, then the household
needs to go once more to the module to finish procedures.
The problem under the self-selection method is, although this enables tosave the targeting costs by giving up visiting each household, that some17

households may be excluded from the program. For instance, thosehouseholds that do not know the program or do not have access to theprogram module systematically cannot participate in the program.
Because these households may be more likely to be one of the poorest,
the effectiveness for reducing poverty of each method is worth
comparing.

2.3. Outlook of the Analysis of This Paper

The main objective of this paper is to look at the efficiency of theself-selection method compared with other alternative targeting methods.
Especially, the comparison with the census method, which was altered by
the self-selection method, is paid attention to.
The “efficiency” of the targeting methods is solely measured in terms of
the effectiveness in reducing poverty. To measure this, I use the poverty
measures developed by Foster et al (1984), or the FGT measures, which
are described in section 4. I suppose that under a given budget, thehouseholds in the sample are given a transfer, according to theireligibility status under several targeting methods. Comparing theimprovement in FGT measures after the transfer under several targeting
methods, it becomes possible to compare the effectiveness in reducing
poverty. The targeting methods compared here are the self-selection
method used by Oportunidades in urban areas, the census method used by
Progresa in rural areas, the hypothetical “perfect” targeting based on
consumption, and the uniform transfer.
For the criterion to measure household welfare and hence to measure the18household’s eligibility status for the program, I use consumption of thehousehold,insteadof

Therefore,households

income,
with

which

is

used

by

Oportunidades.

consumption below

thecut-off

point(described in section 4) would be considered poor. This allows me tomeasure the household’s socioeconomic state neutrally apart from theeligibility status judged by Oportunidades. It also enables me to avoid
the possibility that there are errors in the sample itself.

From this analysis, one can find out how effective at reducing poverty
the self-selection method is, compared with the other targeting methods.
Comparing the self-selection method and the census method, thefeasibility of the self-selection method for urban areas will be shown. It
is expected that the self-selection method reduces poverty more
effectively than the census method, or at least equally, because in urban
areas applying the census method is predicted to be costly due to a lower
concentration of poor households. Next, the comparison between the
self-selection

methodandtheuniformtransfer

would

give

the

effectiveness of applying the targeting at the household level. In the caseof Oportunidades, the self-selection process is taken after the geographic
targeting is executed. As described in the following section, the sample I
use was taken after the geographic targeting had been made. Hencegiving a transfer equally to the whole sample means the results from thegeographic targeting. Comparing the differences in reduction in the FGT
measures between the self-selection and the uniform transfer, one cansee how effective taking the secondary targeting in addition to thegeographic targeting is.

19

3. Data Description

For the analysis in this paper, I basically rely on ENCELURB 2002
(Encuesta de Evaluación de Hogares urbanos 2002). This is the baseline
survey carried out for the evaluation process of Oportunidades in urban
areas, and is the only available data for evaluating Oportunidades inurban areas.

3.1. Data Contents

ENCELURB 2002 consists of three kinds of questionnaire data: (1)screeningquestionnaire;(2)

socioeconomic

characteristics

questionnaire; and (3) biologic specimens, anthropometric measurements,
breastfeeding, and food frequency questionnaire. According to INSP
(2005:14), these data was collected for the following aims:(1) The screening questionnaire was used to collect basic data on thesocioeconomic characteristics of households to determine their status ofeligibility in the Program and thus perform the selection of the final
sample of households. It was applied to all resident households except
for households in “neighboring street blocks” . As described later, thisquestionnaire was carried out as a part of the sampling process for
ENCELURB, thus this questionnaire process was applied to the wholesample of households at the first stage of the sampling. 9 This
questionnaire includes simple contents that judge the eligibility status of
9

As described later, some households living in the street blocks
neighboring the original treatment zones were included in the treatmentzone after this questionnaire. So the screening questionnaire does nothave data of these households.
20the households.
(2) The socioeconomic characteristics questionnaire is a multi-topic
questionnaire that collected information on the household and thehousehold members regarding sociodemographic, health, education,
occupation, consumption, expenditure, income, gender, addictions, andreproductive health aspects, among others. 1 0 It was applied to the final
evaluation sample of households that was selected based on the resultsfrom the screening questionnaire. Adequate informants to whom thisquestionnaire was applied were older than 15 years of age andknowledgeable of the household and household members’ data. In
addition, youngsters 10 to 21 years of age were directly surveyed toanswer the section on tobacco and alcohol use. Women 15 to 49 years ofage were also directly surveyed on the questions specific to reproductive
health.
(3)Thebiologic

specimens,

anthropometricmeasurements,

breastfeeding, and food frequency questionnaire was added in order tomeasure how the health and nutrition aspect of Oportunidades affected
the beneficiaries.

Taking into account the contents of ENCELURB 2002, the analysis in
this paper is based on the socioeconomic characteristics questionnairepart of the data. In particular, I use data on households’ consumption,
eligibility status judged by Oportunidades, and incorporation status into
the program.

10

Other variables consist of savings and debts, access to financial
services, experiences of participation in social programs in the past,
immigrations, and so on.
21

3.2. Sampling Method

What should be noted about ENCELURB 2002 (and other following
panel surveys) is that the sampling method employed by Oportunidades
is not random. Since ENCELURB 2002 was designed to evaluate theprogram’s impact, it is needed to measure the program effect, which isany difference in benefited households or individuals that is attributable
only to the program. So a comparison group that is as similar as possible
to the treatment group is needed. For this purpose, Progresa (the former
Oportunidades) set randomly assigned comparison group in rural areas. 1 1
However in urban areas, driven by ethics, financial, or practical reasons,random allocation of a comparison group similar to the one in rural areas
was not applied. To create a comparison group that is as similar as
possible to the treatment group, the following method of sampling wasused.
First, according to the Evaluation Advisory Group’s calculation and aprevious analysis of statistical power, the sample size that is necessary
to have statistically significant differences was calculated. These
calculations yielded a sample size of 14,000 households divided in fourgroups: 6000 eligible households enrolled in the program in intervention
zones; 2000 eligible households, but not incorporated into the program;
4000 almost eligible households (households with a deprivation scoreslightly lower than the cut-off point determining eligibility); and 2000

11

In rural areas, some randomly selected localities that satisfy the
program eligibility were deliberately not incorporated and set as acontrol group.
22

non-eligible households (INSP 2005, 7).

After the sample size was determined, which street blocks to be includedin the sample was examined in the following ways.

3.2.1. Sampling method for intervention zones

For the sample in the intervention zones, 149 street blocks were selectedby a probabilistic method through single-stage, stratified and cluster
sampling, using information from the 2000 Census. In this method thestreet blocks were divided into 6 strata by the number of eligible
households per street block. Street blocks for the sample were selectedwith a probability proportional to the number of eligible households inthem. All the street blocks with 50 or more eligible households were
included in the sample, while street blocks with no eligible households
were excluded.

3.2.2. Sampling method for non-intervention zones

To define the evaluation control group with consideration to ethical andfinancial factors, the Advisory Group proposed to define the control
sample by means of a matching or equalization of characteristics
techniques scheme. To this end, logistic regression models were
estimated using a series of socioeconomic characteristics at the street
block level. The nearest neighbor matching method was applied using
the estimated values. This is based on the proximity of the estimated23

values using the absolute value of the difference between them. This
procedure guaranteed the comparability between street blocks in at least
one series of observable variable. 1 2
Because non-intervention zones tend to have a smaller number of
eligible households than intervention zones, the blocks with moreeligible households were chosen with higher probability.

3.2.3. Identifying households

Then sample households were selected. In the selected blocks inintervention and non-intervention zones, census was taken for all
resident households to collect the necessary sociodemographic data forreclassifying the households according to their status of eligibility forthe program (eligible, almost eligible and not eligible). This involved
applying the screening questionnaire. According to this classification ofeligibility status, the sample of households to whom the socioeconomic
characteristics questionnaire was applied were selected within the fullsample.
After this process, because the number of households in the treatmentgroup was not statistically sufficient, additional households were
included by tracking administrative records and adding households
included in Oportunidades in the blocks neighboring those blocks thatwere already selected. Because these households were not included in
the screening questionnaire described above, households in these blocks
have no data on the screening questionnaire.

12

For details, see INSP (2005, 8).
24

In non-intervention zones, all household members were included in thesample.

After the final evaluation sample was formed, questionnaires on othertopics were carried out. Here again, all the households in the sample
werereclassified

accordingtothesocioeconomic

characteristics

questionnaire. Table 3.1 shows the final composition of the sample.

Table 3.1 The number of the households classified by their eligibility
status

Source: INSP (2005, 12)3.3. Remarks Implied by the DataAs

described

earlier,

ENCELURB

wasespeciallydesignedforevaluations of the program impact, and it was constructed so that thecomparison between incorporated households and not incorporated
households would be the easiest. Apparently this is not as suitable forthe evaluation of targeting results, because the structure of thesocioeconomic status of the sample should be as similar as possible to
the real society. However, as this is the only available data that shows25the incorporation status to Oportunidades and socioeconomic data
including consumption, it was necessary to use this dataset to carry out
the analysis. In reading this paper, the following limitations of thedataset should be kept in mind.

Note that the selection of the sample was carried out after the geographic
targeting. Because the samples were already “selected” in this data, poor
households are more likely to be concentrated than in the actual
population. In this sense, discussing the absolute value of targeting
results obtained by this sample makes no sense, and only comparing
several schemes within this sample would be feasible. For example, thepoverty rate in this sample is 58%, calculated using the eligibility status
estimated by Oportunidades. This poverty rate would be considered very
high, because more than a half of the households are in the state ofpoverty. But this number from ENCELURB never reflects the reality,
and is due to the concentration of the poor households in the sample. So
there is no point discussing an absolute number taken from this sample.
The analysis is only feasible if comparisons are taken only within thisENCELURB 2002 sample.

Although the analysis in this paper includes some restrictions, thefollowing things will be implied. First, it should be taken into accountthat the sample was taken after the geographic targeting. This assumes
that what will be compared here (self-selection method and method bytaking survey) shows which method is more effective for identifying theeligible households after eligible localities (or street blocks) are26

determined. Given the lower concentration of poor households in urban
areas compared with rural areas, the targeting at the household levelplayed an important role for the effectiveness of the program; 1 3analyzing the effectiveness of the targeting of the households after thegeographic targeting will be also meaningful.

Additionally, the characteristics of the communities included in thesample are urban (relatively) poor ones where poor households are lessconcentrated compared to rural communities. Although this result cannot
be applied to another community, the result here will imply that one
method can be more optimal for detecting the poor households, in such
localities that poor households do not concentrates so much, like urban
areas in developing countries or some areas in more developed countries
where poor households are relatively concentrated.

For the analysis here, I use the sample only in the treatment zones. By
including the sample in the control zones the effectiveness of thetargeting of Oportunidades would be underestimated because thesehouseholds in the control zones are intentionally not incorporated to theprogram, not because of the errors of targeting. The data needed for thisanalysis is available for 10,748 of the 11,563 households in thetreatment zones. The sample I use is made up entirely of these 10,748
households.

13

For instance, Gutierrez et al (2003) pointed out that with targeting on
household level the necessary budget to cover all the eligible households
was only 8% of the budget needed in the case of applying only thegeographic targeting.
27

4. Evaluation Framework

4.1. Outline of the Framework

The objective of the analysis here is to compare how much poverty is
reducedunder

severaldifferent

targetingmethods.

I

followthemethodology used in Skoufias et al (1999) and its revised version
Skoufias et al (2001) which evaluated the targeting results of Progresa inrural areas. Comparing the targeting results measured by one unique
number, we will be able to say which method is more effective. I use thepoverty measures developed by Foster et al (1984), or FGT measures,and I will see by how much these measures are reduced after the transfer
given by each targeting method, under a given budget. Targeting costs
and eligible households targeted differs across the various methods.
Giving the available funds (that is the total budget minus targeting costs)
to the beneficiary households identified by several methods, we can see
how much poverty is reduced by each targeting method. The targeting
methods compared here are (i) the self-selection method applied in
Oportunidades in urban areas, (ii) the method of taking a census for all
thehouseholds

in

the

locality

that

was

taken

in

Progresa

andOportunidades in rural and medium urban areas (census method), (iii)simulated perfect targeting based on households’ consumption, and (iv)
uniform transfer.

I construct a criterion based on consumption as a measure of households’
welfare or poverty status. This is in order to capture the households’
28welfare status more precisely as well as to compare the targeting
performance using a measure apart from the measure applied in
Oportunidades. So here the amount of consumption of the households is
considered to reflect the “true” economic state of the households.

4.2. Constructing the Consumption-based Criterion

4.2.1. Consumption as a measure of welfare

In an economic point of view, expenditure-based or consumption-based
standard-of-living measures are preferable to income-based measures(Skoufias et al 1999, Deaton and Zaidi 1999). The main theoreticalreason is that according to the permanent income theory of consumption,
estimates of current consumption are likely to provide a more reliable
estimate of the household’s permanent income (sustainable standard ofliving) than are estimates of current income. Current income may be
more volatile and subject to shocks from period to period, especially if
the household engages in agricultural or self-employment activities. In
contrast, consumption can be smoothed at least to some extent by saving
and borrowing.
There are some practical considerations. Income may exhibit higher
seasonal variability compared to consumption. To get an accurate
estimate of the average income over the year, several times of visits
would be required. Given that the data I use here is collected just once, it
is logical to assume that consumption measures households’ welfare
better than income.
29

4.2.2. Calculating the total consumption of the households

In ENCELURB 2002, the consumption data on a number of items isavailable. 1 4 The amount of consumption of each good is presented inmonetary units (Mexican pesos), and the monetary amount of lagged
payments or gifts received is also available. Although the data
aggregated to monthly total consumption is also available, I calculate it
myself by adding the consumption on all items because the aggregatedmonthly total consumption in ENCELURB 2002 is not available for somehouseholds, and how consumption is aggregated is not clear.

One thing should be kept in mind here. A shortcoming of theconsumption data in ENCELURB 2002 is that it does not have data on
consumption out of own production (auto-consumption). As a result, theconsumption of households that cultivate their own crops may beparticularly underestimated. For this reason in Skoufias et al (1999),
using the data by Progresa, ENCEL98M (Encuesta de Evaluación, 98
14

The items available here are categorized as: weekly food expenses (1)(tomatoes, onions, potatoes, chili, carrots, pumpkins, bananas, apples,oranges, other fruits, other vegetables, tortillas, white bread, sweet
bread, paste soup, beans, rice, pie and other cereals), weekly foodexpenses (2) (beef, chicken, pork, tuna, fish, eggs, milk, cheese, othermilk products, other animal products, soda, sugar, powdered drinks,
alcoholic drinks, coffee, vegetable oil, potato chips and other articles),other weekly expenses (matches, transport, newspaper, candles,
cigarettes, alcoholic drinks), monthly expenses (personal cleanlinessarticles, children cleanliness articles, house cleanliness articles, fuel,personal services, diversions, house’s rent), quarterly expenses (adultclothes, children or young people clothes, adult footwear, children
footwear, toys, books and music CDs, health), and annual expenses(school payment, furniture, repairs, utensils, household -electric, cars,other expenses).
30

Marzo; Evaluation Survey 98 March) was avoided. However, I dismiss
this effect of auto-consumption for the following reason. Unlike
Skoufias et al (1999) that evaluated the targeting in rural areas, I
exclusively focus on the targeting in urban areas, where there are
relatively few households cultivating their own crops. In the data ofsocioeconomic questionnaire in ENCELURB 2002, less than 5% ofhouseholds engage in agriculture (445 out of 10831 households).
Although there may be some households that cultivate their own crops
even if they do not engage in agriculture, in urban areas where the landfor cultivating crops is scarce, these households can be considerednegligible.

4.2.3. Per-capita consumption versus adult equivalent consumption

Once households’ monthly consumption is calculated, adjusting it byhousehold size is required. Two ways of measuring this are considered:per-capita consumption and adult equivalent consumption.

Per-capita consumption is obtained by dividing total consumption by the
number of people in the household. This measure is valid under thefollowing set of assumptions: (a) everyone in the household receives anequal allocation irrespective of age and gender; (b) everyone in thehousehold has the same needs irrespective of age and gender; and (c) the
cost for two (or three or more) people living together is the same as thecost of each person living separately (Skoufias et al 1999 in Appendix E).
Although the first assumption (a) is difficult to deny due to lack of31

information on consumption or income at the individual level, the second
and third assumptions (b) and (c) seem to not hold true. Not everyone
has the same needs and in particular their needs vary based on their age
and gender. Also there are economies of scale from living together,because family members benefit from each other’s consumption, or
because there are public goods that can be used by all family members at
no additional cost. However, for the matter of the third assumption (c),or the economies of scale, whether or not to take this into account wasunsettled (Skoufias et al 1999 in Appendix E). They estimated theeconomies of scale in both rural and urban areas in Mexico, and foundthat the estimated value was different from the values in most othercountries, though the estimate implied that economies of scale were
present and significant.

Considering this, I use Adult equivalent consumption, which takes into
account the various needs that vary based on gender and age. This iscalculated by dividing total consumption by the adult equivalent family
size (AEFS). AEFS is the family size adjusted to the number of adults.
This is constructed using different weights for different age and gendergroups as derived by the Instituto Nacional de Nutrición (1987). AEFS isconstructed using the following formula:

AEFS = (0.41)*children0-4 + (0.80)*children5-10 + (1.15)*males11-14+ (1.05)*females11-14 + (1.38)*males15-19 + (1.05)*females15-19 +(1.26)*males20-34 + (0.92)*females20-34 + (1.15)*males35-54 +(0.85)*females35-54 + (1.03)*males>=55 + (0.78)*females>=55.

Table 4.1 compares how the poverty status of the households differs
32depending on per-capita consumption or adult equivalent consumption.
Here my calculation assumes that 58% of the households are in state ofpoverty. This poverty line of 58% is the one applied in this paper, as
described later.
Table 4.1. The composition of the poor and non-poor households withper-capita consumption and adult equivalent consumption
Adult equivalent consumption(Poverty line = 58%)

Per-capita
consumption

non-poor

poor

total

nonpoor

4137

355

4492

poor

355

5901

6256

total

4492

6256

10748

Source: calculation by author.

This table shows that it does not matter so much which consumption
measure is applied, given that the disagreement between the two
measures are negligible: the number of the households that are classified
as poor with per-capita income but non-poor with adult equivalentconsumption (355 households) is equal to the number of the households
that are classified as non-poor with per-capita income but poor withadult equivalent consumption (355 households). Also this difference is
not so large: these numbers are less than 5% out of the total households.

Given this indifference of using per-capita consumption or adultequivalentconsumption

andtheoreticalsupport,adultequivalentconsumption is used as the measure of households’ economic state by33

which the households’ poverty status is judged here.

4.3. Constructing the Cut-off Point

Next, I set the poverty line that divides poor and non-poor in myconsumption based criterion. The question that arises here is how many
households should be included in the program, i.e. how many households
are classified as poor. For the comparison, the poverty line set here
should divide the same percentage of households as Oportunidades did(that is 58%). If my cut-off point is different from that used in
Oportunidades, my criterion will overestimate or underestimate thenumber of the poor households, and a precise evaluation for the targeting
result will become difficult.

As an alternative cut-off point, there is an absolute poverty line set by
CONEVAL (2009). According to them the poverty line in 2002 is 724
Mexican pesos of per-capita monthly income. 1 5 According to thispoverty line, 84% of the households in the sample fall into the poor
category. 1 6 This estimate is much higher than that of Oportunidades.
With

this

divergenceitisdifficulttocomparethemethodofOportunidades with alternative methods and obtain significant results. In
addition, it is possible that Oportunidades could not afford to include
84% of the households, even if this poverty line had been precise. I
15

The available data on the poverty line by CONEVAL (2009) is that in2000. So this poverty line in 2002 was calculated using the consumer
price index obtained from the data of the Bank of Mexico.
1 6 For this calculation I used the adult equivalent consumption insteadof the per-capita consumption that were described in part 4.2.3. If I use
the per-capita consumption, this poverty rate becomes 87%.
34

believe this divergence is due to the budgetary or practical constraints
on program coverage, and not the result of targeting errors. For thisreason I would rather use the same percentage used in Oportunidades forthe cut-off point.

According to the classification of the socioeconomic questionnaire of
ENCELURB 2002, Oportunidades classified 58% of the total households
as poor in the sample I use, and 49% were incorporated in the program. It
is believed that about 9% of the households did not participate in the
program for various reasons such as a lack of information about the
program, incapacity of going to the program module, unwillingness to beincorporated in the program and so on (Coady and Parker 2005b). Here I
assume that these households were planned to be incorporated in the
program, but could not participate. If Oportunidades had a perfectidentification of the poor households, these households would beincorporated to the program.

Therefore, I set the poverty line at 58%. With this cut-off point 58% of
the households are classified as poor, ordered by their consumption
level.

4.4. Targeting Costs and Transfer Amount Assumed

In this analysis I look at how much poverty is reduced after giving atransfer to those who are identified as eligible by the various targeting
methods, under a given budget. Here I assume the budget to be as similar35

as possible to the actual expenditure by Oportunidades. The budget is
calculated using the estimated transfer amount given to the beneficial
households and the targeting costs. The total budget is calculated asfollows:

Assumed budget = total transfer amount given to the incorporated
households + (average targeting costs per incorporated household)*(thenumber of the incorporated households).

For the targeting costs, I rely on the data presented in Gutierrez et al(2003). According to them, the costs of targeting under the self-selection
method were 104.90 Mexican pesos per identified eligible household.
Since the costs of targeting are one-time costs that are paid only at thestart of the program, I distribute these costs equally over time by
dividing them by 36 (the total number of months of the duration of the
program).

To estimate the total transfer amount, I calculated two types of transfer
amount. The first transfer amount is obtained by using the averagetransfer amount per incorporated household (I call this type of transfer
amount “the average transfer amount”). According to Angelucci andAttanasio (2008), the average transfer amount received by one householdat the time of targeting was 316 Mexican pesos. 1 7 Among the households
in the sample of 10,748 households, 5,265 households were incorporated
into the program by the method of self-selection. The total budget

17This figure was calculated based on the average transfer amount in2003. The transfer amount was adjusted in 2002 pesos, using consumer
price index.
36

assumed here becomes:

Assumed budget = 316*5,265 + (104.90/36)*5,265 = 1,678,081.6(Mexican pesos).

The second transfer amount is estimated following the method of
distributing the actual transfer amount applied by Oportunidades (I callthis type of transfer amount “the estimated transfer amount”). As
described in the section 2, Oportunidades gives transfer conditioning theschool attendance and health check. The components of the cash transfer
are the nutritional and educational support. Receiving the nutritional
support is conditioned on executing the medical check, and theeducational support is conditioned on school attendance. As the transfer
amount varies depending on the household member’s school level andgrade, the data on household members’ schooling situation was utilized.
However, there are a number of errors in these data in ENCELURB 2002,
which required that I make corrections to this data. 1 8 I assume that if theperson is actually going to school, the person is receiving the
educational support. I also assume that all the beneficiary households arereceiving the nutritional component. 1 9 Giving these transfers to those18

The correction was made based on the following principles. First, I
assumed that there no one skipped the grade. The schooling level of
those who did not satisfy the minimum age was modified to theappropriate level. Second, some members in the sample had inadequate
set of the schooling level and grade, and these data were corrected. For
example, there was a person at the 4 t h grade in the secondary school, but
as the years of education in Mexico are 6, 3, and 3 for primary,secondary, and high school respectively, the person should be at the 4 t hgrade of the primary school. However, as these corrections never can bedone perfectly, I suggest that the reader refer to the results obtained byboth two simulated budgets above.
1 9 For details of the transfer amount, see table 2.1 in section 2.
37

who were incorporated in the program, and summing these transfers, thetotal transfer amount turns out to be 1,905,819 Mexican pesos. Adding
the targeting costs, the total budget yields:

Assumed budget(Mexican pesos).

=

1,905,819+(104.90/36)*5265=

1,921,160.6

In the following section, how to distribute these budgets according to theassumed transfer scheme is explained.

4.5. Comparison of Targeting Methods

Here I describe the targeting methods compared, and which households
are classified as poor under each method. Note that in the socioeconomic
questionnaire of ENCELURB 2002, data on the status of program
incorporation of the households and the poverty status judged by
Oportunidades are available. Which households are given the transfer
depends on this data. The targeting methods compared are the following:Self-selection method

As described earlier, this method was actually applied by Oportunidades
in urban areas. This method leaves some procedures to the households.
Those who desire to be accepted into the program need to go to theprogram module in order to be examined their eligibility status. If they
are proven to be eligible in the module, means testing at the house iscarried out. If they are also judged to be eligible through this means test,38

they are incorporated to the program.
With this method the 5,265 households that were actually incorporated
into the program receive the transfer. As was done in the previous
section, I assume the average transfer amount given to these households
to be 316 Mexican pesos for the first assumed budget, and the amountthat depends on how many the household members enrolled in school forthe second budget. In the analysis, it is assumed that the full amount ofthetransferisconsumed.

Therefore,

thetargetedhouseholds’

consumption amount after the transfer is calculated by adding thistransfer amount to the households’ consumption. Then poverty measures
after the transfer is calculated using the adult equivalent consumption
amount of the households, which is calculated from the households’
consumption amount after the transfer. This calculation is applied to theother targeting methods in the analysis as well.

Census method

This method was applied by Progresa and Oportunidades in rural andsemi-urban areas. In this method, all the households in the eligible
locality are interviewed for the census, and eligible households are asked
to incorporate to the program, while the self-selection method leaves thisprocedure to the households by requiring them to go to the program
module. As described earlier, under the self-selection method, somehouseholds were not incorporated into the program even if they wereclassified as poor, mainly because these households could not complete
the procedure by themselves. Since in this method the households do not39need to go to the module to complete the procedures, more households
are likely to participate in the program than the self-selection method.
For the analysis, I assume that all the households classified as eligible
by Oportunidades will also receive the transfer under this method. The
households that were classified as non-poor but incorporated into theprogram will also receive the transfer, as targeting errors. Those who
receive the transfer in this method will be the 5,265 households actually
incorporated into the program, and the 2,174 households that wereclassified as eligible but did not participate for some reason.
I assume that the total budget is equally distributed among thesehouseholds. 2 0 The number of households that are supposed to receive
the transfer in this method is 7,439. Hence the budget per eligible
household for the first budget will be the total budget of 1,679,081.6
Mexican pesos divided by 7,439 households: 225.71 Mexican pesos. For
the second budget, this becomes 258.26 Mexican pesos. And according
to Gutierrez et al (2003), the targeting costs for this method were 154.90
Mexican pesos per one eligible household. Dividing these targeting costs
by 36 months and subtracting them from the budget per household, thetransfer amount distributed to each household will be 221.41 Mexican
pesos for the first budget and 253.95 Mexican pesos for the second
budget.

Targeting based on consumption

20

In fact, the transfer amount received in rural and semi-urban areas
also was determined as was done in urban areas, but applying thisamount here was not possible because for the analysis the total budget
needs to be the same for all the targeting methods.
40

For a comparison, I add this method as perfect targeting. In this method,
I assume that the eligible households are selected based on the “perfect”
measure of household welfare: consumption. With respect to household
consumption, the bottom 58% of households will receive the transfer.
In order for the program to estimate the household’s consumption
perfectly, it is logical to think that at least taking the census for all thehouseholds within the eligible locality is required. For the targetingcosts of this method, I assume the same amount as with the census
method: 154.90 Mexican pesos per eligible households divided by 36
months.
The number of households who receive the transfer in this method is6,256. As in the case of the census method, assuming the total budget isequally

spent

for

each

eligiblehousehold,thetransferamountdistributed to each household will be 263.97 Mexican pesos for the firstbudget, and 302.79 Mexican pesos for the second budget.

Uniform transfer

This method distributes the transfer equally to all the 10,748 households
in the sample. The transfer amount will be 156.22 Mexican pesos for the
first budget and 178.75 Mexican pesos for the second budget. I call thismethod “uniform transfer” for convenience, however, the sample
households are already selected by Oportunidades after the geographic
targeting, and the households in the sample can be considered a subgroupof all the households chosen by the geographic targeting. This transfer
should be considered the result of the geographic targeting.
41

4.6. FGT Measures

Next, I describe the poverty measure used here, FGT family of poverty
measures.

4.6.1. Description of FGT measures

FGT measures are denoted by the formula: 2 1
1
𝑃 𝛼 =   𝑁

!

1 𝑐! < 𝑧!!!

𝑧 −    𝑐! !
,𝑧

where 𝑁 is the total number of households, 𝑐! is the per capita
consumption (or income) of the 𝑖’th household, 𝑧 is the consumption(or income) at the poverty line, and 𝛼 is the weight attached to theseverity of household poverty (or the distance from the poverty line).
1 ∙ is an operator which takes value 1 if the condition∙ is true and 0

otherwise. When 𝛼 = 0, the FGT measure becomes to the HeadcountIndex, or the percentage of the households that are below the poverty
line.

When 𝛼 = 1, the FGT measure gives the poverty gap, a measure of theaverage poverty. I will give one example to explain how this poverty gap
differs from the poverty line. Let’s consider a society that consists of
two households, where the poverty line is set at $100 (hence thehouseholds with consumption or income below $100 are classified aspoor). Suppose two cases. In the first case, both households consume $90.

21

For more description, see Skoufias et al (1999, 5).
42In the second case, both households consume $10. It is easy to conclude
that the second case is worse off in terms of the poverty. However, onlywith the concept of the poverty line, we cannot explain why the second isworse, because in both cases the number of the poor households is thesame. The poverty gap index is 0.1 for the first case, and 0.9 for thesecond. The poverty gap reflects the average difference between thepoverty line and the income or consumption of the poor households.

When 𝛼 > 1, the FGT measure becomes the Severity of Poverty index. I
use this index with 𝛼 = 2 . This measure assigns more weight tohouseholds that are farther away from the poverty line and thus in more
severe poverty. Let’s consider extreme situations for the above example.
Now, in the first situation, both households can spend $50 for theirconsumption, and in the second situation one household consumes $90
and the other household consumes $10. In these situations, the poverty
gaps are the same. However, it can be said that the poverty in the second
situation is harder to overcome, because there is a household sufferingfrom more extreme poverty. The Severity of Poverty index reflects thesesituations. The index is 0.25 for the first situation, and 0.41 for thesecond.

For each value of 𝛼, the lower the index is, the less poverty there is. So
the objective of the poverty alleviation programs is formulated as
reducing these FGT poverty indices.

4.6.2. The advantage of using FGT measures
43

FGT measures are highly regarded consumption (or income) based
measures of poverty because these measures overcome problems thatarise with other poverty measures. Suppose that poverty lines divide thepoor and non-poor, irrespective to how poor the people are. The poverty
gap measures how far the poor people are away from the poverty line,
but with the poverty gap we still cannot know the whole sketch of thepoverty. Raj (1998, 290-292) explains the problems with these measures
citing the Pigou-Dalton transfer principle to the measurement ofpoverty:

Weak Transfers Principle. A transfer of income from any person below
the poverty line to anyone less poor, while keeping the set of poor
unchanged, must raise poverty.

Both the Headcount Index and the poverty gap fail to satisfy thiscriterion. Suppose that the income is transferred to a household inpoverty from a poorer household, so that the income of the household
will not exceed the poverty line. According to the weak transfers
principle, this transfer increases poverty, but both indices stay the same.
This type of increase in poverty is only captured utilizing the severity ofpoverty index, 𝑃 2

in the FGT measure.

Note that it is not possible to compare different indices with different
values of 𝛼. The comparison is only possible for several indices with thesame value of 𝛼. In this analysis, I compare carefully each index withthe same value of 𝛼, according to each targeting method.
44

5. Results

In the analysis for this paper the FGT measures of the sample in
ENCELURB 2002 are derived, giving a transfer to eligible households
under various targeting methods, under a given budget. The results areshown separately for the budgets that were described in the part 4.4. In
the part 5.1, the FGT measures for the first budget are shown, whichwere estimated using the average transfer amount per incorporated
household; the second budget that was estimated using the estimatedactual transfer amount is shown in part 5.2. For each result, thedifference of the FGT measure from the measure without transfer ispresented. It is useful to compare these differences within the samevalue of 𝛼, in order to see how effective the targeting method is. For
both results, the hypothesis testing is done to check the statisticalsignificance of the differences in the measures from two different
targeting methods. Note that it is not possible to compare the figuresfrom the different budget, but it is suggested to ref er to the observations
from the results of both budgets. Also note that when considering the
FGT measures, it is impossible to compare the figures across different
values of 𝛼. And finally, discussions for these results are made in part
5.3.

5.1. Results from the Budget with Average Transfer AmountAs described in the part 4.4, the budget with average transfer amount
was estimated using the average transfer amount of 316 Mexican pesos
45per incorporated household. The total budget is 1,678,081.6 Mexican
pesos, and the amount calculated by subtracting the targeting costs fromthe budget was equally distributed to those households who areclassified as eligible under the targeting methods.

The FGT measures before and after the transfer are presented in Table
5.1. For all the indices, a smaller number of the index means less poverty.
The targeting method that produces the smallest index number isconsidered to be the most effective at alleviating poverty. Note that theheadcount index (expressed by P(0)) of the state without transfer isnecessarily equal to the poverty line (or cut-off point) I set in part 4.3,
that is 58%. P(1) and P(2) indicate the poverty gap and the severity
index respectively. As discussed in part 4.5, these indices are thought torepresent the state of poverty in a society more accurately than theheadcount index. Thus, more importance should be put on the reductionof these indices. Table 5.1 reports the FGT measures without transfer inrow 1; of the self-selection method in row 2; of the census method in row
3; of the consumption based targeting in row 4; and of the uniform
transfer in row 5. The numbers in the parenthesis indicate the difference
from the index without transfer under the same value o f 𝛼.

Table 5.1. FGT measures with or without transfer for each targeting
method under the first budget
P(0)
P(2)
P(1)
Targeting method(Headcount(Severity(Poverty gap)
index)
index)
Without transfer (1)
0.582
0.215
0.106
Self-selection (2)
0.508
0.163
0.072
46

(-0.074)(-0.052)(-0.034)
0.510
0.162
0.071
Census (3)(-0.072)(-0.053)(-0.036)
0.462
0.139
0.058
Consumption based (4)(-0.120)(-0.076)(-0.048)
0.511
0.167
0.075
Uniform transfer (5)(-0.071)(-0.048)(-0.032)
The numbers without parentheses indicate the FGT measures withouttransfer or with transfer under the targeting method. The numbers in theparentheses indicates the difference from the FGT measure withouttransfer.
Source: calculation by author.

The intuitive interpretation of the figures in table 5.1 is as follows.
The measures of P(0) indicate the headcount ratio. In the case withouttransfer, 58.2% of the households are classified as poor, and this number
is necessarily equal to the poverty line I set. The measures of P(1)indicate the poverty gap. In the case without transfer the figure of 0.215
means the average difference of the poor household’s consumption fromthe consumption amount at the poverty line. The measures of P(2)indicate the severity index. The interpretation of these measures issimilar to P(1), but more weights are put on the households whose
consumption amount is far away from the poverty line.
Looking at any FGT measure with transfer (row 2-5), the figures are
smaller. Hence it is shown that the FGT measures are reduced by givinga transfer under all the targeting methods. However, it is needed to test
the statistical significance of this reduction. I conducted the hypothesis
testing for the differences between the original FGT measures withouttransfer and the FGT measures after the transfer under each targeting
method. Also, in order to make the comparison across two different
47

targeting methods, the differences in the FGT measures between two
different targeting methods are tested.

The differences in the FGT measures from two different targeting
methods and their t-statistics are shown. It reports the difference
between the FGT measures without transfer and the measures under thetargeting methods. Here, the FGT measures without transfer is assumed
to be exogenous, 2 2 and the t-statistics for the difference is calculatedusing the standard error of the other measure that is compared with themeasures without transfer. This shows whether the reduction in the FGT
measure is statistically significant or not by giving a transfer under
several targeting methods. For all the tables in this section t-statistics ofthe differences are shown in parentheses.

Table 5.2. No transfer versus other targeting methods under the firstbudgetTargeting method
P(0)
P(1)
P(2)
-0.074***-0.052***-0.034***Self-selection (1)(-15.396)(-25.250)(-27.481)-0.072***-0.053***-0.036***
Census (2)(-14.992)(-25.919)(-29.527)
Consumption based-0.12***-0.076***-0.048***(3)(-24.918)(-39.750)(-44.817)-0.071***-0.048***-0.032***
Uniform transfer (4)(-14.646)(-22.992)(-25.472)
The figures without parentheses indicate the difference of the FGT
22

For all the calculation of testing the statistic significance of thedifferences between two different FGT measures, I assume that one
measure is exogenously given, and the statistical significance of thedifference from that measure is calculated using the standard error of theother measure, instead of looking at the standard error of the difference
itself, due to availability of the necessary information.
48

measures between no transfer and othert-statistics are shown in parentheses.
*: indicates significant at the 10% level.
**: indicates significant at the 5% level.
***: indicates significant at the 1% level.
Source: calculation by author.

targetingmethods.

The

Table 5.2 shows that giving the transfer under any of the targeting
methods significantly reduced the FGT measures obtained withouttransfer. This shows that giving a transfer does reduce poverty, as
expected.

Next, the differences between the FGT measures after the transfer for
two different targeting methods and their t-statistics are shown. This
tests if there are large enough differences in efficiency across several
targeting methods. As was done for the measures without transfer above,it is assumed that the measures from one targeting method are exogenous
and t-statistics are calculated using the standard error of the measuresfrom the other targeting method. This means that for one combination of
two targeting methods, there exist two different t-statistics. However,here I present one t-statistic per combination of targeting methods for
simplicity. The whole analysis is not shown here but in fact it is foundthat the two different t-statistics from one combination of targeting
methods make little difference. The following table 5.3 shows thedifferences in the FGT measures between two different targeting
methods. In panel A the differences between the self-selection method
and the other methods are shown, assuming that the FGT measures fromthe self-selection method are exogenous. In panel B, similarly, the49differences between the census method and other methods are shown,
except for the self-selection method that is already compared in panel A,
assuming the measures from the census method are exogenous. Finally,in panel C the differences between the consumption based targeting andthe uniform transfer are shown. In panel C the measures from theconsumption based targeting are assumed to be exogenous. For eachdifference, the t-statistics are given in parentheses.

Table 5.3. Comparison of the FGT measures from several targeting
methods under the first budget
Targeting method
P(0)
P(1)
P(2)
Panel A. Self-selection method versus the other targeting methods
0.002-0.001-0.001
Census (1)(0.405)(-0.375)(-1.073)-0.046***-0.024***-0.014***
Consumption based (2)(-9.480)(-12.448)(-12.849)
0.004
0.004*
0.002**
Uniform transfer (3)(0.753)(1.949)(1.986)
Panel B. Census method versus consumption based targeting and uniform
transfer-0.048***-0.023***-0.013***
Consumption based (1)(-9.886)(-12.047)(-11.643)
0.002
0.005**
0.004***
Uniform transfer (2)(0.347)(2.315)(3.022)
Panel C. Consumption based targeting versus uniform transfer
0.049***
0.028***
0.016***
Uniform transfer (1)(10.207)(13.320)(13.022)
The figures without parentheses indicate the difference of the FGT
measures between the consumption based targeting and other targeting
methods. The t-statistics are shown in parentheses.
*: indicates significant at the 10% level.
**: indicates significant at the 5% level.
***: indicates significant at the 1% level.
Source: calculation by author.
50

Table 5.3 reveals that not all differences are statistically significant.
This means that for some combination of targeting methods, theirefficiency for reducing poverty is almost the same. The following
conclusions can be made from these tables.
First, looking at row 2 in panel A, row 1 in panel B and panel C, against
all other targeting methods, consumption based “perfect” targetingshows its significant effectiveness. Second, more importantly, from row
1 in panel A, none of the differences between the measures under the
self-selection

methodandthecensusmethodwere

statisticallysignificant. This suggests that the effectiveness for reducing poverty ofthe self-selection method and the census method can be consideredalmost equal. Third, from row 3 in panel A, the differences between theuniform transfer and the self-selection are significant for P(2) at the 5%
level, and P(1) at the 10% level. From row 2 in panel B also thedifferences between the uniform transfer and the census method are
significant for P(1) at the 5% level, and P(2) at the 1% level. Taking into
account that the sample is taken after the geographic targeting was
executed, in this analysis the uniform transfer can be considered tocorrespond to the samples derived by the geographic targeting. Hence
this supports that taking the secondary targeting after the geographic
targeting is effective.
In summary, if preferences of targeting method are based on poverty
reduction, the self-selection method and the census method are preferred
equally and only second to the perfect targeting based on consumption.
The uniform transfer is found to be the least effective method.
51

5.2. Results from the Budget with Estimated Transfer AmountIn this part, the results from the budget estimated by calculating theactual transfer amount are shown. The estimated transfer amount wascalculated using the data on schooling situation of the household
members in my sample. The resulting average transfer amount perhousehold is approximately 362 Mexican pesos in this case. This yielded
the total budget of 1,921,160.6 Mexican pesos. As was done in 5.1, theamount calculated by subtracting the targeting costs from the budget was
equally distributed to those households who are classified as eligible
under the targeting methods. Note that because of the necessity ofmaintaining the same budget amount for all the targeting methods, theway of distributing the transfer according to the schooling information is
not applicable to the other methods. 2 3 In order to make the comparison
with the previous results and to see the effect on alleviating poverty ofthe distribution of the transfer, I added the measures with the transfer
equally

distributed

to

the

incorporated

households

under

the

self-selection as well as with the transfer distributed following theactual transfer amount. For convenience, I call this way of distributing“self-selection with average transfer amount”. Table 5.4 shows theresults for this budget.
23

If I apply this way of distributing the transfer to the other methods,
the total budget spent will not be the same with the budget spent under
the self-selection method because the number of the households
receiving the transfer differs on the targeting method. This brings a
problem on how to adjust the amount of the budget, where I will beforced to reduce or raise the amount of transfer artificially. To discuss
this problem, further discussions are needed, and hence I avoid using
this way of distributing for the other methods.
52

Table 5.4. FGT measures with or without transfer for each targeting
method under the budget with estimated transfer amount
Targeting method

P(0)

P(1)

P(2)

Without transfer (1)Self-selection(with estimated transfer
amount) (2)Self-selection(with average transfer
amount) (3)

0.582

0.215

0.106

0.508
(-0.074)

0.160(-0.054)0.071(-0.035)0.498(-0.084)

0.156(-0.058)

0.069(-0.038)

0.499
0.155
0.067
(-0.083)(-0.060)(-0.039)
0.444
0.130
0.053
Consumption based (5)(-0.138)(-0.085)(-0.053)
0.511
0.167
0.075
Uniform transfer (6)(-0.071)(-0.048)(-0.032)
The numbers without parentheses indicate the FGT measures withouttransfer or with transfer under the targeting method. The numbers in theparentheses indicates the difference from the FGT measure withouttransfer.
Source: calculation by author.
Census (4)From table 5.4, one can say that the measures without transfer (in row 1)
are necessarily equal to the ones shown in table 5.1. Given the larger
budget compared with the previous part, in this table the measures after
the transfer are reduced more than in table 5.1. Note that the difference
between the two measures from the self-selection method in row 1 and 2
depends only on the way of distributing the transfer. Under the
self-selection method with the estimated transfer amount (row 1) the
budget is distributed according to the household member’s schooling53

status. On the other hand, under the self-selection method with theaverage transfer amount the budget is distributed equally to thehouseholds incorporated in the program, as was done in the first budget.
Under the other targeting methods, the budget is distributed equally tothe households classified as eligible under the targeting method.

Now, as was done in the previous part, the statistical testing is
conducted for this result. First, the differences between the FGT
measures without transfer and other targeting methods are shown in table
5.5.

Table 5.5. No transfer versus other targeting methods under the second
budgetTargeting method
P(0)
P(1)
P(2)
Self-selection-0.074***-0.054***-0.035***(with estimated transfer(-15.396)(-26.449)(-28.283)
amount) (1)Self-selection-0.084***-0.058***-0.038***(with average transfer(-17.477)(-28.682)(-30.946)
amount) (2)-0.083***-0.060***-0.039***Census (3)(-17.130)(-29.806)(-33.936)-0.138***-0.085***-0.053***
Consumption based (4)(-28.829)(-45.991)(-52.168)-0.071***-0.048***-0.032***
Uniform transfer (5)(-14.646)(-22.992)(-25.472)
The figures without parentheses indicate the difference of the FGT
measures between no transfer and other targeting methods. The
t-statistics are shown in parentheses.
*: indicates significant at the 10% level.
**: indicates significant at the 5% level.
***: indicates significant at the 1% level.
54

Source: calculation by author.

Table 5.5 shows that all the differences are statistically significant, too.
It is shown that also with this budget, giving a transfer does reduce thepoverty.

Finally, the differences between the FGT measures after the transfer for
two different targeting methods and their t-statistics for this budget are
shown. As was done in the previous part, only one t-statistic is
calculated per combination of targeting methods. The whole analysis is
not shown here but there are no large differences in the statisticalsignificance between the two different t-statistics for this budget, either.
The following table 5.6 shows the differences in the FGT measures
between two different targeting methods. In panel A, the differences
between the self-selection method with the estimated transfer amountand the other methods are shown, assuming that the FGT measures fromthe self-selection method with the estimated transfer amount areexogenous.

In

panel

B,

similarly,

thedifferences

betweentheself-selection method with the average transfer amount and othermethods are shown, except for the self-selection method with theestimated transfer amount that is already compared in panel A, assuming
the measures from the self-selection method with the average transfer
amount are exogenous. In panel C the differences between the census
method and the consumption based targeting, and the differences
between the census method and the uniform transfer are shown. In panel
C the measures from the census method are assumed to be exogenous.
55

Finally, in panel D, the differences between the consumption basedtargeting and the uniform transfer are shown, assuming the measures
from

the

consumption

basedtargetingareexogenous.

For

each

difference, the t-statistics are given in parentheses.

Table 5.6. Comparison of the FGT measures from several targeting
methods under the second budget
Targeting method
P(0)
P(1)
P(2)
Panel A. Self-selection method with the estimated transfer amountversus the other targeting methods
Self-selection-0.010**-0.004*-0.002*(with average transfer(-2.083)(-1.903)(-1.916)
amount) (1)-0.008*-0.005***-0.004***
Census (2)(-1.736)(-2.594)(-3.560)-0.064***-0.031***-0.018***Consumption based (3)(-13.337)(-16.541)(-17.359)
0.004
0.006***
0.004***
Uniform transfer (4)(0.753)(3.032)(2.856)
Panel B. Self-selection method with the average transfer amount versus
the census method, the consumption based targeting and the uniform
transfer
0.002-0.001-0.002
Census (1)(0.347)(-0.660)(-1.555)-0.054***-0.027***-0.015***
Consumption based (2)(-11.240)(-14.448)(-15.061)
0.014***
0.010***
0.006***
Uniform transfer (3)(2.836)(4.881)(4.726)
Panel C. The census method versus the consumption based targeting andthe uniform transfer-0.056***-0.025***-0.014***
Consumption based (1)(-11.590)(-13.734)(-13.279)
0.012**
0.012***
0.008***
Uniform transfer (2)(2.489)(5.512)(6.176)
Panel D. The consumption based targeting versus the uniform transfer
Uniform transfer (1)0.068***
0.037***
0.021***
56

(14.009)(17.648)(16.983)
The figures without parentheses indicate the difference of the FGT
measures between the consumption based targeting and other targeting
methods. The t-statistics are shown in parentheses.
*: indicates significant at the 10% level.
**: indicates significant at the 5% level.
***: indicates significant at the 1% level.
Source: calculation by author.

The results for this budget are quite similar to the previous results forthe other budget. First, from row 3 in panel A, row 2 in panel B, row 1 inpanel C and panel D, the “perfect” targeting based on consumption issignificantly more effective than the other targeting methods, as it mustbe. Second, from row 2 in panel A and row 1 in panel B, the differences
between the measures under the self-selection method and the census
method are significant only if the estimated transfer amount is applied in
the former method. Using the same method of distributing the budget,the differences are not significant between the self-selection method andthe census method. Again the effectiveness for reducing poverty of theself-selection method and the census method is shown to be almost equal.
Third, as there is a larger budget compared with that of the previous
section, the advantage of applying the secondary targeting (which means
targeting at the household level made after the geographic targeting)becomes more apparent than in the case of the other budget. From row 4
in panel A, row 3 in panel B and row 2 in panel C, all the differences in
the measures between the uniform transfer and the secondary targeting(i.e. the self-selection method and the census method) are shown to besignificant, except for P(0) under the self-selection method with theestimated transfer amount in row 4 in panel A. Compared with the results
57

in the previous part, the differences and their statistical significances
seem to be larger in this budget than in the previous budget. Thissuggests that using a secondary targeting method is more effective whena larger budget is given.
In addition to these results, the measures from different ways of
distributingthetransferarecomparable

with

this

budget.

Thedifferences in row 1 in panel A are the differences depending on how thebudge is distributed, that is, applying the way of distributing ofOportunidades or distributing equally. From these measures, one can saythat distributing the budget equally is slightly more effective, though thedifferences are significant only at the 10% level.

Summarizing the results for the second budget, the perfect targeting
based

on

consumption

is

the

most

effective,

followedbytheself-selection method with the average transfer amount and the census
method, which have an almost equal effect. The self-selection methodwith the estimated transfer amount comes in fourth place, and theuniform transfer is found to be the least effective.

5.3. Implications from the Results

By summarizing the results in part 5.1 and 5.2, the following three
things are observed. First, the self-selection method and the census
method reduced poverty almost equally. Second, the uniform transfer
was less effective than both the census method and the self-selection
method. Third, distributing the budget equally to the eligible households
58reduced poverty slightly more than the transfer amount applied by
Oportunidades. For the first observation, further analysis will be givenin the following section 6. Here, discussions are made for the latter two
points.

5.3.1. Necessity for the targeting at the household level

Comparingtheresults

concerningtheuniformtransferandtheself-selection method or the census method, the uniform transfer is
found

to

be

less

effective

than

the

two

other

methods.

The

ineffectiveness of the uniform transfer compared with the self-selection
method or the census method supports the necessity for targeting at thehousehold level. The same conclusion was also drawn by Gutierrez et al(2003). This result would support the targeting itself for social programs,
especially for countries with limited budgets. With targeting, the limited
resources are distributed more effectively to those who need them themost rather than without targeting. Targeting at smaller levels seems
better for poverty alleviation.
Comparing the results from the two different budgets, the differences
between the uniform transfer and the self-selection method or the census
method are even larger under lager budgets. This would suggest that
there is greater necessity of taking the secondary targeting if the
program coverage is wider, and hence a larger budget is expected.

5.3.2. Distribution of the transfer

59

By comparing the results for the second budget concerning theself-selection method with the estimated transfer amount and with thetransfer distributed equally, the latter distribution of the transfer
reduced more FGT measures. Although the differences are only
significant at 10% for P(1) and P(2), this suggests that distributing thetransfer equally is slightly more effective for reducing poverty. The
transfer amount applied by Oportunidades highly depends on thehousehold’s composition, and the households with more school age
children are likely to receive a larger transfer payment. The results
suggest that the households with more children are not necessarily more
likely to be extremely poor.

60

6. Further Analysis Regarding the Performance of Self-selection
Method and Census Method

The results in part 5.1 and 5.2 reveal that the self-selection method (withaverage transfer amount in the case of the second budget) and the census
method reduce the poverty indices by almost equal amount. This result is
contrary to what was expected. In fact, Oportunidades abandoned thecensusmethod

in

urban

areas

where

poor

households

are

less

concentrated because applying the census method was predicted to berelatively more costly against its effectiveness than the self-selection
method.
Then, is it logical to conclude that applying either self-selection method
or census method is indifferent? To answer this, I need to look moreclosely at the targeting performance of the two methods. In this section,
I try to compare them by executing other additional analyses.

6.1. The Rate of Targeting Errors

One way to evaluate the performance of targeting methods is to calculate
the

rates

of

“under-coverage”

and“leakage”

under

both

the

self-selection and census method. Under-coverage rate is defined as the
proportion of poor households who are not included in the program, outof the total poor households in the sample. Leakage rate refers to the
proportion of non-poor households who participate in the program, outof the total beneficiary households. As was done in the previous sections,the state of poverty is judged by the adult equivalent consumption
61amount of the household with the same poverty line. The following table
6.1 shows this.

Table 6.1. The rate of under-coverage and leakage in the sample
Under-coverage
Leakage
Targeting method(1)(2)
Self-selection
40.0%
28.8%
Census19.4%
32.2%
Source: calculation by author.

According to column 1 of Table 6.1, one can find that under the
self-selection method, 40% of the poor households could not participate
in the program. If census method includes all the eligible households
into the program, this rate becomes about a half of the rate under the
self-selection method. This means that out of the total poor households
in the sample, 20.6% (1293 households) could not participate in the
program under the self-selection method but would participate under the
census method. Knowing their characteristics is of my interest here.

It can be said that leakage rate, shown in column 2 of table 6.1,
represents the efficiency of the targeting. It is implied that the higher therate is, the more budget is being spent on the non-poor households,
which should be given to the poor households for the sake of poverty
reduction. Comparing the two targeting methods, the census method hasslightly higher rate. This is because the classification of the householdsby the program is not perfect and under the census method a substantial
number of non-poor households would also be included in the program
mistakenly. Therefore, under the census method less proportion of the62budget is reaching to the poor households, even if we take into accountits higher targeting cost.
As

anadditional

implication,

thissuggests

that

some

non-poor

households are surely excluded from the program by targeting. Recall
that in this sample, the poverty rate is 58%, meaning that there are 42%
of non-poor households in the sample. In other words, without targeting,the (natural) leakage rate would be 42%. The leakage rate of the twotargeting methods suggests that around 10% of non-poor households
would be excluded from the program, even though whether or not thisrate is sufficient is unclear.

Simply looking at these rates, one can find that if the two methods
perform almost equally according to the results in section 5, it is naturalto think that the census should succeed in incorporating many extremely
poor households, because giving a transfer to these households improves
the FGT measures a lot and this compensates the fact that less amount isreaching to the poor households under the census method. In thefollowing analyses, I try to reveal the difference of the targetedhouseholds between the two methods, in terms of consumption amount.

6.2. Average Distance of the Targeted Households’ Consumption
Amount from the Cut-off Point Consumption AmountHere, two types of indices are calculated: the average gap of the poor
households targeted under each method from the poverty line; and theaverage

gap

of

the

non-poor

households
63

targeted,

in

terms

of

consumption amount. Making use of these indices, I can compare theeffectiveness of the targeting over different size of the number of thehouseholds targeted.

6.2.1. Average distance of consumption amount from the poverty line forpoor households targetedFirst, I try to look at how much poor the targeted households are. If thehouseholds detected under one targeting method are poor enough, it canbe said that the targeting method works effectively. On the other hand, if
the detected households contain substantial non-poor households, thetargeting method would be paying some unnecessary costs. Here, thefollowing index is introduced:
𝑈 𝛼 ≡𝑐! − 𝑧 !
𝑧
,
!
1
𝑐
<
𝑧
!
!!!

!
!!! 1 𝑐! < 𝑧

where 𝑁 is the number of the households targeted, 𝑐! is the amount of
adult equivalent consumption of the household 𝑖, 𝑧 is the consumption
amount at the cut-off point, and 𝛼 is the weight attached to the distancefrom the poverty line ( 𝛼 = 1,2 is supposed here).	
  1 ∙ is an operator
which takes value 1 if the condition∙ is true and 0 otherwise.

Note that the denominator breaks down into the number of poor
households out of the households targeted. Therefore, when 𝛼 = 1, thisindex means the average distance of the targeted poor households fromthe poverty line, in terms of their consumption amount. When 𝛼 = 2,
more weights are attached to the households with relatively lessconsumption amount.
64

If this index is applied for all the households from the sample, it will
indicate the state of average poverty state in the sample, where notargeting is executed. I shall call this state “status quo”. The following
table 6.2 shows the indices from the households from status quo, thehouseholds targeted by the self-selection method, and the households
targeted by the census method.

Table 6.2. Average distance of the targeted households’ consumption
amount from the cut-off point consumption amount for the targeted poor
households
U(1)
U(2)
Status quo (1)
0.369
0.183
0.392
0.201
Self-selection (2)(1.064)(1.100)0.386
0.195
Census (3)(1.046)(1.070)The figures in parentheses indicate the ratio to the index from status quo(in raw 1).
Source: calculation by author.

From row 1 of table 6.2, one can find that for all the poor households inthe sample, the consumption amount is less than the consumption amountat the poverty line by on average 0.369 (36.9%), and the squared
difference in the consumption is on average 0.183 (18.3%). The row 2
and 3 indicate the indices from the households targeted by self-selection
and census method, and their ratio to the indices from status quo. Theysuggest that the average gap from the poverty line of the households
targeted by self-selection is 1.064 times larger than from status quo. As
for the census method, the average gap turns out to be 1.046 times larger
65than from status quo. The ratios larger than 1 imply that the households
targeted by both self-selection and census are more likely to be poor,
compared with the households from whole sample. Comparing theindices from self-selection and census, it is found that the households
targeted by self-selection method are slightly more likely to be poor thanthose targeted by census method.
Looking at the index U(2), a similar trend is found as the index U(1).
That is, even putting more weight on poorer households, the average
distance from the poverty line is larger than that from status quo, bothunder the self-selection and census method. Comparing the self-selection
and census method, again the index from the self-selection method islarger. This, along with the result from U(1), suggests that out of thepoor households targeted, the households under the self-selection
method are on average poorer than those under the census method.
Therefore, it can be said that the self-selection method targets more
efficiently the poorer households.

6.2.2. Average distance of consumption amount from the poverty line fornon-poor households targeted

Next, for the targeted non-poor households in contrast, the average
distance from the poverty line in terms of the consumption amount iscalculated for each targeting method. The index becomes in thefollowing form:
𝐿 𝛼 ≡𝑐! − 𝑧 !
𝑧
,
!
1
𝑐
≥
𝑧
!
!!!

!
!!! 1 𝑐! ≥ 𝑧

66

where the 𝑁, 𝑐! , 𝑧 and 𝛼 are defined in the same manner as 𝑈 𝛼 .

This index indicates how far from poverty the non-poor households
wrongly targeted are. When 𝛼 = 1, this means the non-poor households’
the average additional consumption to the consumption amount at thepoverty line. When 𝛼 = 2, more weights are attached to relatively more
wealthy households. As was done in the case of 𝑈 𝛼 , 𝐿 𝛼

is calculated

for each group of the households targeted by the self -selection andcensus method, and the households from the whole sample (i.e. statusquo). If this index is larger for one targeting method, it implies that thetargeting method detects more relatively wealthy households and its
precision would be less. The following table 6.3 shows the results.

Table 6.3. Average distance of the targeted households’ consumption
amount from the cut-off point consumption amount for the targeted
non-poor households
L(1)
L(2)
Status quo (1)
0.753
2.310
0.553
0.980
Self-selection (2)(0.735)(0.424)0.569
0.925
Census (3)(0.755)(0.400)The figures in parentheses indicate the ratio to the index from status quo(in raw 1).
Source: calculation by author.

The results regarding index L(1) shows that by executing either
self-selection or census targeting, we can exclude effectively wealthierhouseholds. Without targeting, the average consumption amount of
non-poor households is higher than the amount at the cut-off point by67

0.753(75.3%)

of

it.

The

non-poor

households

targetedunder

self-selection method consumes on average 0.553 (55.3%) more than the
cut-off point consumption amount, and this average is 0.735 times(73.5%) of the average of status quo. Similarly, the non-poor households
targeted under census method consumes on average 0.569 (56.9%) more,
and this is 0.755 times (75.5%) of the average of status quo. This
indicates that by applying either self-selection or census targeting we
can effectively exclude non-poor households with large amount ofconsumption, resulting in a reduction in the average consumption
amount (above the poverty line) by from 24.5% to 26.5%.
Comparing the index of self-selection and census method, the average
consumption amount of non-poor households detected is slightly lessunder the self-selection method. This suggests better efficiency of theself-selection method.

As for the index L(2), – though this is not of so much interest because
whether or not weighting the wealthier households is beyond the scopeof the discussion – both under self-selection and census method theweighted average is reduced substantially (from 2.310 of status quo to0.980 and 0.925 of self-selection and census, respectively). The index isslightly less for census method than self-selection method, just meaningthat the consumption of the non-poor households targeted under the
self-selection exhibits more variance than those under the census
method.

6.2.3. Implications from the analysis
68

From the analysis in this part, the following things are implied. First,
similarly to the results in section 5, by carrying out targeting at thehousehold level, the program can focus on relatively poorer households,
excluding relatively wealthier households.
Looking at the index U(1), the average gap of the targeted poor
households’ consumption is larger than that from the whole sample, by
6.4 percent point under the self-selection method and 4.6 percent point
under the census method. Looking at the index U(2), even if moreweights are put onto relatively poorer households, it is found that boththe self-selection and census method target those households. On theother hand, the indices L(1) and L(2) suggest that even if the twotargeting methods wrongly select non-poor households, their average
consumption amount is less than the average of all the non-poorhouseholds in the sample.

Second, comparing the self-selection and census method in terms of theconsumption amount of the targeted households, the former exhibits
slightly more efficiency than the latter, except for the index L(2).
Although the self-selection method misses about 20% more poor
households than the census method according to the results from 6.1, theaverage distance of the targeted poor households’ consumption from thepoverty line is larger under the self-selection method. This suggests theself-selection method is likely to target relatively poorer households.

6.3. Density Estimation of the Consumption Amount69

The next step to compare the performance of self-selection and census
method is to see the density of the consumption amount of thehouseholds targeted by each method. Using Kernel density estimation, 2 4
the probability density function of the consumption amount from thedifferent groups of households is visualized.

The main interest is to see how the distribution of consumption amountdiffers over those targeted under the self-selection and census method.
More specifically, how the distribution of consumption amount amongpoor households varies over different groups is of interest. Therefore, inthis analysis, for the estimation of density function I focus on thehouseholds with consumption amount not far from the poverty line. 2 5

The next graph 6.1 shows the density of consumption amount of thehouseholds grouped by their eligibility status to the program judged by
Oportunidades. This graph enables us to know how the eligibility status
by Oportunidades and the criteria based on consumption differ. The
vertical line in the graph shows the poverty line (about 441 Mexican
pesos).
24

The kernel density estimator is generally defined as:!
1
𝑥 − 𝑋!
𝑓! 𝑥 =   
𝐾
,
𝑛ℎ
ℎ
!!!

where 𝐾 ∙ is the kernel function and ℎ is the band width. Here theEpanechnikov kernel is used as the kernel function, and the band width
is the width that would minimize the mean integrated squared error.
2 5 Specifically, I limit the sample for the households with adultequivalent consumption amount less than 1000 Mexican pesos. Note thatthe adult equivalent consumption amount at the cutoff point is about 441
Mexican pesos.
70Graph 6.1. Distribution of adult equivalent consumption amount by
eligibility status judged by Oportunidades

The vertical line in the graph indicates the poverty line.
Source: calculation by author.
	
 
Graph

6.1

revealsthat

consumption

amountofthenon-eligible

households is likely to be higher than the eligible households, as itshould be. However, there can be found a number of households with lessconsumption than poverty line among the non-eligible households.
Moreover, the peak of the density of the consumption of non-eligible
households is less than the poverty line. This suggests that, though it isobvious, the eligibility for the program and the poverty state measured
by consumption do not coincide well. Note that the graph is shown justto indicate the difference between the two criteria, not to judge them.
71Then, the following graph 6.2 shows the density of consumption amountof the households targeted by the self-selection method, of thosetargeted by the census method, and of those targeted by the census
method but not by the self-selection method.

Graph 6.2. Distribution of adult equivalent consumption amount of thehouseholds targeted by the self-selection method, the census method,and only by the census method.

Each curve shows the density of the adult equivalent consumption
amount of the households targeted by each method. The curve “Onlyunder census” indicates the consumption of the households that are
targeted by the census but not by the self-selection method. The verticalline in the graph indicates the poverty line.
Source: calculation by author.

Graph 6.2 shows how the distribution of consumption differs across the72households targeted by each targeting method. This shows that the
distribution of the households targeted by self-selection and census
method exhibits quite similar shape, but the distribution of self-selection
is a little on the left. This means, out of the households targeted, under
the self-selection method the rate of poor households is slightly higher
than under the census method. Also, the density of the households above
the poverty line is slightly higher under the census method, meaning thatthe census method is slightly more likely to target the non-poorhouseholds. This is the same implication as what was shown in part 6.2.

Looking at the curve “Only under census”, which shows the distributionof consumption among the households targeted by the census method but
not by the self-selection method, 2 6 one can find that in this curve there
is more density on the right than the other two curves. This explains why
the curve of census method is slightly on the right of the curve ofself-selection. However, the consumption amount at the peak of thedensity for “Only under census” is still less than the poverty line, which
implies there is substantial number of poor households in this group. It
can be said that the advantage of the census method is to be able toinclude these households into the program.

In summary, the targeting of the self-selection method can be said to beefficient because the targeted households are likely to be poorer thanthose targeted under the census method. On the other hand, the census
method can include the larger number of poor households, which can be
26

Note that these households are those who are classified as eligible by
the Oportunidades’ criterion but not incorporated into the program.
73

said to be suitable for programs with wide coverage.

6.4. Overall Discussions on the Self-selection Method versus the Census
MethodThe

analysesin

thissectioncomparedtheperformance

of

the

self-selection and census method more closely. In part 6.1 it is shownthat under the census method around 20% more of the poor households
would be incorporated into the program, compared to the self-selection
method. However, the census method also includes non-poor households
mistakenly, and the rate of non-poor households out of the total
incorporated households is higher than the self-selection’s case. In part6.2 and 6.3, the detail of the consumption amount of the selectedhouseholds under each targeting method is described. It suggests that the
self-selection method targets poor households efficiently because theconsumption amount of the targeted households is likely to be less,while the census method allows to include more number of the poor
households. Here, discussions are made on some implied remarks.

First, the results from section 5 show that by giving a transfer to thetargeted households the self-selection method and census method reduce
the FGT measures by almost equal amount. On the other hand, the resultsfrom section 6 imply that the households targeted by the self-selection
method tend to be poorer than the census method. Why did the census
method perform as well as the self-selection method in section 5?
One of the causes for this might be the relatively large transfer amount.
74

The transfer amount given under the self-selection method is 316
Mexican pesos, which is almost three quarters of the consumption
amount at the cut-off point (441 Mexican pesos). Even under the census
method, the targeted households would receive about 221 Mexican pesos
for the first budget and 254 Mexican pesos for the second budget. Thesetransfer amount seem so large that with them most poor households
would be able to go beyond the poverty line set here.
If the transfer amount is relatively large compared to the poverty line,
the FGT measures should be reduced more if the transfer is given to thelarger number of poor households.

Second, the characteristics of those households who are eligible but notincorporated into the program should be analyzed more closely, because
these households affect the performance of the self-selection method
crucially. Knowing why these households did not participate in the
program would help improving the implementation of the self-selection
method.
One example of describing a characteristic of these households would be
the knowledge of the program. Under the self-selection method, those
who want to participate in the program first need to go to the program
module, and then their poverty status is examined and program
incorporation is determined. However, according to Coady and Parker(2005b), 31% of the poor households did not know about the program
module. Applying the same calculation to my sample, the percentage ofthe poor households that did not know about the module declined toabout 25%. However, if I limit the sample to those households who are75

classified as poor but not incorporated, the percentage rises to 49%.
Also, the analysis in part 6.3 shows the consumption amount of thesehouseholds is likely to be more than that of the households targeted
under the self-selection method (hence incorporated into the program).
One possible reason for this might be that on the process of theself-selection targeting, wealthier households tend to give up making theprocedure. That is, households would be less likely to apply for the
program if they consider their probability of acceptance of the program
is low. It seems natural to think that out of the households that knew
about the program, the majority of the eligible but not incorporated
households gave up the participation procedure because they did notconsider themselves so much poor. Therefore, the consumption amountof the eligible but not incorporated households is found to be larger than
the incorporated households.
	 To conclude this section, I summarize the advantages and disadvantagesof the self-selection method against the census method.
The most notable advantage of the self-selection method would be its
lower targeting costs compared with the census method. This advantage
actually made Oportunidades decide to apply the self-selection methodin urban areas where poor households are less concentrated. In addition,
Gutierrez et al (2003) found that the self-selection method was effective
for the localities with large population s. As the second advantage of theself-selection method, lower targeting costs under the self-selection
method mean that the government can set a lower budget amount. So it
can be said that the self-selection method is more likely to be applicable76

for developing countries, whose financial resources are limited.
The demerit of applying the self-selection method is that it is difficult totake into account the participation of the targeted households who are
expected to participate in the program. The self-selection method
sometimes fails to involve poor households into the program, who need
the program the most. However, this problem can be solved, or at least
alleviated by improving the targeting process. One way of doing thiswould be to inform potential recipients more precisely. It is expectedthat the efficiency of the self-selection method could be improved.

777. Conclusion

In this paper, the effectiveness in reducing poverty of several targeting
methods is compared. Assuming that a transfer is given to the eligible
households under several targeting methods, under two types of budgets,by how much the FGT measures are reduced is observed. From thisanalysis, one can see how effective the method of targeting applied by
Oportunidades is compared to alternative targeting methods.

The results from section 5 suggest the following three things. First, in
my sample of ENCELURB 2002, the self-selection method and thecensus method reduced poverty index by almost equal amount. Second,
the larger reduction in the FGT measures by the self-selection method
and the census method compared with uniform transfer suggests thattaking secondary targeting is effective. Third, distributing the budget
equally to the eligible households reduces poverty slightly more thanapplying the amount of transfer used by Oportunidades.
Further analyses made in section 6 for the first topic also shows that the
self-selection method targets poor households efficiently because theconsumption amount of the targeted households is likely to be less,while the census method allows to include more number of the poor
households.
Here are several points that are implied as concluding remarks from theresults of the paper.

First, the results suggest that the decision of applying the self-selection78method turns out to not be bad, although there are not apparent gains
from applying it in terms of poverty reduction. Also note that in theanalysis the performance of the census method might be overvalued
because of the strong assumption that all the eligible households would
be included under that method. This is not always the case because there
must be some households who do not participate in the program even if
they

can

be

incorporated.

Indeed,takingintoaccountthattheself-selection method can be carried out with less budget amount, theOportunidades program’s decision of having applied this to the urban
population can be supported.

Second, the efficiency of the self-selection method highly depends onthe characteristics of the targeted areas. If the households have greater
capacity to go to the program module independently, the self-selection
method would be more effective in detecting poor households. Also, therelative efficiency of the self-selection method against the census
method depends on the size of the locality and the extent of theconcentration of poor households. For further studies, it would be worth
investigating how the efficiency of the self-selection method and thecensus method varies, depending on the locality. Unfortunately, in thisanalysis I was only able to use the data that was taken after thegeographic targeting, due to availability. With a dataset from the wholecountry, it would be possible to compare the effectiveness acrossdifferent localities.

Third, it might be also worth investigating more closely the relationship79between the effectiveness for poverty reduction and the transfer amountand the way of distributing it. The results from section 5 suggest that thealmost equal effectiveness of the self-selection and census method might
be due to the relatively large transfer amount. The results also show that
distributing the transfer equally to each household reduces slightly morepoverty than the transfer amount used by Oportunidades. Though this
could not be done in this paper, revealing how the results differ
depending on these two factors would contribute to evaluate thetargeting.

Fourth, as a limitation of this paper, there was no data on the targetingcosts but the ones presented by Gutierrez et al (2003). It was unclear
whether this data includes not only the administrative costs of thetargeting but also such types of costs as the disutility among the poor
households for going to the program module individually. However,there was no alternative way of estimating the costs for this paper.

Fifth, I could not make a sufficient discussion on the asymmetric
information between the households and the program officers. As
discussed in Conning and Kevane (2002), beneficiary households havean incentive to underreport their welfare state at the screening
questionnaire made by program officers. Also, there might be problems
of moral hazard once the households get the benefit of the program.
Regardingthetargetingmethods

seen

in

this

paper,

boththeself-selection and census method may suffer from these problems, underwhich the proxy means test is carried out.
80

Similarly, there might be a reporting bias in the data on the households’
consumption. However, coping with these issues is beyond the scope of
the paper.

Finally, there would be two alternatives for the way of examining thepoverty state of the households.
The first way is to look at the poverty dynamics instead of the static
consumption. Jalan and Ravallion (1998) argue that a considerable
proportion of poverty is accounted for by the “transient poverty”, which
is attributed to intertemporal variability in consumption. They point out
that static consumption data contain considerable noise about long-term
poverty status. If our concern is about the “chronic poverty” which
persist over time rather than the transient poverty, it can be costly to use
static data. This might be the case for this paper since one of theobjectives of Oportunidades is to reduce the long-term poverty.
The second is to compare the targeting performances by looking at the
distributions of the households’ consumption without setting any certain
poverty line. As is done in Duclos et al (2004), making use of thestochastic dominance literature instead of focusing on certain indiceslike FGT measures would enable us to avoid the weakness of such
indices that the households are clearly distinguished by a certain poverty
line, in spite of the similarity in welfare state of the households among
the poverty line. 2 7 However, these strategies are outside the scope of
this paper, too.

27

For a discussion of this, see, for instance, Atkinson (1987).
81

This paper does not give an evaluation of the program itself. The
program is evaluated only by its objective: the improvement in thehuman capital of the poor households. Neither does it give an evaluationof the targeting methods in general. The results are only for the specificgroup of population in Mexico of my sample. However, especially for
Latin American countries that have a similar social structure, the results
are considered applicable to some extent. In addition, the analysis in thispaper would be replicable for the analysis of other countries.

82References

Alatas, Vivi, Abhijit Banerjee, Rema Hanna, Benjamin A. Olken andJulia Tobias. 2012. "Targeting the Poor: Evidence from A field
Experiment in Indonesia." American Economic Review 102 (4): 1206–40.

Atkinson, A. B. 1987. "On the Measurement of Poverty." Econometrica
55 (4): 749-64.

Coady,David,Margaret

Grosh

and

John

Hoddinott.

2002.

"The

Targeting of Transfers in Developing Countries: Review of Experienceand Lessons." Social Safety Net Primer Series. Washington D.C.: WorldBank.

Coady, David P. and Susan W. Parker. 2005a. "Evaluación Del
Mecanismo De Focalización de Oportunidades En Zonas Urbanas "In Evaluación externa de impacto del Programa Oportunidades 2004,
195 - 243.

Coady, David P. and Susan W. Parker. 2005b. "Program Participation
under Means-Testing and Self-Selection Targeting Methods." FCND
Discussion Paper. 191. Washington D.C.: International Food Policy
Research Institute.

Conning, Jonathan and Michael Kevane. 2002. "Community-Based
Targeting Mechanisms for Social Safety Nets: A Critical Review." World
83

Development 30 (3): 375-94.

Consejo Nacional de Evaluación de la Política de Desarrollo Social.
2009.

"Reporta

SocialCifrasDe

Pobreza

Por

Ingresos

2008." Comunicado de Prensa. Mexico City: Dirección de Información y
Comunicación Social, Consejo Nacional de Evaluación de la Política deDesarrollo Social.

Deaton,Angus.

1997.

The

AnalysisofHouseholds

Surveys:A

Microeconomic Approach to Development Policy. Meryland: The Johns
Hopkins University Press.

Deaton, Angus and Salman Zaidi. 2002. "Guidelines for Constructing
Consumption Aggregates for Welfare Analysis." LSMS Working Paper.
Washington D.C.: The World Bank.

Duclos,Jean-Yves,

Paul

Makdissi

andQuentin

Wodon.

2005.

"Poverty-Dominant Program Reforms: The Role of Targeting andAllocation Rules." Journal of Development Economics 77 (1): 53-73.

Escobar Latapí, Agustín and Mercedes González de la Rocha. 2003.
"Evaluación Cualitativa del Programa Oportunidades En Zonas Urbanas,
2003." In Evaluación Externa de Impacto del Programa Oportunidades
2003, edited by Bernardo Hernández Prado and Mauricio Hernández
Ávila. Mexico City: Instituto Nacional de Salud Pública.

84

Fiszbein, Ariel, Norbert Schady, Francisco H.G. Ferreira, Margaret
Grosh, Niall Keleher, Pedro Olinto and Emmanuel Skoufias. 2009.
Conditional Cash Transfers - Reducing Present and Future Poverty.
Washington D.C.: The International Bank for Reconstruction andDevelopment, The World Bank.

Gutiérez, Juan Pablo, Setefano Bertozzi, and Paul Gertler. 2003.
"Evaluación de la Identificación de Familias Beneficiarias en el Medio
Urbano." In Evaluaci ón de Resultados de Impacto del Programa de
Desarrollo

HumanoOportunidades.

MexicoCity:

SecretaríadeDesarrollo Social.

Instituto Nacional de Nutrición. 1987. Valor Nutritivo De Los Alimentos
Mexicanos. Tablas de uso práctico. Mexico City: Instituto Nacional de
Nutrición.

Instituto Nacional de Salud Pública. 2005. General Urban Methodology
Notes. Mexico City: Secretaría de Desarrollo Social.

Jalan, Jyotsna and Martin Ravallion. 1998. "Transient Poverty in
Postreform Rural China." Journal of Comparative Economics 26 (2):338-57.

Levy, Santiago. 2006. Progress against Poverty: Sustaining Mexico's
Prgresa-Oportunidades

Program.

Institution Press.
85

Washington

D.C.:

Brookings

Levy, Santiago and Evelyne Rodríguez. 2004. Economic Crisis, Political
Transition,

andPoverty

Progresa-Oportunidades

PolicyProgram.

Reform:Washington

Mexico’s

D.C.:

Regional

Operations Dept. II, Inter-American Development Bank.

Orozco, Monica and Celia Hubert. 2005. "La Focalizacion en el
Programa de Desarrollo Humano Oportunidades de Mexico." Serie deDocumentos de Discusión sobre la Protección Social. The World Bank.

Orozco,Mónica.

2007.

Oportunidades:

NormasBeneficiarias."

Paper¨Evaluación

desafios

y

"El
De

Programa

Ingresoprovided
de

for
los

Y
El

De

Desarrollo

EgresoDe

Seminario

programasdeLas

HumanoFamilias

Internacionaltransferencias

condicionadas.¨ Brasilia.

Ray, Debraj. 1998. Development

Economics. New Jersey: Prinston

University Press.

Secretaría de Desarrollo Social. 2003. Regras de Operación 2003.
Mexico City.

Skoufias, Emmanuel, Benjamin Davis and Jere R. Behrman. 1999. An
Evaluation of the Selection of Beneficiary Households in the Education,
Health, and Nutrition Program (PROGRESA) of Mexico. International
Food Policy Research Institute.
86Skoufias, Emmanuel, Benjamin Davis and Sergio de la Vega. 2001.
"Targeting the Poor in Mexico: An Evaluation of the Selection ofHouseholds into Progresa." World Development. 29 (10): 1769-1784.

Wooldridge, Jeffrey M. 2009. Introductory Econometrics: A Modern
Approach: South Western, Cengage Learning.

87