metadata: authors: 0: author: Cadot,Olivier
1: author: Jaud,Melise
count: Senegal
docna: 0: docna: A second look at the pesticides initiative
            program : evidence from Senegal
docty: Policy Research Working Paper
subtopic: Climate Change and Agriculture,Crops and Crop Management Systems,Pest Management,International Trade and Trade Rules,Food Security
teratopic: Private Sector Development,Macroeconomics and Economic Growth,International Economics and Trade,Finance and Financial Sector Development
abstracts: cdata!: This paper investigates whether the
            Pesticides Initiative Program has significantly affected the
            export performance of Senegal' shorticulture industry.
            The authors apply two main microeconometric techniques,
            difference-in-differences and matching
            difference-in-differences, to identify the effect of the
            Pesticides Initiative Program on exports of fresh fruits and
            vegetables. They use a unique firm-level dataset containing
            data on sales, employment, and exports by product and
            destination markets, as well as firm enrolment year, over
            2000-2008. The results suggest that wile the program had no
            significant effect on exports pooled over all products and
            destinations, it had a positive effect when considering
            fresh fruits and vegetables exports to the European Union.
geo_reg_key: 1927320,517191
repnme: repnme: A second look at the pesticides initiative
            program : evidence from Senegal
display_title: A second look at the pesticides
            initiative program : evidence from Senegal
listing_relative_url: /research/2011/04/14185492/second-look-pesticides-initiative-program-evidence-senegal
topicv3: Crop Production m1326627 174,Trade m1326924 698,Food Safety m1327139 180,Pesticides m1504403 2914,Vegetable and Fruit Production m1583835 4720
publishtoextweb_dt: 2011-04-01T00:00:00Z
historic_topic: Private Sector Development,Macroeconomics and Economic Growth,International Economics and Trade,Finance and Financial Sector Development
no_of_pages: 34
author_keys: 0: author_key: 615253
1: author_key: 988481
model_extractions: WPS5635 Policy Research Working Paper 5635

# Evidence from Senegal

_Melise Jaud Olivier Cadot_ The World Bank Poverty Reduction and Economic Management Network International Trade Department April 2011
Policy Research Working Paper 5635

# **Abstract**

This paper investigates whether the Pesticides Initiative Program has significantly affected the export performance of Senegal’ shorticulture industry. The authors apply two main microeconometric techniques, difference-indifferences and matching difference-in-differences, to identify the effect of the Pesticides Initiative Program on exports of fresh fruits and vegetables. They use a unique firm-level dataset containing data on sales, employment, and exports by product and destination markets, as well as firm enrolment year, over 2000–2008. The results suggest that wile the program had no significant effect on exports pooled over all products and destinations, it had a positive effect when considering fresh fruits and vegetables exports to the European Union.

This paper is a product of the International Trade Department, Poverty Reduction and Economic Management Network. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank. org. The author may be contacted at ocadot@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
A Second Look at the Pesticides Initiative Program: Evidence from Senegal<sup>_∗_</sup> Melise Jaud<sup>_†_</sup> Olivier Cadot<sup>_‡_</sup> <mark>May 2011</mark> Keywords: Senegal, EU, Agriculture, Export Promotion, Technical Assistance, Impact Evaluation.

JEL classification codes: F13, F14, L15, L25, O17, O24, C23.

> _∗_ This paper is part of a joint program of Switzerland’s NCCR and the World Bank on the application of impact-evaluation methods to trade-related interventions. Without implicating them, we would like to express our thanks to C ́eline Carr`ere, Ana Fernandes, Julien Gourdon, Daniel Lederman, Aaditya Mattoo, Jaime de Melo, Marcelo Olarreaga, Alberto Portugal and Akiko Suwa for useful comments. We are also grateful to the European Commission and to Senegal’s _Direction G ́en ́erale des Douanes_ and _Direction Nationale des Statistiques_ for access to their data.

> _†_ The World Bank and Paris School of Economics; mjaud@worldbank.org

> _‡_ The World Bank, University of Lausanne, and CEPR; ocadot@worldbank.org
<!-- Start of picture text --> Abstract<br>This paper investigates whether the Pesticides Iniative Program has sig-<br>nificantly affected the export performance of Senegal’s horticulture industry.<br>We apply two main microeconometric techniques, difference-in-differences<br>and matching difference-in-differences, to identify the effect of the Pesti-<br>cides Initiative Program on exports of fresh fruits and vegetables. We use a<br>unique firm-level dataset containing data on sales, employment, and exports<br>by product and destination markets, as well as firm enrolment year, over<br>2000-2008. The results suggest that while the program had no significant<br>effect on exports pooled over all products and destinations, it had a positive<br>effect when considering fresh fruits and vegetables exports to the European<br>Union.<br><!-- End of picture text --> <!-- Start of picture text --> 2<br><!-- End of picture text -->
# **1 Introduction**

A poll featured in the _Financial Times_ of July 12, 2010 highlighted two striking facts. First, between 60 and 80 percent of the respondents in Western countries felt that budget cuts would help rather than hurt the economy; second, aid to developing countries was ranked first as a candidate for cuts in the U.S. and U.K., and second in all other countries. Against this background, in the words of a recent OECD document, “[t]here is a growing political demand among academics, NGOs, international agencies and voters for results in development.” (OECD 2010a).

The demand for evaluation applies to aid for trade (AFT) as well as to other forms of aid.<sup>1</sup> After declining as a share of Overseas Development Assistance (ODA) in the second half of the 1990s, AFT commitments have increased by 62% since the launch of the AFT initiative at the 2005 WTO Ministerial in Hong Kong, reaching $41.7 billion or 37% of “sector-allocable” ODA in 2008. For the EU, traderelated assistance (TRA) is an important part of the trade-and-development policy nexus, responding to a widely-perceived need to provide adjustment assistance to Southern partners affected by increasingly stringent product standards and by recent changes in the EU’s system of preferences (see te Velde et al. 2006).

What do we know about the impact of aid for trade (AFT)? The literature is limited. As for its allocation, Gamberoni and Newfarmer (2009) found that, after controlling for absorption capacity (governance etc.), it correlated with an indicator of ‘demand’ based on indices of under-trading. As for its impact, Wagner (2003) explored whether aid generated trade in the form of exports from the _donor_ country to the recipient (up to the early 1990s, over half of all bilateral aid was at least partially tied to donor exports); he found, using a gravity equation, that this particular form of trade was indeed boosted. By contrast, Osei, Morrissey and Lloyd (2004) ran a gravity equation in first differences on a panel of four European donors and 26 African recipients after testing for the direction of causalitybetween bilateral aid and trade flows. They found an unstable and, on the whole, insignificant impact of aid on trade (also in the form of exportsfrom donor to recipient). Nelson and Juhasz Silva (2008) used a moreconventional gravity equation including bilateral aid flows as a regressor (instrumented by their one-year lagged value) and found a significant although small elasticity, again, for flows from donor to recipient.

> 1Aid for Trade was defined by the 2006 WTO AFT Task Force through six categories: (i) Trade policy & regulations; (ii) Trade development; (iii) Trade-related infrastructure; (iv) Building productive capacity; (v) Trade-related adjustment; (vi) Other trade-related needs. The first three are referred to as Trade-related assistance (TRA); the last three are referred to as the ‘wider aid-for-trade agenda’ (EC 2008).

3
Only some of the most recent studies have looked at whether aid raised the export capacity of _recipient_ countries. Cali and te Velde (2009) regressed trade costs and the value of exports on control variables and lagged AFT disbursements, using data from the OECD’s Credit Reporting System. Their sample was a panel of countries covering 1995-2007, and they dealt with endogeneity and measurement errors in AFT flows by instrumenting them with the World Bank’s index of civil liberties. The message that seems to emerge across their various specifications is that aid to trade facilitation and infrastructure seems (by and large) to have a significant effect (the former on trade costs, the latter on export values), while aid to productive capacity is insignificant. When looking at sectorally-targeted aid, controlling for country _×_ sector fixed effects (comparative advantage) they found, again, that aid to infrastructure had a significant impact but aid to productive capacity didn’t. Brenton and von Uexkull (2009) combined mirrored product-level (HS4) export data with export-development (ED) aid data from the GTZ (covering 1975-2005) and from the OECD/WTO Trade Capacity Building Database (covering 2001-2005) for 48 countries. They useda differencein-differences (DID)<sup>2</sup> approach where the performance variable was exports and regressors included lagged exports, country and year _×_ product fixed effects, and contemporaneous and lagged aid coded in binary form (ED program in force = 1). By and large, once outliers were dropped, matching yielded insignificant coefficients in spite of the relatively large sample size, suggesting that once selection effects were taken into account, export-development programs provided little significant boost to exports. All in all, it is fair to say that, as the literature stands, the effect of AFT on the export performance of beneficiary countries has not been clearly established on the basis of aggregate numbers.

Among TRA programs, technical-assistance ones have scarcely been evaluated.<sup>3</sup> A brochure published by the EU Commission (EC 2006) tells the interesting story of a Kenyan fruit and vegetables exporter who got assistance from the EU’s Pesticides Initiative program (PIP); the case study presents the program as “[...] to provide support to companies like Myner [the Kenyan exporter], to help them get up to speed with European food safety and traceability requirements. [...] Since it began in July 2001, the PIP has had a positive effect on more than 26’000 ACP producers, many of whom are small-scale farmers. Nearly 6.25 million Euros has been

> 2See section 3 below for a discussion of the DID and matching approaches.

> 3Marcano and Ruprah (2009) is one of the few papers looking specifically at technicalassistance programs. Using a multiple-treatment approach, they found a positive impact to Chile’s Neighborhood Improvement Program and Guatemala’s Social Investment Fund (both water-access program) over and above public-works spending.

4
committed to the program, with each applicant allocated around 86’000 Euros. In line with the principles of the ACP-EU partnership agreement singed in Cotonou in 2000, the PIP aims to contribute to the development of the ACP’s private sector and to promote regional integration.” As for the case study itself, the brochure explains that “[w]hen Myner Exports began working with the Pesticides Initiative Program, or PIP in 2002, it was exporting about 300 tonnes of French beans, snow peas, passion fruit, and sugar snaps a year to the European Union. Today, the company exports some 900 tonnes a year. ” Quoting this particular case study in their assessment of the EU’s TRA, te Velde et al. (2006) noted that “in an ideal world, one would compare this supported company with a similar one that was not supported” (p. 21). This is precisely what we set up to do in the present paper, although using Senegal instead of Kenya as our sample of study.

We chose Senegal because the data we had access to provided a unique combination. Through a World Bank project run by Denisse Pierola and Paul Brenton at the World Bank, we obtained raw files from Senegal’s customs with export data at the transaction level for every year between 2000 and 2008. The data include the exporter’s ID, the product code, the country of destination, and the export value for over 500 HS8 fresh fruit and vegetable (FFV) products. We were able to merge the customs’ dataset with industrial-survey data and a list of PIP beneficiaries provided by the EU Commission.<sup>4</sup> This unique combination made it possible to construct a treatment group of firms that got assistance and a control group of firms that did not, and this for a sample period that ran from before the program to its end.<sup>5</sup> Using this rich data set, we used a wide array of approaches to estimate the effect of the PIP on firm-level exports of treated products (FFV). We ran DID regressions of the value of exports, by product _×_ firm _×_ destination, on control variables as well as a dummy variable marking ‘treatment’ by the PIP. In order to deal with selection issues, we combined the DID approach with propensity-score matching, although the small size of the sample and the systematic differences in size between treatment- and control-group firms considerably reduced the power of

> 4Thanks to Morag Webb for providing us with the data.

> 5Although we had similar customs data from a number of other countries, none covered years _before_ the start of the PIP. After merging, firm IDs have been deleted from the dataset for confidentiality reasons.

5
matching. We also used a control-function approach similar to Heckman’s selection model (Heckman 1979).

In most specifications we tried, we failed to find a significant impact. Only when considering the firm aggregate exports of FFV to the EU, did we find a positive and significant effect of the program. Given the small size and peculiarities of the sample, our results should be treated very cautiously and we would certainly stop short of concluding that the PIP was useless on the basis of this single impactevaluation exercise. Additionally, beneficiary firms self-selecting in the program would introduce a bias in the estimated treatment effect. However the direction of the bias is not clear. If self-selected firms are larger and potentially more efficient than others the estimated treatment effect should be biased upwards. Inversely less productive firms may be more likely to rely on financial aid or rent-seeking, biasing the effect downwards. If so, clearly more research is needed on this issue, possibly on other, larger samples to assess whether the PIP had any impact or not.

The paper is organized as follows. Section 2 presents some stylized facts on the exporting environment of Senegalese FFV producers and on how the PIP addresses its objective of alleviating some of the constraints that these producers face. Section 3 presents the impact evaluation: data, estimation issues. Section 4 presents the baseline results and some robustness tests. Finally, section 5 concludes.

# **2 Stylized Facts**

## **2.1 Senegal’s Exports of Fresh Fruits & Vegetables**

According to Senegal’s National Horticultural Direction, national exports of fresh fruits and vegetables (FFV) have been rising at a rate of about 15% a year since – – 2001. French beans alone the main export crop account for most of this increase, with volumes up from 652 tons to almost 9’000 between 2001 and 2007 (Figure 1).

6
<!-- Start of picture text --> 30°000<br>25°000<br>20°000<br>€<br>&<br>FSS5 15000<br>£3<br>wax 10000<br>$000<br>0<br>1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007<br>Year<br>@ Mangoes @ Beans, green O Tomatoes O Others fruit & vegetables<br><!-- End of picture text -->
marketing standards for FFV; (2) sanitary and phytosanitary (SPS) measures; (3) general hygiene rules based on HACCP control mechanisms; and (4) traceability standards. The EU’s SPS measures have become notably more stringent in the 1990s. Particularly relevant to FFV are reduced tolerance levels for chemical residues.<sup>8</sup> First, about 350 active substances initially approved for use in the EU have been gradually withdrawn (out of the 823 initially allowed). Second, Maximum Residue Levels (MRL)<sup>9</sup> and Import Tolerances (IT)<sup>10</sup> are imposed at levels specific to particular protection chemical-crop combinations. When an IT has not been established, a default value of 0.01 mg/kg corresponding to Level of Detection (LOD) in inspection labs are used. The registration of an IT is a complicated process involving the submission of a complete residue dossier including field trials and lab analysis results. For exporters of minor crops – most tropical crops except bananas– from developing countries, the challenge is compounded by the fact that agrochemical companies have little incentive to provide registration residue data for those crops because the benefit would not cover the cost.

HACCP and tracability requirements came into force with the General Food Law of 2002 (EC R 178/2002). Traceability means that EU food companies must document from (to) whom they are buying (selling) so that products can be traced back to their origin if they prove defective or dangerous. Although traceability is legally limited to a ‘one step forward, one step back’ principle within the EU (with no obligation to keep records in third countries), in practice EU buyers tend to go beyond the strict legal requirement. Complete traceability throughout the chain all the way up to the overseas producers is part of many private standards like the GlobalGAP. Legislative changes in EU standards and their potential detrimental effects on small growers/exporters were one of the primary drivers behind the establishment of the COLEACP’s Pesticides Initiative Program (PIP).<sup>11</sup> Maertens and Swinnen (2009) show that the E.U.’s rising standards have profoundly altered the structure of the supply chain in Senegal’s horticulture sector.

8In the late 1990s, an updated harmonised legislation package on pesticide Maximum Residue Limits (MRL) – EC Directive 91/414 and subsequent Regulation 396/2005– created concern for ACP horticultural exporters because of its stringency.

> 9The MRL is the level of residue legally permitted to remain in/on a food or animal feedstuff following the use of a Crop-Protection-Chemical (CPC) under Good Agriculture Practice (GAP), i.e. under the specific label instructions of the approved product.

10An import tolerance is an MRL set for imported products containing active plant-protection substances not authorised in the EU for reasons other than public health, or when a different level is appropriate because the existing Community MRL was set for reasons other than public health.

> 11The purpose of the Europe-Africa-Caribbean-Pacific Liaison Committee (COLEACP) is to facilitate the flow of trade in fresh fruit and vegetables between Africa-Caribbean-Pacific (ACP) countries and the EU.

8
First, the financial constraints generated by the need to comply with increasingly stringent standards have induced consolidation at the intermediation stage, with only the larger firms able to cope. Second, the relationship between intermediaries and producers has changed, with more control by intermediaries over farming methods. Tighter control has been obtained through increasingly precise contracts, technical assistance, and the provision of credit and farm inputs. Third, the induced changes have also affected the structure of upstream farm production, with a sharp decrease in the incidence of contract farming and a rise in that of large-scale estate production. Interestingly, Maertens and Swinnen’s analysis suggests that these changes have been accompanied by a rise in the incomes of households.

# **2.2 The PIP in Senegal**

The Pesticide Initiative Program (PIP) is financed by the European Development Fund (EDF) and implemented by the COLEACP. With an overall budget of 34.1 million euros, the PIP’s first phase started in 2001, initially for a five-year period; it was extended by two additional years. The program has two main objectives. The first is to enable ACP exporters of FFV to comply with European traceability and food-safety requirements (in particular as regards pesticide residues). The second is to consolidate the position of small-scale producers in the ACP horticultural value chain. The PIP’s support activities are organised around five components: (i) good company practises, (ii) training, (iii) capacity building, (iv) regulation & standards, and (v) information & communication. The core of the support (almost 30% of program budget) goes to component (i), which consists of helping producers and exporters to set up internal food-safety management systems in production and marketing operations. The regulation & standards component ensures that all substances recommended in crop protocols (‘technical itineraries’) are authorised in both the EU and origin country. Additionally, when needed, the program introduces registration of active substances as well as import-tolerance applications. This is especially important for minor crops of interest to smallholders. Finally, the capacity-building component aims at developing national capacity to provide the services needed by the industry. Beneficiaries of capacity-building activities include private consultants (training courses on food safety, pesticide use, and IPM); accredited laboratories (pesticide residue analysis); public services (including extension services and pesticide registration bodies); and strong professional organizations.

New traceability requirements and recent changes in the EU’s pesticides regulation have been of particular concern to Senegal’s horticultural industry—mainly the green bean industry. Senegal ranks fourth, after Kenya, Ghana and Uganda, 9
for the number of PIP protocols signed. PIP beneficiaries produce and export essentially green beans, cherry tomatoes and mangoes to the European market. Their needs relate essentially to traceability systems, staff training, access to information, advice on pesticide MRLs, and infrastructure improvement. Over the project’s life, several missions went to Senegal and met with beneficiary companies to review and adjust the activities carried out as part of the program. PIP support activities in Senegal have consisted of in-company training on hygiene and foodsafety procedures, development of traceability systems, safe use of pesticides, and recognition of mango pests for mango exporters. Four companies also requested support to obtain GlobalGAP certification; for those, pre-audits were conducted to help identify and correct problem areas. SEPAM obtained its certification in 2004; Soleil Vert, Baniang and AgriConcept obtained theirs in 2007. Remaining exporters, mainly smaller ones, are neither certified nor undertaking particular investments to get it. Except for SEPAM, SAFINA and GDS, Senegal’s FFV exporters have outgrower contracts with smallholders (involving training and support) rather than own production sites. In 2006, in cooperation with Senegal’s ANCAR ( _Agence Nationale de Conseil Agricole et Rural_ ) the PIP launched ‘Golden Bean’, an awareness campaign directly targeting 1’000 small FFV producers.

Finally, under the regulations component, field trials have been conducted on PIP priority crops including green beans, cherry tomatoes, okra, avocado, passion fruit, mango, papaya and pineapple.<sup>12</sup> Trials were initiated in November 2003 on green beans and cherry tomatoes, and in early 2004 on the remaining crops, with crop samples collected and shipped to European GLP-certified laboratories for residual analysis. A particular technical itinerary was developped for mangoes with the objective of bringing mango production in line with European regulations.

# **2.3 Selection into the Program**

Eligibility starts with the completion and submission of a request for PIP intervention addressing the applicant’s particular needs and objectives. The request identifies by self-assessment the problem to be resolved —e.g. MRL, non-accredited plant-protection products, or traceability— and puts forward possible fixes such as training in ICM/IPM systems or safe use of pesticides, implementation of foodsafety and traceability systems, or ‘technical itineraries’. It also specifies anticipated results, like getting into conformity or maintaining current export volumes. Finally, the request assesses a time line and budget, specifying what is requested

> 12Field trials are an important part of the establishment or amendment of crop protocols, enabling experts to analyze pesticide residual levels in fruit and vegetables. Following field trials, crop protocols are revised as required to achieve compliance with EU MRLs, and the information gathered in the process feeds back into farming practices.

10
from the PIP and what is provided by the applicant. Applications are considered on a first-come first-served basis, with no prioritization or selection criteria. To be accepted, a requested intervention must help to achieve product compliance with EU traceability and food-safety (pesticide residues) regulations. Upon acceptance, a protocol stating the actions to be implemented by each party on a cost-sharing basis (50% for each, except for smallholders who are expected to contribute only 20%) is signed. Wages and investment costs are _de facto_ excluded. The actions listed in the protocol are chosen among a menu offered by the PIP under its five components; however, the combination in each protocol is specific to a firm and varies across beneficiaries. So far, most of the financing has gone to training costs, technical support and the development of a food-safety toolbox containing crop protocols, GAP guidelines, and Hortitrace, a traceability software developed by the PIP. The program’s first phase has covered 21 countries<sup>13</sup> and 320 export companies. Out of those, 219 firms benefited from the ‘good company practices’ component (advice and assistance for setting up sanitary quality and traceability systems, and certification pre-audits). 153 benefited from training under the capacity-building component.

# **2.4 The PIP’s Evaluation**

An evaluation of the PIP’s first phase was undertaken in June 2008. The appendix’s valuation matrix summarizes the report’s findings and details objectives, expected results, performance indicators, and outcomes. Performance indicators are both quantitative and qualitative. The evaluation relies on trade data reported by the firms themselves and from Eurostat. Additionally, a survey was conducted among PIP beneficiaries and EU importers. As highligthed in the appendix’s evaluation matrix, program impacts are evaluated at the aggregate level. Outcomes for beneficiary firms are generally reported without controlling for location, size or experience in exporting FFV to the EU market, with the exception of specific objective (S1), for which outcomes are reported by ACP country or type of intervention (O1, O2, O3).

Overall, the evaluation report drew up a very positive image of the program’s impact, contributing to the launch of a second five-year phase in 2009. While fairly comprehensive, the PIP’s evaluation suffers from a typical drawback of this type of exercise—namely, the lack of a counterfactual to benchmark the performance of treated firms and products.

13The countries covered are Benin, Burkina Faso, Cameroon, the Ivory Coast, Gambia, Ghana, Guinea, Mauritius, Jamaica, Kenya, Mali, Mozambique, Namibia, Uganda, the Dominican Republic, Senegal, Surinam, Tanzania, Togo, Zambia, and Zimbabwe.

11
# **3 Impact Evaluation**

## **3.1 The Data**

Our dataset is constructed using three primary databases which together form a rich and unique combination. First, we have export data at the transaction level (aggregated to the firm level) from raw customs files over 2000–2008. Each record includes the firm’s tax ID, the product code, the country of destination, and the export value (in US dollars) and quantity (in tons) for over 500 HS8 products to 90 countries. Exports flows are reported annually. Second, the PIP’s administration in Brussels provided us with a list of the Senegalese firms that got assistance from the program in each year of the sample period.<sup>14</sup> Finally, we obtained data on employment and sales from the CNI, Senegal’s National Statistics Direction.<sup>15</sup> As the CNI also identifies firms by their tax ID, we could merge the three datasets. Among the reporting firms, almost 3% appear only once in the dataset. That is, they export only one product to one destination one year. As these observations are likely to be mis-reports or individuals, we drop them from our sample. We also drop international organizations and embassies, as well as trading and transport companies (the latter group represents about a quarter of all observations). This leaves us with a sample of almost 2’000 observations.

Let _i_ an exporting firm, _k_ be a product, _j_ a destination country, and _t_ a year. As the PIP targets _products_ (FFV), some of a firm’s products may be covered and some others not. In addition, technical assitance provided under the PIP helps make FFV marketable on EU markets, but does not necessarily help on other markets with different (or no) food-safety requirements. In view of this, we take the ( _i, k, j, t_ ) vector as our Primary Sample Unit (PSU). Our dependent variable is the annual flow of exports of product _k_ to destination _j_ by firm _i_ in year _t_ , _yikjt_ . That is, we take the intensive margin as our baseline measure of performance. Table 1 gives descriptive statistics of firm-level covariates and performance indicators for treated and non-treated flows. Covariates include firm _i_ ’s annual (overall) turnover, _salesit_ , and employment, _Nemployeeit_ , the number of products it exports. We also include _Nprodit_ , the number of products it exports to destination _j Nprodijt_ , the number of destinations to which it exports product _k_ , _Ndestikt_ and a binary variable equal to one if firm _i_ has more than one year of experience in exporting product _k_ to destination _j_ , _expikj_ are included. Performance indicators at the intensive margin include firm _i_ ’s exports of product _k_ to

> 14Variation in the intensity of the treatment across firms would induce a bias in the estimation of the effect. Unfortunately, we do not have data on the intensity of the treatment.

> 15The data is collected by the _Centre National d’Identification_ (CNI), part of the National Statistics Direction.

12
Table 1: Descriptive Statistics, non-Treated and Treated Flows.

||Non-Treated Flows|Treated Flows|||
|---|---|---|---|---|
||Mean|Mean|t-Stat|p<br>t|
|Sample|1370|492|||
|Firm characteristics|||||
|sales_it_|21.7|21.1|4.9|***|
|Nemployee_it_|4.4|5.3|-6.9|***|
|Export performace: intensive margin|||||
|export_ikjt_|8|10.1|-17.6|***|
|total<br>~~e~~xport<sup>_FFV_</sup><br>_ijt_|8.2|11.9|-22.8|***|
|total<br>~~e~~xport_ikt_|8.5|11.1|-20.9|***|
|total<br>~~e~~xport<sup>_FFV_</sup><br>_it_|8.6|13.21|-34.9|***|
|total<br>~~e~~xport_it_|12.4|13.1|-7.7|***|
|Export performace: extensive margin|||||
|Ndest_ikt_|1.5|2.2|-11.5|***|
|Ndest<sup>_FFV_</sup><br>_it_|1.3|8|-19.5|***|
|Ndest_it_|5.08|3.4|12.3|***|
|Nprod_ijt_|14.9|3.2|20|***|
|Nprod<sup>_FFV_</sup><br>_ijt_|0.6|2.7|-20.1|***|
|Nprod<sup>_FFV_</sup><br>_it_|1.3|8|-19.5|***|
|Nprod_it_|38.9|9.3|231|***|

Variables are in logs. *, **, and *** denote statistical significance of the t-statistics at the 10%, 5%, and 1% levels, respectively.

destination _j_ in year _t_ , _exportijkt_ ; its total exports of FFV to country _j_ in year _t_ , _total_ _~~e~~ xport_<sup>_FFV_</sup> _ijt_ ; its total exports of product _k_ worldwide, _total_ _~~e~~ xportikt_ ; its total exports of FFV worldwide, _total_ _~~e~~ xportsit_<sup>_FFV_</sup> ; and its total exports worldwide, _total_ _~~e~~ xportit_ . Export performance indicators at the extensive margin include firm _i_ ’s number of destinations with product _k_ in year _t_ , _Ndestikt_ ; its number of destinations with FFV, _Ndest_<sup>_FFV_</sup> _it_ ; its total number of destinations, _Ndestit_ ; its number of products to destination _j_ , _Nprodijt_ ; its number of FFV products to destination _j_ , _Nprod_<sup>_FFV_</sup> _ict_ ; its number of FFV products wordwide, _Nprod_<sup>_FFV_</sup> _it_ , and its total number of products wordwide, _Nprodit_ .

Table 3.1 shows descriptive statistics for participating and non-participating _firms_ . Two observations are in point. First, participating firms are larger than non-participating. Thus, size must be controlled for in order for non-participating firms to provide a credible counterfactual. Second, these are all small firms, as 13
Table 2: Descriptive Statistics, non-Treated and Treated Firms.

||Non-treated frms|Treated frms|||
|---|---|---|---|---|
||Mean|Mean|t-Stat|p<br>t|
|Sample|131|100|||
|Firm characteristics|||||
|sales_it_|19.5|20.3|-2.0|**|
|Nemployee_it_|2.9|4.7|-4.7|***|
|Export performace: intensive margin|||||
|total<br>~~e~~xport<sup>_eu_</sup><br>_it_|10.1|11.9|-6.7|***|
|total<br>~~e~~xport<sup>_ffv_</sup><br>_it_|10.0|12.0|-6.8|***|
|total<br>~~e~~xport_it_|10.7|11.9|-4.7|***|
|Export performace: extensive margin|||||
|Ndest<sup>_ffv_</sup><br>_it_|1.6|4.2|-5.9|***|
|Ndest_it_|2.4|2.5|-0.3||
|Nprod<sup>_ffv_</sup><br>_it_|1.6|4.2|-5.9|***|
|Nprod_it_|5.1|4.8|0.4||

Variables are detailed in Table 1. *, **, and *** denote statistical significance of the t statistics at the 10%, 5%, and 1% levels, respectively.

average numbers of products and destinations are very small.

# **3.2 Estimation Issues**

Estimating the effect of the PIP poses a standard missing-data problem—estimating how much smaller would have been the export flows that got assistance, had they not gotten assistance.<sup>16</sup> Formally, let and That is, _dikj_ marks the treatment group. The basic estimator for the problem at hand is the difference-in-differences (DID):

> 16Exports to non-EU destinations may be reduced because of a reallocation effect. If this is the case, the effect of the treatment on the treated would be further increased.

14
_yikjt_ = **x** _ikjtβ_ + _γdikjt_ + _δikj_ + _δt_ + _uikjt_ (1) where _δikj_ and _δt_ are respectively firm _×_ destination _×_ product and time fixed effects, and _uikjt_ is an error term. Fixed effects _δikj_ control for time-invariant firm characteristics potentially affecting both performance and selection into the program, like managerial ability (see Angrist and Krueger 1999, Smith 2000, or Jaffee 2002).

Next, we combine the DID estimator (1) with matching, following Heckman, Ichimura and Todd (1997). From now on, for simplicity of exposition, let us denote by a ‘flow’ a (firm _×_ product _×_ destination) triplet. Using results by Rosenbaum and Rubin (1983), matching is done on the basis of the estimated propensity score (PS), using a probit or logit regression of the participation status on a vector **z** of observable firm characteristics. Letting _vikjt_ be an error term orthogonal to _uikjt_ , the first-stage selection equation can be written as In (2), the vector **z** , which contains **x** and may be identical to it if no outside determinant of participation is available, must be evaluated at the time the participation decision is made–typically its initial value. The estimated propensity score is then retrieved from (2), and the control group is constructed by selecting untreated firms whose propensity scores are “close enough” to those of treated ones. How close is close is the analyst’s choice. Under nearest-neighbour matching, each treated flow is matched with the untreated flow having the closest PS, or with a combination of the _n_ closest untreated flows. Under kernel matching, flow _g_ is matched with a weighted average of untreated flows within a chosen radius, using either uniform weights or weights that decrease with distance in the PS space.<sup>17</sup> Practically, DID-with-matching estimation is done in two steps: In the first, the participation equation is estimated, yielding an estimated PS and a common support; in the second, the DID equation is estimated on the common support. The latter is formed by disregarding unmatched individuals as well as those with estimated PS of zero or one. Note that, if the first-stage regression predicts treatmentgroup status too well, it will reduce the common support and thus the sample on which the DID regression is run; if it predicts it too badly, the matching will be poor and the conditional-independence assumption will be unlikely to hold. The choice of RHS variables to include in **z** is thus a matter of judgement. The quality of the matching can be assessed by so-called ‘balancing tests’. One, proposed by Rosenbaum and Rubin (1985), compares the mean value of each covariate between 17 On this, see Leuven and Sianesi (2003) or Smith and Todd (2005).

15
the treatment and control group. When the difference is too large, the null hypothesis (that the samples are balanced with respect to the covariates when they are balanced with respect to the propensity score) is rejected.

We also control for selection using a control-function approach that closely resembles the Heckit procedure (Heckman 1979). The approach proceeds, again, in two steps. The first-step regression is as before, i.e. (2). In the second step, inverse Mills ratios retrieved from the first step are added to (1) as additional regressors.

Before we turn to the results, note that, besides selection bias, other issues may complicate the estimation of _γ_ in (1). One is serial correlation. Persistence in the process driving the error term may be aggravated by the extreme form of serial correlation in the treatment variable. Bertrand, Duflo and Mulainathan (2004) show that ignoring this source of serial correlation can lead to an inflated probability of type-I errors (wrongly rejecting the null hypothesis of no effect, i.e. being over-optimistic in the evaluation of the treatment’s impact). This calls for a correction that they suggest. The correction consists of a two-step procedure in which, in Step 1, individual performance for both treated and untreated individuals is regressed on all observables except the treatment. In Step two, residuals from Step 1 for the treated individuals only are retrieved and averaged for (i) the pretreatment period, (ii) the treatment period. The procedure requires a common treatment period, since otherwise the pre- and post-treatment periods would be undefined for the control group. Those average residuals form a two-period panel. They are then regressed on where _τ_ = 1 denotes the treatment period. The estimated ATT is the coefficient on _dikjτ_ in the second step.

# **4 Results**

## **4.1 Baseline Results**

Balancing properties are addressed by testing for equality of means between treated and matched controls for nearest-neighbour matching. Table 4.1 reports results from balancing tests. The table reports, for each covariate included in the probit model determining selection into treatment, the percentage bias after matching, the reduction in the bias, and the t-test statistics for the difference in means between treated and control groups after matching. Variables included in the 16
propensity score specification are the natural logarithm of firm _i_ ’s initial turnover, ln( _salesit_ 0, the natural logarithm of its initial number of employees, _ln_ ( _NEmployeesit_ 0), the initial number of products exported by firm _i_ to destination _j_ , _Nprodij_ 0, the initial number of countries served with product _k_ , _Ndestikt_ , the initial natural logarithm of total export value of FFV products from firm _i_ to destination _j_ .

Table 3: Balancing Properties of Covariates in Treated and Control Groups

||Sample|Mean|Mean|%|%|t-t|est|
|---|---|---|---|---|---|---|---|
|||treated<br>”fows”|control<br>”fows”|bias between<br>treated|reduction<br>bias|Mean(t<br>=Mean|reated)<br>(control)|
|||||and controls||t|p ¿ t|
|ln(sales_it_0)|Unmatched|21.39|21.25|11.5||1.3|0.19|
||Matched|21.39|21.56|-13.6|-18.6|-1.72|0.08|
|ln(Nemployees_it_0)|Unmatched|5.02|4.18|55.2||7.05|0.00|
||Matched|5.02|4.94|4.7|91.6|0.44|0.66|
|Nprod_ij_0|Unmatched|2.38|32.03|-144.2||-14.77|0.00|
||Matched|2.38|2.37|0.1|100|0.16|0.87|
|Ndest_ikt_|Unmatched|2.41|1.37|114.5||15.1|0.00|
||Matched|2.41|2.3|9.7|91.6|0.84|0.40|
|ln(total<br>~~e~~xport<sup>_FFV_</sup><br>_ijt_0 <sup>)</sup>|Unmatched|11.93|7.51|154||16.63|0.00|
||Matched|11.93|12.13|-7|95.4|-1.21|0.23|

Matching is by nearest neighbour. *, **, and *** denote statistical significance of the t statistics at the 10%, 5%, and 1% levels, respectively.

Results show that, for many covariates, there is a strong bias before matching but matching eliminates it. The null hypothesis of balanced sub-samples is not rejected except for turnover.

Table 4 reports difference-in-difference (DID) estimates on the treated, for our baseline specification. That is, the average effect of the PIP on assisted firms where the export performance indicator is the export value in US dollars, of product _k_ , from firm _i_ , to destination country _j_ , in year _t._ Column (1-3) reports DID estimates without matching, and with or without covariates. Column (4) reports 17
DID estimates with matching, i.e. restricting the sample to the common support defined in the NN-PSM procedure. Matching is done at the firm, product destination level. Column (5) reports treatment-effect estimates using Heckman’s two-step procedure, i.e. estimates from the second-step regression run with the inverse Mills ratio. Finally, column (6) reports results from the second stage of the procedure suggested by Bertrand, Duflo and Mulainathan (BDM) (2004). Results for the first step of the Heckman and BDM procedures are reported in Appendix 9 and 10. All regressions are run at the _i, k, j, t_ level and standard errors are clustered at the firm level. Here the variable of interest is the treatment indicator variable _treatmentikjt_ taking value one if firm _i_ exporting product _k_ to destination country _j_ in year _t_ , benefited from the PIP program.

Coefficients in all specifications are not significant, suggesting no effect of the program on firms export performance. The dependent variable is the natural logarithm of the export value of product _k_ from firm _i_ to country _j_ in time _t_ . All regressions control for (firm _times_ product _times_ country) and time fixed effects. Our main variable of interest is _treatmentikjt_ , a dummy variable taking value 1 if firm _i_ exporting product _k_ to country _j_ benefited from the PIP in year _t_ . Columns 1-3 show difference-in-difference estimations, where the control variables include the natural logarithm of annual turnover of firm i in year t ln( _salesit_ , the natural logarithm of the number of employees for firm _i_ in year _t_ , ln( _Nemployeesit_ ), firm _i_ ’s experience in servicing product _k_ to country _j_ , and a dummy variable taking value 1 if the firm exported at least two years product _k_ to country _j_ before time _t_ , _experienceikjt_ . Column 4 shows matching difference-in-difference estimation where results are reported using the Nearest Neighbourg (NN) estimator with caliper (0.04). Column 5 shows two-stage Heckman estimation where _λ_ is the inverse Mill’s ratio retrieved from the first step. The first-step is a regression of the participation status on a vector of observable firm characteristics (see Appendix 9). Column 6 shows two stage BDM estimation where residuals from the first step are retrieved and averaged for (i) the pre-treatment period, (ii) the treatment period. Step 1, individual performance for both treated and untreated individuals is regressed on all observables except the treatment (see Appendix 10).

18
Table 4: Baseline Results, Average Effect of the PIP on Assisted Firms.

||(1)|(2)|(3)|(4)|(5)|(6)|
|---|---|---|---|---|---|---|
||Dif<br>in<br>Dif|Dif<br>in<br>Dif|Dif<br>in<br>Dif|Dif<br>~~i~~n<br>~~D~~if<br>with<br>Matching|Two stage<br>Heckman|BDM<br>Correction|
|treatment_ikjt_|0.354|0.216|0.112|-0.013|0.238|0.111|
||(0.235)|(0.288)|(0.322)|(0.418)|(0.362)|(0.293)|
|ln<br>~~s~~ales_it_||1.062***|1.125***|1.246***|1.263***||
|||(0.212)|(0.262)|(0.243)|(0.246)||
|ln<br>~~N~~employee_it_|||-0.099|0.017|0.009||
||||(0.155)|(0.152)|(0.148)||
|experience_ikjt_|0.183*|-0.047|0.018|-0.143|-0.145||
||(0.104)|(0.168)|(0.172)|(0.169)|(0.169)||
|_λ_|||||-0.153||
||||||(0.153)||
|constant|7.94***|-14.59***|-15.94***|-18.98***|-19.39***|-0.073|
||(0.225)|(4.552)|(5.422)|(5.128)|(5.246)|(0.166)|
|Observations|1,862|1,193|1,071|698|698|176|
|R-squared|0.132|0.179|0.188|0.207|0.207|0.007|
|Number of id|1,134|657|577|369|369|155|

Robust standard errors clustered at firm level are in parentheses. *, **, and *** denote statistical significance at the 10%, 5%, and 1% levels, respectively.

# **4.2 Robustness**

In this section we present estimation results of the effect of the PIP on assited firms, for two alternative export performance indicators.

Table 5 reports DID estimates when considering firm _i_ export of product _k_ to the EU-15 in year _t_ , as the export performance indicator. Balancing tests results are provided in Appendix 7. There is no problem of unbalanced covariates in our model. All regressions are run at the _i, k, t_ level. The coefficients on the treatment variable are not significant for any of the specifications in Table 5. Results for the first step of the Heckman and BDM procedures are reported in Appendix 9 and 19
# 10.

Finally, Table 6 reports results from regressions run at the level of the firm instead of the firm _×_ product _×_ destination combination. Estimation at the firm level may drastically reduces sample size and means mixing up exports that are covered by the program with exports that are not (namely, products other than FFV). However, it is advisable, since the decision to participate and some of the covariates are at the firm level. The export performance indicator is the export value of FFV to the EU-15 exported by firm _i_ in year _t_ . All regressions are run at the _i, t_ level. The coefficient on the treatment variable is significant at the 5% level only in column (1). Results for the first step of the Heckman and BDM procedures are reported in Appendix 9 and 10.

All in all, results suggest that while there seem to be an effect of the program when considering total FFV exports to the EU, the effect disappears when looking at a more disaggregated level (Table 4 and 5). These results are in line with findings in the program evaluation report.

In Table 5, the dependent variable is the natural logarithm of the export value of product _k_ from firm _i_ to the EU-15 at time _t_ . All regressions control for firm x product and time fixed effects.

20
Table 5: Robustness I, Average Effect of the PIP on Assisted Firms.

||(1)|(2)|(3)|(4)|(5)|(6)|
|---|---|---|---|---|---|---|
||Dif<br>in|Dif<br>in|Dif<br>in|Dif<br>~~i~~n<br>~~D~~if<br>with|Two stage<br>Heckman|BDM<br>Correction|
||Dif|Dif|Dif|Matching|||
|treatment_ikt_|0.100|-0.023|-0.115|-0.114|-0.669|-0.410|
||(0.215)|(0.264)|(0.305)|(0.327)|(0.813)|(0.564)|
|ln<br>~~s~~ales_it_||0.909***|0.923***|0.948***|0.890***||
|||(0.217)|(0.287)|(0.293)|(0.290)||
|ln<br>~~N~~employee_it_|||0.063|0.074|0.078||
||||(0.105)|(0.107)|(0.103)||
|experience_ikjt_|0.299|-0.252|-0.228|-0.253|-0.213||
||(0.248)|(0.373)|(0.473)|(0.504)|(0.485)||
|_λ_|||||0.333||
||||||(0.357)||
|constant|8.95***|-9.36|-10.01|-18.98***|-8.54|0.17|
||(1.008)|(5.834)|(5.975)|(5.128)|(6.054)|(0.311)|
|Observations|681|368|288|281|286|93|
|R-squared|0.153|0.211|0.227|0.218|0.230|0.036|
|Number of id|373|189|142|139|140|78|

Robust standard errors clustered at firm level are in parentheses. *, **, and *** denote statistical significance at the 10%, 5%, and 1% levels, respectively.

21
Table 6: Robustness II, Average Effect of the PIP on Assisted Firms.

||(1)|(2)|(3)|(4)|(5)|(6)|
|---|---|---|---|---|---|---|
||Dif<br>in<br>Dif|Dif<br>in<br>Dif|Dif<br>in<br>Dif|Dif<br>~~i~~n<br>~~D~~if<br>with<br>Matching|Two stage<br>Heckman|BDM<br>Correction|
|treatment_it_|0.533**|0.381|0.666|0.666|0.711|0.477|
||(0.239)|(0.128)|(0.421)|(0.426)|(2.898)|(0.292)|
|ln<br>~~s~~ales_it_||0.529|0.496|0.520|0.518||
|||(0.372)|(0.532)|(0.538)|(0.626)||
|ln<br>~~N~~employee_it_|||-0.181|-0.188|-0.187||
||||(0.262)|(0.266)|(0.265)||
|experience_it_|-0.007|-0.007|-0.218|-0.237|-0.236||
||(0.233)|(0.233)|(0.460)|(0.468)|(0.530)||
|_λ_|||||-0.026||
||||||(1.570)||
|constant|10.07***|0.855|2.193|1.992|2.017|-0.278|
||(0.704)|(7.943)|(11.724)|(11.795)|(12.850)|(0.214)|
|Observations|199|119|85|79|79|16|
|R-squared|0.301|0.380|0.354|0.362|0.362|0.308|
|Number of id|69|38|26|21|21|9|

Robust standard errors clustered at firm level are in parentheses. *, **, and *** denote statistical significance at the 10%, 5%, and 1% levels, respectively.

# **5 Concluding Remarks**

By and large, we find no significant impact of the PIP on Senegal’s FFV export flows when taking similar, untreated export flows as the counterfactual. There are two ways of interpreting such a no-impact result.

The naive interpretation is that the PIP simply fails to achieve its objective. That may well be true, but our failure to reject the null of no impact is not sufficient to reach that conclusion. First, as we briefly discussed in the introduction, the choice of Senegal as a testing ground was driven by data availability (Senegal 22
was the only country for which we had data for the pre-treatment period). It has no claim to be a representative or random sample. Different conclusions may be reached from other samples, and clearly a full, cross-country impact evaluation should be undertaken. Second and more importantly, it is possible that the PIP affected not only the treated export flows, but also untreated ones, through spillovers. Participating firms are the largest and more efficient of Senegal’s FFV sector. On one hand. this means that they are more susceptible than others to benefit from the program. Thus, what we obtain is an estimate of the average effect of the treatment on the treated (ATT), which may over-state the program’s potential effect on the whole population of producers. On the other hand, it may also mean that smaller firms, although left out of the program, can benefit from it through imitation of best practices and even unobserved assistance from larger firms. The argument is even more potent for untreated _products_ : when a firm gets PIP assistance for its FFV activities, it is quite possible (indeed, likely) that its other activities benefit as well from improved managerial practices; or _destination countries_ : export flows to non-EU destinations may benefit from the program for the same reason. In the presence of such unmeasured spillovers, the PIP’s impact would be underestimated by impact-evaluation methods. This is important to keep in mind, as public assistance (whether from local governments or donors) should be justified by market failures, like spillovers, rather than a positive rate of return to beneficiaries (which would simply create a market demand for assistance services without justifying use of public funds). Thus, impact evaluation of technical-assistance programs like the PIP is a double-edged sword and must be interpreted with caution.

The second conclusion that should be _avoided_ is that, either because the data are not sufficiently reliable or comprehensive or because of the caveats just discussed, rigorous impact evaluation should not be undertaken. The lack of rigorous impact evaluation undermines the credibility of claims about the program’s benefits made on the basis of case studies, because it is impossible to know whether they are representative or not. Indeed, this is the message conveyed by the 2006 assessment of the impact of EU TRA (te Velde et al. 2006). Worse, in a context where taxpayers are asking for acccountability and results in development aid, Paul Milgrom’s ‘unraveling principle’ applies: Rational taxpayers are likely to take all the news that is not told to be detrimental. In other words, the bad news that impact evaluation can possibly generate (as in the present case) is probably fully anticipated. The more the development community can provide rigorous evidence that at least some programs do make a difference; or that some components do; or that, when not, failure is part of useful experimentation and action is being taken to remedy the observed ineffectiveness, the more support there will be for development aid.

23
However, as the present study highlights, it is difficult to ‘improvise’ impact evaluation _ex-post_ when a program was not designed to be evaluated. Far better would be to think seriously about evaluation _ex-ante_ , so that TRA programs generate experimental settings out of which useful lessons could be drawn. We hope that this study will help convince the European Commission and other development agencies of the need to plan for impact evaluation at program-design time. By this, we mean to (i) clarify what measurable performance indicator the program seeks to improve; (ii) collect, before, during and after the program period, the data needed to track this indicator for treated and non-treated firms and products, as well as its non-program determinants; (iii) amend the design of the program (in particular the assignment rule) to ensure the existence of a proper control group against which to benchmark its impact.

24