metadata: authors: 0: author: Ravallion,Martin
count: World
docna: 0: docna: Knowledgeable bankers ? the demand for
            research in World Bank operations
docty: Policy Research Working Paper
subtopic: Economic Conditions and Volatility,Educational Sciences,Food Security,Energy Demand,Energy and Environment,Energy and Mining,Inequality
teratopic: Agriculture,Information and Communication Technologies,Poverty Reduction,Rural Development,Finance and Financial Sector Development
abstracts: cdata!: Development impact calls for
            knowledgeable development practitioners. How then do the
            operational staff of the largest development agency value
            and use its research? Is there an incentive to learn and
            does it translate into useful knowledge? A new survey
            reveals that the bulk of the World Bank's senior staff
            value the Bank's research for their work, and most come
            to know it well, although a sizable minority have difficulty
            accessing research to serve their needs. Another group sees
            little value to research for their work and does not bother
            to find out about it. Higher perceived value is reflected in
            greater knowledge about research, though there are frictions
            in this process. Staff working on poverty, human development
            and economic policy tend to value and use research more than
            staff in the more traditional sectors of Bank lending --
            agriculture and rural development, energy and mining,
            transport and urban development; the latter sectors account
            for 45 percent of lending but only 15 percent of staff
            highly familiar with Bank research. Without stronger
            incentives for learning and more relevant and accessible
            research products, it appears likely that this lag in demand
            for research by the traditional sectors will persist.
repnme: repnme: Knowledgeable bankers ? the demand for
            research in World Bank operations
display_title: Knowledgeable bankers the demand for
            research in World Bank operations
listing_relative_url: /research/2011/12/15544357/knowledgeable-bankers-demand-research-world-bank-operations
topicv3: Macroeconomic Vulnerability and Debt
            m1326057 1104,Agriculture and Food Security m1326205 435,Energy and Extractives m1326595 507,Poverty m1326687 695,Education m1327048 470
publishtoextweb_dt: 2011-12-01T00:00:00Z
historic_topic: Agriculture,Information and Communication Technologies,Poverty Reduction,Rural Development,Finance and Financial Sector Development
no_of_pages: 41
author_keys: 0: author_key: 99002
model_extractions: WPS5892 Policy Research Working Paper 5892 Knowledgeable Bankers? The Demand for Research in World Bank Operations _Martin Ravallion_ The World Bank Development Research Group Director’s office December 2011
Policy Research Working Paper 5892

# **Abstract**

Development impact calls for knowledgeable development practitioners. How then do the operational staff of the largest development agency value and use its research? Is there an incentive to learn and does it translate into useful knowledge? A new survey reveals that the bulk of the World Bank’s senior staff value the Bank’s research for their work, and most come to know it well, although a sizable minority have difficulty accessing research to serve their needs. Another group sees little value to research for their work and does not bother to find out about it. Higher perceived value is reflected in greater knowledge about research, though there are frictions in this process. Staff working on poverty, human development and economic policy tend to value and use research more than staff in the more traditional sectors of Bank lending—agriculture and rural development, energy and mining, transport and urban development; the latter sectors account for 45 percent of lending but only 15 percent of staff highly familiar with Bank research. Without stronger incentives for learning and more relevant and accessible research products, it appears likely that this lag in demand for research by the traditional sectors will persist.

This paper is a product of the Director’s office, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at mravallion@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Knowledgeable Bankers?

The Demand for Research in World Bank Operations

## Martin Ravallion<sup>1</sup>

_Development Research Group, World Bank_

### **Keywords:** Research, incentive to learn, absorptive capacity, operations, development aid

> 1 This is a background paper for the 2011 _Report on the World Bank Research Program._ The paper draws on a survey of the views of World Bank staff carried out by Sharon Felzer and Jessica Cameron. The overall task manager for the Report was Jean-Jacques Dethier and the director responsible was Asli Demirguc-Kunt. The author is grateful to Asli for encouraging him to write this paper and to Asli, Marianne Fay, Aart Kraay, Tamar Manuelyan Atinc, Will Martin, Berk Ozler, Mike Toman, Dominique van de Walle and Adam Wagstaff for useful comments. However, the author alone is responsible for the interpretations of the survey data provided in this paper. In particular, the findings, interpretations and conclusions of this paper should not be attributed to the World Bank or any affiliated organization.
# **1. Introduction**

In trying to understand the impact of development aid, much attention has been given to the role played by the circumstances of recipient countries, notably whether their policy environment and governance are conducive to high social returns from aid.<sup>2</sup> The literature has given less attention to the role played by the staff and managers in donor organizations. In the case of the World Bank, there have been concerns about the Bank’s ―lending culture,‖ which tends to reward operational staff for the volume of their lending, with (it is argued) too little weight given to the quality of lending.<sup>3</sup> In one of the few studies of aid effectiveness to look at these issues, Denizer, Kaufmann and Kraay (2011) confirm that the ―macro‖ variables concerning recipient countries are relevant to the (subjectively but independently assessed) quality of the Bank’s development projects. But they also find that the bulk of the variance in the quality of the Bank’s lending operations is within countries rather than between them. The quality of the staff in charge of projects on the donor’s side matters at least as much as the policy environment on the recipient’s side.

The knowledge of operational staff within donor agencies is likely to be important to the quality of development aid. There is evidence that the stock of prior analytic work by the Bank on a recipient country is a strong predictor of the subsequent quality of its lending operations to that country (Deininger, Squire and Basu, 1998; Wane, 2004) and that the quality of prior analytic work matters to the quality of its projects (Fardoust and Flanagan, 2011). The stock of practitioners’ knowledge depends on their demand for knowledge, as well as its supply. Thus the incentives for learning within aid organizations have been identified as one factor in the quality of development aid (Wane, 2004; Ravallion, 2011). The generation of relevant knowledge and its diffusion within donor agencies is a poorly understood factor in development effectiveness.

While research is clearly a key element of knowledge generation and innovation, any large and complex organization like the World Bank will face challenges in assuring that

> 2 An influential early paper arguing this point was Burnside and Dollar (2000), although others have questioned their findings (including Hansen and Tarp, 2001). For a critical overview see Temple (2010).

> 3 20 years ago, the Wapenhans (1992) report laid out these concerns in forthright terms. Since then there has been an effort to do better through tighter quality control on projects at entry and better project implementation practices. But few observers would argue that these concerns are no longer salient.

2
relevant basic research is both produced and used.<sup>4</sup> As the literature on organizations has emphasized, a key factor is the ability to exploit the internal division of labor to support innovation (Hage, 1999). The role of a designated ―research department‖ in bringing together internal research capabilities and in absorbing new external information to assure innovation has long been recognized in the literature (following, in particular, Cohen and Levinthal, 1990). But having such a department does not guarantee that practitioners will have the incentive to learn from research, and that the research produced will serve their needs. In practice, there may well be frictions within an organization that inhibit learning even when the incentive is there.

The World Bank (WB) has a department dedicated to research, the Development Research Group (DECRG), within the Development Economics Vice-Presidency (DEC), as well as researchers scattered across other units. Research accounts for roughly 1% of Bank staff; there are about 75 full-time research staff in DECRG and probably another 25 or so full-time equivalent researchers outside DECRG.<sup>5</sup> DECRG spans all sectors of the Bank’s work. Its research is almost invariably empirical and typically draws on economics as the primary discipline, though increasingly drawing on insights from other social sciences.<sup>6</sup> By objective criteria, the scale and quality of its research on development exceeds that of virtually all other international agencies and universities.<sup>7</sup> DECRG aims in part to serve the needs of its operational units, which provide development lending and policy advice to developing countries.<sup>8</sup> However, we know very little about the demand for the Bank’s research among the organization’s operational staff. How familiar are they with Bank research? How much do they rely on it for their work? How do the answers to these questions vary across units and sectors of

> 4 On the (broadly similar) challenges facing private firms in deciding whether to invest in basic research see the discussion in Rosenberg (1990).

> 5 A precise count is not available. My count excludes research assistants.

> 6 It has been argued that the heavy reliance on economics reflects an undesirable lack of ―ideological pluralism‖ at the Bank (Weaver, 2009, p. 505). While multiple ideologies might well balance out, Bank research (in common with mainstream economics and other social sciences) aims to be non-ideological. It is unlikely that the work of DECRG would attain its current high level of academic credibility if it failed in that aim.

> 7 This is based on the IDEAS rankings using a composite index of 20 variables measuring publications, downloads and citations based on the RePEc data; the rankings and details can be found here. If one confines attention to the Bank’s research department then the University of Chicago alone does better. Also see the data on citations assembled in Ravallion and Wagstaff (2010).

> 8 External audiences are also important but are not the focus of this paper. For data and analysis on the impacts of World Bank research on external clients see Ravallion and Wagstaff (2010).

3
the Bank? We know even less about the incentives for learning among the Bank’s operational staff. Those incentives depend crucially on the value attached to WB research by staff for their work. Do the Bank’s practitioners value research for their work, and (if so) does this incentive to learn translate into greater familiarity and use of the Bank’s research?

This paper tries to throw new light on the demand for research by development practitioners within the World Bank. A key empirical question is how operational staff are mapped into the four groups identified in the following table. If the Bank’s operational staff highly value Bank research for their work, and the research done is relevant to their needs and accessible to them, then the bulk of staff will fall into the category ―functionally well-informed.‖ For this group, the incentive to learn comes with acquired knowledge. There may also be staff who are well-informed about research, but not because it matters to their work; they are ―independently well-informed.‖ And some value research for their work but face hurdles in accessing it, so their familiarity is low; these are the ―frustrated uninformed.‖ The remainder comprise those who do not see much value to Bank research for their work, and so do not seek it out; they can be said to be ―happily uninformed.‖

|||Incentive to learn: Perceived val<br>Low|ue of Bank research for own work:<br>High|
|---|---|---|---|
|||**“Happily uninformed:”**|**“Frustrated uninformed:”**|
|Knowledge:|L|Do not feel the need for research|Need to know more from research|
|Staff member’s|ow|and do not have a general interest|but cannot access or finds current|
|personal||in learning from new research.|research of little practical use.|
|familiarity with<br>the Bank’s<br>|Hih|**“Independently well-informed:”**<br>Do not need research for own|**“Functionally well-informed:”**<br>Research is an important input to the|
|research findings|g|work, but has a general interest in<br>learningabout development.|staff member’s workand<br> access to<br>relevant research is not aproblem.|

The paper uses a specially commissioned survey of the World Bank’s senior operational staff to determine how they are mapped to these four groups. The paper’s analysis of these data aims to better understand the demand for Bank research by the Bank’s practitioners—reflecting both their incentives to learn and the responsiveness of the supply of research to their needs.

# **2. The survey**

World Bank staff typically work in (overlapping) teams assigned to specific projects, each team having a ―Task Team Leader‖ (TTL) in the Bank’s terminology. The population for 4
the survey was as all ―senior staff,‖ defined as those at grades ―GG‖ and above, which account for about one-quarter of all staff. They are all likely to be the TTLs of one or more operations. The survey was only sent to GG+ staff excluding DEC.<sup>9</sup> The survey was implemented using a confidential, web-based, survey tool. On top of basic information on Vice-Presidential Unit (VPU), ―sector‖ (essentially synonymous with what is called a ―network‖) and years of Bank service, the survey asked a series of questions on familiarity with WB research, sources of knowledge about that research, and the value of WB research.

There were 2,900 recipients of the survey instrument and 555 responses. There was only a modest variation in response rates across grades, namely 18%, 21% and 15% for grades GG, GH and GI+ respectively (as compared to 19% overall).<sup>10</sup> More worrying is that the response rate varied markedly across VPUs, as can be seen in Table 1. The lowest response rate was for the Middle East and North Africa (MNA) at 12%, while the highest was for Poverty Reduction and Economic Management (PRM) at 50%. To the extent that such differences in response rates are correlated with perceptions of research there will be a bias for drawing inferences about the means for the population of the Bank’s senior staff, although correlations and regressions are likely to be more robust.

# **3. Knowledge about research among the Bank’s operational staff**

Learning about research findings is not costless. To be well-informed, practitioners must spend precious time searching out and studying research products. The costs for doing so will vary across staff, as will the expected benefits.

The survey asked a number of questions about familiarity and use of World Bank research. The key question was as follows: _“How familiar would you say you are with WB research products/services on a scale from 1 to 10 where 1 means not familiar at all and 10 means extremely familiar?”_<sup>11</sup> Figure 1 gives two density functions for the responses, one for the

> 9 Although six responses came from DEC, presumably reflecting changes in VPU assignments.

> 10 These are based on the grade distribution for Operational Units at September 2011 (67.7%, 27.3% and

> 5.0% in grades GG, GH and GI+ respectively).

> 11 The other options allowed were: ―Not applicable‖ and ―Prefer not to answer.‖ All calculations in this paper filter out these responses as missing values.

5
raw data and one with regression adjustment for years of service at the WB (and its squared value) and the sector and Vice-Presidency of the respondent. The densities are fairly symmetric around a mode and median of 6, and mean of 5.74, with a standard error of 0.10.

To interpret an average score of 6 we should recall the fact that researchers are a small proportion of the Bank’s staff count—somewhere around 1%. To help interpret any given mean score, let us assume that—through their writings, presentations and personal contacts—the 100 or so researchers influence an average number of _n_ Bank staff to become _“very familiar”_ with research over the relevant time period. Of course there is an overlap, in that the same operational staff come back again and again. So _n_ should be interpreted as the mean net gain attributed to the influence of individual researchers. To keep this simple, let us also assume that the rest of the staff know nothing about research. So the researchers are able to make 100 _n_ staff very familiar— a score of 10 on the scale. There are about 2,900 staff in the relevant population. Clearly, to achieve the top score of 10, researchers would need to bring _n_ =29 staff on average up to a 10. What then will be the value of _n_ implied by an average score of _s_ ? It is readily verified that the answer is 3.22( _s_ -1). So a mean score of 6 requires _n_ =16; individual researchers would have made an extra 16 Bank staff (on average) ―very familiar‖ with WB research. To get a mean score of 8 they would need to reach 23 Bank staff; by contrast, if they only reached 10 staff the mean score would be 4.

There is substantial variation in familiarity with research across Bank units and sectors of work. The Bank has a matrix structure whereby staff are mapped to both a VPU and a ―sector‖ (also called ―network‖). Table 2 gives summary statistics stratified by VPU. There is less variation in familiarity between regional VPUs than sectoral VPUs. Among the Bank’s regional VPUs, the lowest mean score for familiarity is for East Asia and the Pacific (EAP) while the highest is for the Middle-East and North Africa (MNA). Defining ―high familiarity‖ as those with a self-rated score of 6 or more, we find that over two-thirds of respondents in MENA have high familiarity, as compared to less than 45% in EAP. Of course, not all of these differences are statistically significant; indeed, only for two VPUs—namely the Human Development Network (HDN) and PRM—are the differences in either the mean score or the proportion with high familiarity significantly different (at the 5% level) from the proportion for the World Bank Institute (WBI). (When relevant, all significance tests reported are based on robust standard 6
errors.) However, the difference between sectoral VPUs is striking with only 36% of WBI staff rating their familiarity over 5, as compared to 81% in HDN and 71% in PRM.

A potential source of bias is suggested by the fact that the two VPUs with the highest response rate also had the highest familiarity with WB research, namely HDN and PRM (comparing Tables 1 and 2). The overall correlation coefficient between response rate and mean familiarity across the 12 VPUs is 0.65. However, re-weighting the data to accord with the population proportions (the numbers of people invited across VPUs in Table 1) only reduces slightly the mean familiarity score, to 5.66 as compared to 5.74 for the un-weighted data. While there is clearly some sample selection bias, it appears to be modest. There may well be a bias within VPUs based on other unobserved factors, though this cannot be assessed. The rest of this paper will rely mainly on the un-weighted data.

The differences in the extent to which different VPUs draw on the Bank’s research can be expected to reflect differences in their staffing. As noted, WB research is dominated by economic analysis. Some VPUs have hired more staff trained in research, as indicated by a PhD (in any subject), and some rely more on economists. Low demand for WB research might stem from high internal capacity for research in a VPU. Alternatively, internal capacity for research may enhance the unit’s absorptive capacity for learning and innovation (as postulated by Cohen and Levinthal, 1990).<sup>12</sup> Internal capacity reduces search costs and creates the kinds of overlaps in knowledge that facilitate the assimilation of new knowledge from external sources.

The idea of absorptive capacity from the literature on industrial organization is consistent with what we see when we look at how the incidence of senior staff with a PhD and the share of ―economists‖ vary across VPUs.<sup>13</sup> On comparing the proportion of PhDs with the mean familiarity scores in Table 2 one finds a correlation coefficient of 0.66 while it is 0.54 for the % above 5; these are significant at the 1% and 2% levels respectively. There is an even higher correlation across VPUs between average familiarity with WB research and the share of staff that have ―Economist‖ in their job title (Table 1). The correlation coefficients between the ratio of

> 12 Following Cohen and Levinthal (1990), the literature on innovation within firms has emphasized the role of internal knowledge and its diffusion; see, for example, Fabrizio (2009), which provides evidence using firm-level data that in-house research capacity reduces search costs for new innovations.

> 13 The calculation of the share of PhDs for GG+ staff was for 2009. The number of ―economists‖ is measured by the number of staff who have the word ―economist‖ in their job title. This excludes managers.

7
economists to senior staff (sent the survey) and the mean score for familiarity is 0.84, while the correlation with the % above 5 is 0.75; these are both significant at the 1% level. It appears that economists and staff with a PhD play a key role in the diffusion process for WB research throughout the institution.<sup>14</sup> An objective measure of demand for WB research is found in the intra-Bank financial flows designated as (paid) ―cross-support.‖ Each of the research staff in DECRG is required to sell a minimum of 13 weeks of their time per year to other units, almost all of which are in operations, either regional units or network anchors.<sup>15</sup> This arrangement exists in part to help assure that researchers are exposed to the problems and challenges faced in the Bank’s operational settings in the hope that this will generate relevant research, but also that operational staff will learn to better appreciate the gains from research. TTLs in operations typically come to the researchers with a specific task, such as supervising new data collection or undertaking analytic work for an analytic product, called ―AAA.‖ Table 1 gives the three-year mean cross-support provided by DECRG to each VPU. When deflated by the number of GG+ staff, the ―cross-support per capita‖ is correlated with the mean familiarity scores across VPUs; the correlation coefficient is 0.53 (which is significant at the 1% level). Cross-support is also higher to VPUs with a higher share of PhDs (r=0.74) and more economists per capita (r=0.79).

The differences between the Bank’s sectors come out sharply in Table 3.<sup>16</sup> Over 80% of responding staff in the Poverty unit of PRM rate themselves as highly familiar with WB research, with a mean score of almost 7. Yet less than one third of those in the Energy and Mining (EM) and Urban Development (URB) had high familiarity. There is a clear tendency for lower familiarity with WB research amongst the Bank’s traditional ―infrastructure‖ sectors— agriculture and rural development, energy and mining, urban development, transport—than the

> 14 This is consistent with the role economists have played in the diffusion of economic policy ideas more generally (Kogut and Macpherson, 2011).

> 15 The total of this paid cross-support amounts to over 20 person years per year of direct support by researchers. However, that is still a very small share of the Bank’s staff—the total paid cross-support supplied by the Bank’s research department is only around 0.3% of all Bank staff. Indeed, only 1% of which are in the only department dedicated to research, although research is also done in other units.

> 16 ARD, EM, ENV, SDV. TRN and URB all map to SDN; EP, POV and PSG map to PRM; EDU, HNP and SP map to HDN. Some sectors were not explicitly identified by the survey, such as Gender. Note that the aforementioned data on PhDs, economists and cross-support are only available at VPU level, not by sectors.

8
newer ―economic and human development sectors‖ (economic policy, poverty, education, health and nutrition, and social protection). The mean score for ARD, EM, TRN and URB is 5.19 (s.e.=0.24) as compared to 6.48 (0.14) for EP, EDU, HNP, POV and SP (and the difference is significant; t=4.66). The corresponding proportions of staff with a high familiarity score (over 5) are 44.58% (5.47%) and 70.68% (3.30%). In all, 41 staff in the ―hard sectors‖ (ARD, EM, TRN and URB) gave a score of 6 or more, while 279 did so in the sample as a whole. Thus the hard sectors accounted for 15% of the staff with high familiarity. By contrast, these sectors accounted for 45% of Bank lending.<sup>17</sup> Table 3 also gives my estimates of the allocation of DECRG’s staff (including managers) across sectors.<sup>18</sup> Mean familiarity amongst operational staff is positively correlated with DECRG’s staffing, with a correlation coefficient of 0.527 (t=2.85; prob.=0.016). The positive correlation is consistent with the view that staffing of DECRG matters to familiarity with research amongst the Bank’s operational staff. The marked difference in familiarity with research seen in Table 3 match the allocation of staff in DECRG. The sectors ARD, EM, TRN and URB account for 14.9% of staff in the research department, as compared to 59.1% for EP, EDU, HNP, POV and SP.

The survey also asked for a rating (also on a 10-point scale) of other sources of research besides the Bank.<sup>19</sup> The two most important sources for Bank operational staff are likely to be academia and external consultants. Tables 2 and 3 also give the mean scores by VPU and sector respectively. In both cases, the mean scores tend to be positively correlated with ratings of familiarity with the Bank’s internal research.<sup>20</sup> This is also true across staff, with a positive correlation between ratings of familiarity with WB research and ratings of both academia and

> 17 Based on aggregates for the last three years for lending on ―agriculture, fishing and forestry,‖ ―energy and mining,‖ ―transportation‖ and ―water, sanitation and flood protection‖; the sectoral breakdown of Bank lending can be found in World Bank (2011); the relevant tabulation is here.

> 18 I did this by allocating each staff member a primary and secondary sector in 2011—based on their research outputs—and giving two-thirds weight to the former. This requires a degree of judgment, but the allocation proved relatively easy. Note that the high proportion of staff in ―economic policy‖ might be somewhat deceptive, since these are often researchers who are relatively fungible, and work across many development topics.

> 19 The specific question was as follows: _“When considering all the sources of research globally, how useful is each of the following in terms of its research and knowledge have a positive impact on the quality of your AAA and/or lending operations?”_

> 20 Across sectors there is a very high correlation between familiarity with Bank research and ratings of academia (r=0.81). It is less strong for consultants but still positive (r=0.31).

9
external consultants, though only significant for academia.<sup>21</sup> It does not appear to be the case that the sectors and staff that know less about the Bank’s own research tend to value these external sources more highly.

Why then do we see these differences in familiarity with WB research? It might be argued that they just reflect the nature of the work done by operational staff in different sectors. By this view, staff in the ―hard infrastructure sectors‖ do not need the Bank’s research as much as in the ―social sectors.‖ To paraphrase an argument I have heard: _“We all know that roads are good for development and we don’t need economic analysis to confirm that; what we need is to make sure that the roads meet sound engineering standards.”_ It is hard to say what validity should be attached to this view. There are clearly differences in the nature of the knowledge needed by staff in different sectors. But the need for research is surely common between the ―hard‖ and ―soft‖ sectors. Economic analysis is important in formulating strategies and policies for infrastructure, notably in assessing the effectiveness of lending operations (both _ex ante_ and _ex post_ ) and in devising sound pricing and investment policies. And the trade-offs invariably faced by governments between building roads and schools (say) must be informed by economic analysis. The uneven familiarity with WB research across sectors evident in Table 3 is not easily justified from the point of view of sound development policy and WB lending.

It might be conjectured that the fact that economics is the dominant discipline in Bank research makes that research less relevant to some sectors. As discussed above, this does not seem credible as an explanation for why Bank research is used less in the traditional infrastructure sectors. Nor does it fit well with the fact that staff in Health, Nutrition and Population (HNP)—dominated by non-economists—are very familiar with research (Table 3). Possibly the idea has more salience for the Social Development (SDV) sector, which uses noneconomics social sciences more than economics. (Note that SDV has the third lowest mean familiarity score in Table 3.) This division of labor is not, however, exogenous, but has evolved in a path-dependent way through hiring decisions. And it can change, albeit slowly. For example, DECRG’s latest _Policy Research Report_ (its flagship publication) is a substantial evaluative study, done by two economists, of the main lending instruments used by SDV operations

> 21 The correlation coefficients between individual ratings for familiarity with WB research and academia is

> 0.37 (significant at the 0.1% level) and 0.09 (only significant at the 12% level) respectively.

10
(Mansuri and Rao, 2011). This report stemmed from demands from operations for more knowledge about the impacts of these interventions.

To get an idea of how demand for research amongst operational staff is changing, the survey also asked: _“To what extent do you currently rely on Bank research for your work?”_ and _“To what extent do you expect to rely on Bank research for your work in the next few years?”_ Again, respondents were presented with a 10 point scale, this time from ― _not at all_ ‖ (1) to ― _very much_ ‖ (10). Figure 2 gives the density functions for responses. The responses on current reliance on WB research are strikingly bi-modal, with a ―low-usage‖ mode around 3 and a high-usage mode around 7. The overall mean response is 5.40 and the median is 6, though this might be considered deceptive given the bi-modality. If we define the low-reliance and high-reliance groups as those with a score of 5 or lower versus 6+ this splits the sample almost equally, with 50.91% in the high group.

Equally striking is the fact that the lower mode largely vanishes when one turns to the second question, about future reliance on WB research (Figure 2). This reflects a noticeably higher expected future reliance on WB research than for the past. This is consistent with the view that the WB is becoming a ―Knowledge Bank.‖ But it might also be a psychological effect—a ―New Year’s resolution‖ effect—whereby staff hope to learn more from research.

Those who have used WB research in the past are more likely to expect to use it in the future; the correlation coefficient is 0.87 (significant at better than the 0.1% level). (With controls for years of service, location, VPU and sector the correlation is 0.85, and similarly significant.) One possibility is that staff learn about how useful research can be, creating persistence; another possibility is that the correlation stems from time-invariant characteristics of respondents, whereby some are intrinsically more attracted to research.

Figure 3 plots responses to the question on expected future reliance on WB research against that for past reliance. (In both cases, the scatter plot controls for respondent characteristics, namely years of service, location, VPU and sector.) While we see a strong positive relationship, the overall slope is less than unity. (The straight line in Figure 3 is the 45 degree line, while the other line is the non-parametric regression line of best fit.) Across individual staff, the regression coefficient of the expected increase in demand (future less 11
current) on the current reliance on research is -0.175 (s.e.=0.027). This suggests that the disparities in usage of research can be expected to decline over time, interpretable as ―convergence.‖ It is of interest to look more closely at where this expected increase in demand for WB research is coming from. Tables 4 and 5 give the summary statistics across VPUs and sectors respectively. The VPU for which demand for research is expected to rise most on average is MNA while the lowest expected increase is for PRM. MNA is also the regional VPU with highest mean score for current reliance on research. The sector where demand is expected to increase most is EM while the lowest is SP. In both cases there is a tendency for the units that currently rely less on WB research to expect larger growth in usage in the future; the correlation coefficients between the expected increase and current levels are -0.32 and -0.22 across VPU and sectors respectively.

However, it remains that expected future reliance on WB research is significantly lower for the sectors with low current familiarity and use. The mean score for future reliance on research for staff in the seven sectors with lowest mean familiarity scores in Table 3—namely ARD, EM, ENV, PSG, SDV, TRN and URB—is 5.55 (s.e.=0.21) as compared to 6.29 (s.e.=0.14) for all other staff and the difference is significant at the 0.3% level (t=2.96).<sup>22</sup> Confining attention to the traditional ―hard sectors‖ (ARD, EM, TRN and URB) their mean expected future reliance on research is 5.34 (0.28) as compared to 6.72 (0.16) for EP, EDU, HNP, POV and SP (and the difference is significant; t=4.31). Even with the signs of convergence we have seen, the hard sectors can be expected to persist in their relatively low reliance on research for their work.

# **4. Accessing research within the Bank**

How do the Bank’s practitioners obtain the research they use? In addition to papers and reports, there are learning programs, seminars, formal and informal contacts with researchers, and formal and informal networks. The survey data on self-assessed familiarity with Bank

> 22 All but one of these sectors is mapped to SDN; the mean future reliance scores are 5.49 (s.e.=0.23) and 6.25 (0.14) for SDN and non-SDN sectors, and the difference is significant (t=2.85). Note that the staff in sectors mapped to SDN need not be in the SDN VPU; many are in regional units. However, there is very little difference in the score between staff in these sectors who are in the SDN VPU versus other VPUs; the mean scores are 5.52 and 5.47 respectively (and the difference is not significant; t=0.09). It is a ―sector effect‖ rather than VPU effect.

12
research analyzed above offer a way of assessing how well these various channels are performing. This reveals some sources of friction in the diffusion process for research findings.

The Bank has a dedicated formal expenditure item for ―learning.‖ This is mainly staff time devoted to attending various internal learning events. Tabulations are available of the expenditures devoted to learning by VPU. I find that the expenditure per staff member in 201011 is uncorrelated with familiarity scores across VPUs; the correlation coefficients are 0.01 and 0.11 for the mean score and the % of 5+.<sup>23</sup> So this does not appear to be an important channel for learning about the findings from WB research. However, there is likely to be some noise in these data, due to errors in time-recording by staff.

The survey asked the following question that gets at a broader range of channels for learning: _“When obtaining research from the World Bank, how frequently do you get the research from:...”_ Respondents were then presented with a list of options and a 10-point scale. Table 6 summarizes survey responses. The most highly-rated sources of WB research are ―reports or papers,‖ the Bank’s intranet, flagship reports, and the Bank’s main working paper series. Seminars etc., and authored reports are close behind. The lowest rated source (on average) is ―DECRG researcher cross-support.‖ Arguably a better way of thinking about the relative importance of these various sources of knowledge is to ask how well they predict the respondents’ own assessments of their familiarity with WB research. Staff may use source A more frequently than B (with hence a higher score for A in the survey response) yet B has the stronger impact on familiarity with research. For example, when a researcher provides cross-support to a TTL in operations, he or she will probably get a deeper understanding and appreciation of the contributions of research to operational work than reading a final research product that is less distantly related to the needs of that person. The TTL may read more papers, but the cross-support has greater impact.

To explore this alternative interpretation, Table 7 gives a regression of the familiarity scores on the scores for sources of knowledge. I include controls for location, sector and VPU; filtering out these sources of heterogeneity helps in identifying the relationship in the data. In explaining the differences in overall familiarity with WB research, the use of DECRG

> 23 N=11; the data were missing for WBI.

13
researchers in cross-support now emerges as the most important factor, with both the highest regression coefficient and the most significant. Also important are informal discussions with researchers and journal articles, though the latter variable is only significant at the 12% level. (The results were similar if one dropped the controls for years of service and VPU/sector.) The direct (formal and informal) interactions with DECRG researchers (including cross-support) have high weights in affecting familiarity with WB research, even though they score relatively poorly in the mean ratings for frequency of their use given in Table 6.

Similarly to other large organizations, the Bank has created official ―networks.‖ (Recall that these map 1-to-1 with ―sectors‖ in the Bank’s matrix structure.) The networks are intended to play a key role in linking sources of knowledge (including WB research) with practitioners within the Bank. How well do they serve this function? The survey asked: ― _To what extent does the network that you belong to help you navigate the World Bank’s body of research and researchers when you request information?”_ Again, respondents were asked to answer on a 10 point scale from 1=― _Not at all_ ‖ to 10=― _Very much_ .‖ The mean response was 4.86 and the median was 5. The modal value was 1; 15% of respondents gave this answer.

The differences across sectors can be seen in Table 8. The most highly rated sector is education, for which 70% of those mapped to the sector rated its performance as a 6 or higher, as compared to 42% for all staff. The next highest in terms of mean score is HNP, with POV coming third; the sector with the lowest mean score is ENV, followed by SDV.

Having a network that performs poorly adds friction to the diffusion process for Bank research. This is evident in the fact that the performance ratings of the networks are positively correlated with familiarity with research. This is clear from comparing the sector means in Tables 3 and 8; the correlation coefficient between mean scores is 0.508. Across all respondents the correlation coefficient is 0.230 (which is statistically significant at the 0.1% level).

The in-house research department (DECRG) is the key unit aiming to supply relevant research findings to Bank operations, but there are differing views about its success in this role. The survey asked: _“Which of the following statements best represents your views about research at the Bank: (1) I would find it more valuable and useful if the institution were to out-source_ 14
_more of its research; (2) I would find it more valuable and useful if the institution committed more resources to research within an in-house research department.”_ Over the whole sample, 63.0% supported option (2), for more resources for in-house research. The VPU with the lowest support was ECA, with 56.6% support, and, the highest was WBI at 85.7%. However, excluding WBI, the inter-VPU differences are not statistically significant at the 5% level. Across sectors, the lowest support for in-house research is again EM, with only 40.0% supportive, while the highest was the EP sector, for which 82.2% of respondents preferred option (2), followed closely by POV (81.5%).

Those who are more familiar with and rely more on WB research tend to be more supportive of higher resources being devoted to in-house research. With the same set of controls, the correlation coefficient between support for the in-house research department and current reliance on research is 0.40. A similar pattern is found for familiarity with research, though the correlation coefficient was lower at r=0.23. An even stronger correlation is found between support for in-house research and expected future reliance on WB research (r=0.49).

Table 9 gives the Probit regression coefficients for the probability of supporting the inhouse research department (i.e., picking (2) above). Controlling for years, location, VPU and sector, support for greater resources for the research department is greater among those who rate highly the technical quality and policy relevance of WB research, those who think it helps with AAA. It also tends to be greater for those who have a less favorable view of external consultants.

The direct responsiveness of the research department to operational demands is a potentially important channel for knowledge diffusion in an organization such as the World Bank, but how well does it work? The survey asked two questions about the experiences of staff in working with DEC, namely: ― _To what extent are researchers at DEC responsive to your needs?”_ and _“To what extent are DEC researchers responsive and available for cross support?”_ Again, a 10-point scale was used from 1=― _Not at all_ ‖ to 10=― _To a very high degree_ .‖ For the first question there was also an option for _“Depends on the researcher; some are responsive, some are not.”_ Since a minimum amount of (paid) cross support is mandatory for DECRG researchers, the degree of ―responsiveness‖ will undoubtedly depend in part on whether the researcher in question is above or below the minimum (set at 13 weeks per year). So there is likely to be some random variability depending on when the researchers are approached. Figure 15
4 gives the density functions for scores 1-10. Again we see signs of bi-modality, with a larger sub-group of staff centered around an answer of 7-8 and a smaller mass centered around 2-3. The mean and median responses to the first question on a 10-point scale were 5.80 and 6 respectively with n=241 (there is a large number of missing values for both these questions). 12.9% of respondents gave the answer _“Depends on the researcher.”_ The mean and median for the second were 5.47 and 6 respectively (n=254).

Tables 10 and 11 give the breakdown of these indicators across VPUs and sectors. The proportion rating DEC researchers as 5+ on their responsiveness varies from 50% in OPC to 100% in WBI. The proportion with 5+ for availability for cross-support varies from 36% in South Asia to 79% in ECA. The differences across sectors in assessments of DEC responsiveness are particularly striking, with only 20% for staff in SDV rating responsiveness as high, compared to 90% of staff in POV, with EP close behind at 86%.

The survey asked about other factors relevant to demand for research, notably its relevance, accessibility and timeliness. (Each category was given an explanation in the survey instrument.<sup>24</sup> ) And again a 10 point scale was used. Figure 5 gives the densities. A marked bimodality is evident for ―relevance‖ and (less so) ―accessibility‖ and the answers to these questions were highly correlated (r=0.74). ―Timeliness‖ is distributed somewhat differently though still correlated with the other measures (r=0.72 for both relevance and access). The mean score on relevance was 5.80 (n=445), 6.12 (n=449) on access and 5.25 (n=383) for timeliness, which appears to be the more important constraint on demand for research. Research takes time, and often requires a longer time horizon than development projects allow.

# **5. The value of research in operational work**

The incentive to learn about the Bank’s research is greatest when it is perceived to be of high value to operational staff in their work. The survey asked: _“please rate the overall value of World Bank research for your work, on a scale of 1-10 where 1 means not valuable at all, 10 means extremely valuable.”_ And again respondents were presented with a 10 point scale where 1=‖not valuable at all‖ and 10=‖extremely valuable.‖ Figure 6 gives the densities of responses to

> 24 ―Relevance‖ was qualified by ―not too theoretical;‖ ―accessible‖ was explained as ―well written and easy to use‖ and ―timeliness‖ was qualified as ―the research was delivered in the necessary time frame to allow you to integrate into AAA and/or lending operation.‖

16
the ―value‖ question. We see a marked bi-modality, with a high value mode around 7-8 and a low value mode around 3. The mean score is 5.61 and the median is 6. The density function in Figure 6 suggests that a score of 5 is reasonable as the lower bound in defining the ―high-value‖ group. We find that 65.5% of respondents attached high value to WB research.

Sample selection bias is inflating these numbers, though the effect appears to be modest when one corrects for the variation in response rates across VPUs. This adjustment brings the mean assessed value down to 5.54 from 5.61. While this is small, there may well be a larger bias stemming from latent differences between staff within a given VPU.

There are marked differences in the value attached to research across VPUs and sectors. Tables 12 and 13 give the summary statistics. The VPU that attaches the lowest average value to WB research is SDN, with a mean score of 5, and 55% of respondents giving a rating of 5 or more. The VPU with the highest value attached to WB research is WBI, with an average score of 7, and all the WBI respondents rated WB research highly. The next highest is PRM, with a mean score of 6.5. The sectoral VPUs generally put higher value on research than the regional VPUs. The regional VPU with the highest mean score is (again) MNA.

The traditional ―hard infrastructure‖ sectors tend to put lower average value on research for their work, suggesting lower incentives to learn about WB research. The mean score for ARD, EM, TRN and URB is 4.82 (0.41) as compared to 6.27 (0.27) for EP, EDU, HNP, POV and SP; the difference is significant (t=2.94). The sector for which staff attach the lowest value to WB research for their work is EM (a mean score of 4.1, with only 41% rating it highly), followed by TRN, URB and ARD. The sector with the highest mean value is the poverty sector within PRM, with a mean score of 7 and for which 93% of respondents rated WB research highly in terms of its value for their work.

How then are staff distributed across the four categories identified in the introduction? The following table gives the breakdown for the 519 responses with complete information.<sup>25</sup> We see that amongst those respondents who feel that WB research is important for their work roughly two-thirds have ―high‖ familiarity with WB research products. The independently wellinformed are a relatively small group. Amongst those with low perceived familiarity with

> 25 Recall that 5+ defines ―high‖ for the ―value‖ variable, while it is 6+ for ―familiarity.‖

17
research slightly less than half put low value on research for their work, while the rest (one quarter of all respondents) think research is valuable, and are clearly are not getting enough of it.

|||Perceived value of WB researc<br>Low|h for the staff member’s work<br>High|
|---|---|---|---|
|||**“Happily uninformed:”**<br>N=117 (22.54%)|**“Frustrated uninformed:”**<br>N=123 (23.70%)|
|Staff member’s<br>personal|Low|Mean familiarity=3.35<br>Mean value=2.72|Mean familiarity=4.00<br>Mean value=6.35|
|familiarity with<br>WB research|High|**“Independently well-informed:”**<br>N=62 (11.95%)|**“Functionally well-informed:”**<br>N=217 (41.81%)|
|||Mean familiarity=7.26<br>Meanvalue=2.90|Mean familiarity=7.58<br>Meanvalue=7.57|

The ―frustrated uninformed‖ can be thought of as a target group for extra effort at research dissemination. So it is of interest to see where they are found. Tables 14 and 15 give the VPU and sectoral distributions of the four types of staff. WBI has (by far) the highest share of the ―frustrated uninformed‖ staff, with 65% in this category, the rest being ―functionally wellinformed‖—both valuing research for their work and familiar with it, who comprise 42% of respondents.<sup>26</sup> The MNA region has the lowest, as 13%. Breaking it down by sectors, it is transport staff that have the highest share of ―frustrated uninformed‖ group, at 35%, with URB and SDV close behind. EDU has the lowest, at 9%. The share of ―functionally well-informed‖ varies from 35% in SDN to 63% in PRM, while amongst sectors it varies from 14% in energy and mining to 83% in the poverty sector.

A closer look at the ―happily uninformed‖ does not suggest that they have simply switched their favors to external (non-WB) sources of research. They give a relatively low rating to academia as a source—a mean score of 5.53, as compared to 6.87 for all other staff, and the difference is significant at the 0.1% level. They also rate consultants lower than average, at a mean of 6.07 as compared to 6.54 for the rest, and the difference is significant at the 5% level. At the same time they have an above average desire to increase their use of the Bank’s research; their mean difference between expected future usage and past usage (both on the 10-point scale) is 0.80 versus 0.54 for the rest of the staff and 0.43 for the ―functionally well-informed.‖ (The

> 26 This number depends (of course) on the precise cut off point used in defining ―high‖ and ―low.‖ For example, if instead one raises the lower bound for defining the ―high-value‖ group by one step of the 10 point ladder then the % of the sample are deemed to be ―functionally well-informed‖ falls from 42% to 38%.

18
former difference is only significant at the 13% level, but the latter makes it at 3%.) Evidently the ―happily uninformed‖ are not entirely happy with the status quo.

# **6. Does an incentive to learn translate into knowledge?**

Attaching a high value to research for one’s work need not translate into actual knowledge about research. That link will be broken if the Bank’s internal research organization is unresponsive, or its products inaccessible.

However, the evidence suggests that stronger incentives for learning do generally translate into higher investment by staff in learning, as reflected in the familiarity with research. Those respondents who put a higher value on research for their work are more likely to be familiar with it, as can be seen in Figure 7 (with the same set of controls used earlier). The (nonparametric) regression line shows a persistent increase over the whole range of familiarity scores. If we focus solely on those respondents who believe they have a high familiarity with WB research—again a score of 6 or more—then the mean assessed value is 6.53 (s.e.=0.14), as compared to 4.49 (0.14) for those who acknowledge they are not very familiar with research; the difference is highly significant (t=10.22). Amongst those familiar with research, 69.2% rated it of high overall value (s.e.=2.8%), compared to 35.4% (3.1%) for those relatively unfamiliar.

How much does familiarity with research respond to stronger incentives to learn? The regression coefficient of the familiarity score on the value score is 0.45 (with a standard error of 0.036). Going from a 1 to a 10 on perceived value of research for one’s work translates into a mean increment of four levels for familiarity. Table 16 gives regressions for familiarity scores with controls. The first specification uses only the controls for years of experience, its squared value, location, sector and VPU, while the second includes the assessed value of research for the respondent’s work, as the measure of incentives for learning. (I return to the third.) Allowing for differing incentives for learning more than doubles the share of the variance in familiarity with research that is explained by the controls alone. With the controls, the regression coefficient on the incentive variable falls slightly, to 0.421. A one standard deviation increase in the value attached to research adds about one half of a standard deviation to the 19
familiarity score.<sup>27</sup> About half of the gap between the mean familiarity scores for ARD, EM, TRN and URB (on the one hand) and EP, EDU, HNP, POV and SP (on the other) is accounted for by the difference in the value attached to research by staff working in these sectors.<sup>28</sup> In interpreting these findings, one can view the glass as half empty or half full. The fact that knowledge responds to incentives for learning suggests that the internal system for research and its diffusion is working to create internal absorptive capacity. But the fact that the regression coefficient is significantly less than unity is also suggestive of frictions in the process. (Figure 7 also shows the 45 degree line.) It may well be that the responses to the ―value‖ question are picking up a latent characteristic of respondents whereby some operational staff are just naturally ―research consumers‖—they are more interested in research and seek it out more avidly even when their incentive for learning (as part of their job) is low. This would cast doubt on the interpretation of the ―value‖ question as indicating the respondent’s incentive for learning about Bank research. The survey also asked respondents to rate academia as a source of research relevant to their work. This is likely to pick up the ―research consumer‖ attribute. So too is the respondent’s rating of ―informal discussions with researchers‖ as a source of knowledge about research. Column (3) of Table 16 gives the regression with these two extra controls. Both are significant, and the coefficient on value falls to 0.30, but it is still significantly positive and significantly less than unity, again suggestive of frictions.

Separately to these effects, familiarity with WB research also varies with years of service and this effect is markedly nonlinear. Familiarity rises with years of service up to 16 years (18 years using the specification in Column 3), and falls after that. The median years of service is 10; 20% of respondents had more than 16 years of service.

There is also a difference in familiarity with WB research between those based at the Bank’s head-quarters in Washington DC (where the research department is located) and those in the field offices in developing countries; 65% of respondents were in the former group, versus

> 27 I tested for nonlinearity in this relationship by adding a squared term in ―value‖ but the coefficient was not significantly different form zero.

> 28 Recall that the difference in mean familiarity scores is 1.29. Given that the difference in mean value attached to research for their work is 1.43, this implies a difference in mean familiarity with research of 0.60.

20
34% in field offices (the remaining 1% were in WB offices in developed countries, mainly Europe). The fact that familiarity is lower for those in country offices suggests that the advances in communication technology—the WB has a well-developed ―intranet‖ to facilitate this—have not eliminated the advantages of physical proximity.

It is of interest to look for other evidence for or against the view that stronger incentives for learning translate into knowledge. An objective clue to how much the Bank’s internal system for knowledge management responds to differences in the incentive for learning can be found in how the demand for paid cross-support from the Bank’s research department varies with answers to the value question. Across VPUs, the mean score for the value of WB research is highly correlated with the per capita demand for cross-support from DECRG (using the data in Tables 1 and 12); the correlation coefficients between the three-year mean of cross-support per staff member GG+ and the mean value score is 0.72, and it is also 0.72 with the proportion of VPU staff giving a value core over 5. This is consistent with the view that incentives for learning in the Bank do generate a response.

# **7. Conclusions**

One can think of two stylized models of how research is used by development practitioners. In the first, they have a demand for knowledge that does not stem from its direct bearing on their work. Much development research is a public good. Practitioners might read research findings to better understand the world in which they work, even when that understanding is essentially irrelevant to the specifics of that work. Alternatively, in the second model, research has a direct value in the work of practitioners—such as by informing project choices at the entry stage and assessing impacts later on—and research findings are sufficiently relevant and accessible to assure that practitioners become well informed. To enhance the quality of development aid, we hope that the second model is the best characterization. But is it?

If it was true in general that research findings are largely consumed independently of their direct bearing on the work of development practitioners then we would not expect greater familiarity with research findings to be associated with a higher perceived value of research for the practitioners’ work, or higher stated reliance on research for that work. Indeed, given that 21
there are heterogeneous costs of learning, those most familiar with research findings need not overlap much at all with those who value research most for their work.

But that is not what we see in these survey data. Two-thirds of the responding senior staff put a high value on the Bank’s research for their work. They appear to have a reasonably strong incentive to learn, though stronger for some than others. Operational staff with a stronger incentive to learn tend to be more familiar with Bank research. Only 12% of staff fall into the category of ―independently well-informed‖—those who report high familiarity with research, but attach low value to it for their work. The Bank’s senior staff are mainly found in three other groups. The first are those who are ―happily uninformed‖ about research—they attach low value to it for their own work, and report correspondingly low familiarity with research; 23% of senior staff fall into this group. A second group puts a high value on Bank research but its members are not generally successful in accessing that research or find it of limited relevance. This group is not evenly spread across Bank units and sectors, suggesting that there may be scope for a targeted effort at research and its dissemination. The third and largest group is the ―functionally well-informed,‖ comprising 42% of staff. They put a high value on research for their work (a mean score of almost 8 on a scale of 1-10) and are (by their own assessment) quite familiar with research findings (also a mean score of 8 on a scale of 1-10).

While this paper’s results are consistent with the view that stronger incentives for leaning about research translate into greater knowledge, they are also suggestive of some significant frictions. The slope of the relationship between perceived value and familiarity with research— the effect of an increment to perceived value on familiarity, both on a 1-10 scale—is positive but significantly less than unity, suggesting frictions in how the incentive for learning translates into knowledge. The responsiveness of researchers and the timeliness and accessibility of their outputs are clearly important to how much learning incentives lead to useful knowledge.

There are some marked differences across Bank units. Staff working in the economic policy and human development sectors tend to be both more familiar with the Bank’s research and attach higher value to it than do others, notably in the more traditional areas of the Bank’s lending, such as agriculture, energy and mining, transport and urban development, which account for just under half of Bank lending. For example, while only 7% of senior staff members working on poverty and 10% of those mapped to economic policy are ―happily uninformed,‖ the 22
proportion rises to 37% for staff working on urban development and 41% of staff in the energy and mining sector.

The differences across units in the demand for the Bank’s research are correlated with the incidence of PhDs and economists, suggesting that internal research capacity in operational units helps create absorptive capacity for knowledge in those units. The Bank’s more formal networks also play a role in the research diffusion process, especially in the economic policy, poverty and human development sectors; however, staff working in the traditional sectors tend to give lower ratings to their networks in their performance in helping them connect to research.

What gives rise to these inter-sectoral differences? Both demand- and supply-side factors have clearly played a role. Staff working in the traditional sectors may well have turned away from Bank research in part because they have found it to be of little relevance to their needs, though they also put relatively low value on research sources outside the Bank. Today’s research priorities may well be poorly matched with the issues faced by practitioners in these sectors. For example, the current emphasis on randomized trials in development economics has arguably distorted knowledge even further away from the hard infrastructure sectors where these tools have less applicability (Ravallion, 2009). Making the supply of research more relevant to the needs of development practitioners would undoubtedly help.

However, this is an incomplete explanation, since the supply of research is clearly also determined by demand. In turn, demand stems in no small measure from the extent to which ―development impact‖ is challenged by donors. Impact often appears to be taken for granted in the traditional hard infrastructure sectors (though in truth the evidence is often rather weak, given relatively low levels of investment in ex-post evaluations). This stands in marked contrast to the social sectors where lending and policy operations have had to work hard to justify themselves, and have drawn more heavily on research to do that; the large body of research on poverty and human development in developing countries that emerged in the last 15 years is indicative of this longer-term shift in priorities. Clearly, if the presumption of ―impact‖ is routinely challenged by donors, aid organizations and citizens then project staff will face strong incentives for learning about impact. And (as this paper has shown) strong incentives for learning yield greater familiarity and use of research.

23