metadata: authors: 0: author: Das, Jishnu
1: author: Do, Quy-Toan
2: author: Shaines, Karen
3: author: Srinivasan, Sowmya
count: World
docna: 0: docna: U.S. and them : the geography of academic research
docty: Policy Research Working Paper
theme: Other economic management
majtheme: Economic management
subtopic: Social Analysis,Human Rights,Quality of Life & Leisure,National Governance,Government Policies,Health Care Services Industry,Disability,Youth and Governance,Inequality
teratopic: Agriculture,Macroeconomics and Economic Growth,Information and Communication Technologies,Social Protections and Labor,Education
abstracts: cdata!: Using a database of 76,046 empirical
            economics papers published between 1985 and 2004 in the top
            202 economics journals, the authors report two associations.
            First, per-capita research output on a given country
            increases with the country's per capita gross domestic
            product (GDP). Regressions controlling for data availability
            and quality in the country, indicators of governance and the
            use of English yield an estimated research-GDP elasticity of
            0.37; surprisingly, the United States (US) is not an outlier
            in the production of empirical research. Second, papers
            written about the US are far more likely to be published in
            the top five economics journals, even after the quality of
            research has been partially controlled for through
            fixed-effects for the authors' institutional
            affiliations; the estimates suggest that papers on the US
            are 2.6 percentage points more likely to be published in the
            top-five journals. This is a large effect because only 1.5
            percent of all papers written about countries other than the
            US are published in the top-five journals. The authors
            speculate about the interpretations of these facts, and
            invite further analysis and additions to the public release
            of the database that accompanies this paper.
repnme: repnme: U.S. and them : the geography of academic research
display_title: U.S. and them : the geography of
            academic research
listing_relative_url: /research/2009/12/11490692/geography-academic-research
topicv3: Political Economy of Reform m1326263 836,Poverty m1326687 695,Health m1504661 3480,Discrimination m1505707 3379,Blindness m1582294 4218
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Agriculture,Macroeconomics and Economic Growth,Information and Communication Technologies,Social Protections and Labor,Education
no_of_pages: 38
model_extractions: WPS5152 Policy Research Working Paper 5152 U.S. and Them The Geography of Academic Research _Jishnu Das Quy-Toan Do Karen Shaines Sowmya Srinivasan_ The World Bank Development Research Group Poverty and Inequality Team & Human Development and Public Services Team December 2009
Policy Research Working Paper 5152

# **Abstract**

Using a database of 76,046 empirical economics papers published between 1985 and 2004 in the top 202 economics journals, the authors report two associations. First, per-capita research output on a given country increases with the country’s per capita gross domestic product (GDP). Regressions controlling for data availability and quality in the country, indicators of governance and the use of English yield an estimated research-GDP elasticity of 0.37; surprisingly, the United States (US) is not an outlier in the production of empirical research. Second, papers written about the US are far more likely to be published in the top five economics journals, even after the quality of research has been partially controlled for through fixed-effects for the authors’ institutional affiliations; the estimates suggest that papers on the US are 2.6 percentage points more likely to be published in the top-five journals. This is a large effect because only 1.5 percent of all papers written about countries other than the US are published in the top-five journals. The authors speculate about the interpretations of these facts, and invite further analysis and additions to the public release of the database that accompanies this paper.

This paper—a product of the Poverty and Inequality Team, and the Human Development and Public Services Team, Development Research Group—is part of a larger effort in the department to analyze the geography of academic research. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at research@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# U.S. and Them: The Geography of Academic Research<sup>**1**</sup>

Jishnu Das and Quy-Toan Do, with Karen Shaines and Sowmya Srinivasan

> 1 Das: World Bank and Center for Policy Research, New Delhi; Do: World Bank; Shaines: World Bank and IMF; Srinivasan: World Bank. We thank Liz Braunstein, Zurab Sajaia, Linda Venable and Tatiana Goriainova for assistance in the compilation of the dataset. We are grateful to Abhijit Banerjee, David McKenzie, Rohini Pande, Lant Pritchett, Carolina Sanchez-Paramo, Biju Rao, Tara Vishwanath and Adam Wagstaff for helpful discussions. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the World Bank or the International Monetary Fund, their affiliated organizations or those of their Executive Directors or the governments they represent. Contact information: Jishnu Das: jdas1@worldbank.org, Quy-Toan Do: qdo@worldbank.org. The dataset and supporting documents are available for download at http://econ.worldbank.org/staff/qdo
# **1. Introduction**

Among academic and policy researchers working on countries other than the US, there is a belief that there is too little work on countries outside the US, particularly the poorer ones (see for instance, Bardhan 2003). There is a also a belief that papers written about countries other than the US receive less favorable treatment in the top journals. Some referee reports are indeed memorable for their stance on the relevance of research on low-income countries. For instance, one researcher in a top university shares his/her referee report from a top journal: “ _There are a number of reasons to be concerned about the potential generalizability and policy relevance of the findings in this paper (...) It is hard to generalize away from this very specific context to speak to the policy debate in developed countries._ ” (Emphasis added) In the case of another referee report, _“The editors have examined your paper and have concluded that it is not appropriate for the (...) journal. [M]ost members of the (...) journal’s audience will consider your study to be on a very special population. I think your paper as it stands has considerable value and interest to a specialized audience, even if it does not attain the generality we require here.”_ Researchers working on countries other than the US have become so used to such reports and comments like these do not even strike them as out of the ordinary any longer. Imagine instead a referee report that says: _“Both referees really enjoyed your paper. The question is important and the empirical results are convincing. However, it is hard to generalize from the specific context of the US to speak to the policy debate in China or India.”_ These anecdotes are, at first glance, consistent with the surprisingly low number of empirical economics publications on countries other than the US in the top 5 economics journals.<sup>2</sup> The _American Economic Review_ publishes one paper on India (on average) every 2 years and one paper on Thailand every 20 years. Neither are these numbers particular to this prestigious journal. Over a 20year span dating from 1985 to 2004, the top 5 economics journals together published 39 papers on India, 65 papers on China, and 34 papers on _all_ of Sub-Saharan Africa. In contrast, they published

> 2 By top 5 economics journals, we henceforth refer to the _American Economic Review_ , _Econometrica_ , the _Journal of Political Economy_ , the _Quarterly Journal of Economics_ and the _Review of Economic Studies_ .

2
2,383 papers on the United States. Do these numbers reflect differences in the overall volume of research on different countries? What is the extent of total research on say Kenya relative to the US? What are the determinants of research output on a given country?

To explore these issues, we document some basic facts about the geography of academic research in empirical economics. We build on the description of Ellison (2000) and provide a comprehensive set of tabulations and associations regarding the geography of academic research. To our knowledge, this is the first paper to do so. Our description is based on an article-level database of all empirical economics publications associated with a specific geographical identity, hence excluding theoretical and cross-country studies. More specifically, we record publications from the top 202 journals over the time period 1985-2004.<sup>3</sup> Each entry comes with information on authors’ affiliations, the journal that the article was published in, the year of publication and the country of study. Combining this dataset with basic country information yields several interesting associations. Out of a total of 76,046 country-specific empirical articles published over the 20-year period from 1985 to 2004, 36,649 papers were produced on the US. Remarkably, the US is not an “outlier”. It is on the regression line relating (log) per-capita publications to (log) per-capita Gross Domestic Product (GDP); excluding the US from the regression does not alter the coefficient on GDP per-capita; 75 percent of the cross-country variation in (per capita) publications is accounted for by this single variable. Because a country like the U.S. is rich _with_ a large population, it reports far more publications than other countries with similar per-capita incomes. Put another way, publications per-capita are very similar in the U.S. to other countries at similar levels of wealth. The role of data in explaining this relationship is assessed by explicitly controlling for measures of country-level measures of data availability and quality and by looking at the patterns of research output following the release of major household surveys. At first glance, the lack of data does not seem to be the main impediment.

Looking at publication outlets, 1.5 percent of all papers written about non-US countries are published in the top-5 economics journals compared to 6.5 percent of all papers written about the US. Controlling for authors’ institutional affiliation, and hence partially accounting for research

> 3 The top 202 journals are all the journals that appear on one of the rankings provided by Kalaitzidakis et al. (2003) and Kordrzycki and Yu (2006).

3
quality, the difference is lower but still a statistically and qualitatively significant 2.6 percentage points.

These results provide an empirical basis for a potential debate on the extent of economic research in different countries. To the extent that optimal economic policy depends on local institutions, culture and geography, country-specific research is important. There is some indication that such a debate may indeed already be under way. The Growth Commission, mandated to understand the sources of growth and appropriate policy, argues in its final report that growth strategies are almost certainly country-specific and perhaps time-specific as well. One introductory passage is worth quoting in full: “ _Wedded to the goal of high growth, governments should be pragmatic in their pursuit of it. Orthodoxies apply only so far. This report is the product of two years of inquiry and debate, led by experienced policy makers, business people and two Nobel prize-winning academics, who heard from leading authorities on everything from macroeconomic policy to urbanization. If there were just one valid growth doctrine, we are confident that we would have found it.”_ —Growth Commission Report, Introduction, Page 4 Paraphrasing the main finding of the report, it is difficult, for instance, to argue that we know how to accelerate growth in Burundi based on the 5 papers that have been produced on that country between 1985 and 2004. While country-specific reports such as the World Bank’s Country Economic Memoranda, the IMF”s Country Reports or UNDP’s National Human Development Reports form an important part of the knowledge base on which to draw from, are we satisfied with leaving research on developing country policies entirely to discussions that are not part of a formal referee process? Alternatively, are we confident enough that research on US issues is relevant to the policy debate in Burundi?

The results could also help spur discussion about the role and responsibilities of the top journals in the field, which are all US- and UK-based in furthering research on countries other than the US (and to a lesser extent, as we will see, on the UK for the UK-based journals). If for instance, researchers base their choice of research countries on the likelihood of publications in top journals, bias in these outlets could inefficiently allocate greater research effort towards the US. Alternatively, if researchers 4
who continue to work on low-income countries are not rewarded in the publication (and hence tenure) process, there may be efficiency losses in terms of the networks that they have access to.

At this stage, our findings consist of tabulations and correlations. Like all work on discrimination, identifying differential treatment in any form is difficult in the absence of experimental studies.<sup>4</sup> Furthermore, with sparse information on the availability and quality of datasets from different countries, it is ultimately difficult to rule out the influence of data on the quantity and quality of publications across countries. The findings presented in the paper thus aim only to book-end a debate on why there is more research on some countries than others and on the publication process in the economics discipline, with a focus on empirical studies on and outside the US.

The remainder of the paper is as follows. Section 2 is the description of the data. Section 3 documents and discusses the two main findings of the paper. Section 4 concludes.

# **2. Data Description**

The main data source is constructed using information on journal articles published in selected 202 economics journals during the period 1985-2004 that were associated with a geographical identifier (thus excluding all theoretical contributions and cross-country empirical work). We used journal rankings proposed by Kalaitzidakis and others (2003) and Kordrzycki and Yu (2006) to finalize the list of journals for inclusion in the database. Ultimately, we selected the 202 economics journals that appeared at least in one of their proposed rankings. Table A1 provides the list of these journals and their rankings according to various citation indices. The large number of journals was chosen partly to ensure that country-specific publications in the dataset reflected the volume of research on the country rather than journal selectivity; of note is that the citation index for the bottom ranked 11 journals is 0, and close to 75 journals have a citation rank less than 1 (that is, the average article in the journal is cited less than once in subsequent research).<sup>5</sup>

> 4 Previous work examines whether top economics journals are biased in their publication rates towards authors in the editor’s networks as well as how the introduction of double-blind review changed the publishing process (Laband and Piette, 1994, Blank 1991). 5 Kalaitzidakis and others (2003) construct for each journal a citation rank based on citations in 1998 of articles published only in 1994-1998, excluding self-citations and adjusted for impact (influence) and size. Kordrzycki and Yu (2006) provide citations and reference-intensity-adjusted rankings that evaluate a specified set of journals according to influence of journals and influence of journal articles. These rankings take into account citations in economics academic

5
To obtain the files of article records we used the Econlit database provided by EBSCOHost to conduct a field search for each individual journal title, limited to the years 1985 through 2004.<sup>6</sup> If a journal started publication after 1985 we started with the earliest possible date. Every Econlit record is assigned metadata separated into fields. We kept data from the following fields: Author; Author Affiliation; Journal Name; Journal Issue; Descriptor Classification Codes (JEL codes); and Geographical Descriptors. We manually cleaned up the Author-Affiliation field and identified the 100 first academic institutions in addition to three multilateral organizations (IMF, UN and World Bank).<sup>7,8</sup> Geographic Descriptors refer to either a specific country, or a generic group of countries.<sup>9</sup> An article was thus assigned to a specific country if the Geographic Descriptor made an explicit mention of the country and it was labeled as a “cross-country study” otherwise. Papers with no associated geographic descriptors were identified and classified in the category of “theoretical contributions”. The data were then aggregated up to create a country*year dataset of total publication counts broken down by three types of journals (Top-5, Top-10 journals, all journals) and author affiliations.<sup>10</sup> journals as well as citations in other social science and policy journals. In addition, we use the eigen-factor ranking produced as part of a research project at University of Washington. The eigen factor is associated with a specified set of journals and is a measure of the overall value provided by all the articles published in a given journal in a year. The Article Influence is a measure of a journal’s influence based on the number of citations per article. Thus, according to the Article Influence ranking, one publication in the _American Economic Review_ will count for 4.9 publications, while one publication in the _Journal of Development Economics_ will count for 1.4 publications.

6 Due to the unavailability of data on some governance indicators and growth variables for the years 2005 and onwards, we restrict all the analysis in this paper for the years 1985-2004.

7 We take the 100 first institutions ranked by the number of pages published provided by Kalaitzidakis et al. (2003). These institutions produced a third of the total number of publications over the period 1985-2004. Affiliations we did not uniquely identify were coded “Other”.

8 The codes used to identify the institutional affiliations are open access (http://econ.worldbank.org/staff/qdo/), and we welcome additions to the list of institutions already identified. Above all, we encourage Econlit to code authors, their affiliations, geographic descriptors and other paper attributes in a more standardized fashion.

9 Articles with broadly defined geographical identifiers such as “Selected Countries” or “Europe” were difficult to link to a specific geographical entity. An Econlit representative pointed out that such identifiers usually represent research arising from cross-country empirical techniques. For instance, a continent identifier such as “Europe” would be used for research on a general topic (such as climate change) across a large number of countries in the continent. It is unclear how to deal with cross-country empirical work, and such papers are excluded from this analysis entirely.

10 One limitation of our approach is that important research outlets may still be omitted from this database. Many papers on India, for instance, are published in the _Economic and Political Weekly_ , which does not appear here; neither is research that only appears in reports or books incorporated in this analysis. This database also ignores policy reports and other country specific analyses that are not submitted through the formal academic refereeing system, but with potentially important policy impact such as World Bank’s Country Economic Memorandums, IMF Country Reports, or the United Nations Development Program’s National Human Development Reports.

6
The publications dataset was then merged with data on standard growth variables, governance indices and release of survey data to arrive at a panel dataset and a country-level dataset on 175 countries for the period 1985 to 2004. Table A2 in the Appendix provides a detailed description of variables and their sources. Summary statistics for the data used in the paper are presented in Table 1.

To assess the volume of economics research covered in this paper relative to total economics research, Figure 1 presents a characterization of the nature of research over the 20-year span of our data. We split the set of articles into three groups: theoretical contributions, country-specific case studies, and cross-country analyses. During the 20-year period, the number of publications in all 3 groups increased. Nevertheless, there has been a very small reallocation of research across these 3 groups. There is an increase in the share of cross-country empirical work at the expense of the other two, but theory (40 percent) and single-country empirical papers (50 percent) continue to account for 90 percent of all publications in the field. Therefore, the papers examined here represent the vast majority of empirical work in the field, and almost half the output of the discipline as a whole.

# **3. Two Facts on the Geography of Academic Research**

## _3.1._ **Stylized Fact 1: The Research-Wealth Relationship**

Three figures and a table present the basic facts about empirical economics publications. Figures 2 and 3 and Table 2 use the total number of publications over the period 1985-2004 and relate it to average GDP of countries. Here, we disregard time variation in the data, and take 20-year averages of all variables. Figure 4 returns to the panel aspect of the data and looks at the effect of data dissemination on publications.

Figure 2 plots publications aggregated over the 20 years for all countries in the database and by geographical region, as classified by the World Bank (Table A3 in the appendix summarizes the number of publications for every country in our database, and also indicates the number of papers that were published in one of the top 5 economics journal). Countries in different geographical regions are shown in the seven columns—one each for every geographical region. For every region, 7
countries are aligned in descending order of the number of publications over the 20-year period. Taken as a whole, there is a mass of countries with very few publications; a thin layer above this group, which includes China, Canada, Japan and; above these is the United Kingdom with 6,567 publications over the 20 years. Standing out from the group is the United States with 36,649 publications over these 20 years, accounting for the geographical focus in 48 percent of all the economic empirical research during this time.

Given U.S dominance, publications by other countries seem insignificant; nevertheless within region comparisons are informative. In Latin America, Mexico reports the highest publications followed by Brazil and Argentina with countries like Venezuela and Bolivia at the very bottom. In Sub-Saharan Africa (SSA), a lot more research is produced on South-Africa than the others, but there is some work on Ghana, Kenya and Nigeria. There are almost no papers published on countries like Togo, Benin, The Central African Republic and Somalia over this period. In South Asia (SA) and the East Asia and Pacific (EAP) region, China and India report more publications than all other countries, but twice as much research is produced on China as on India.

For those particularly interested in China-India comparisons, the last finding is puzzling since India has always _produced_ good economists, now spread over various U.S. institutions, who could arguably contribute to research on the country. Three potential explanations for this puzzle stem from the weakness of this database as an indicator of knowledge production in _all_ economics sub-disciplines. First, the database does not include theoretical contributions and many Indian-origin economists work on theoretical questions. Second, it could be that while ethnic Chinese researchers generally seem to focus almost totally on China, the same is not true of ethnic Indian researchers. And third, the database does not include research published in journals like the _Economic and Political Weekly_ —a leading outlet for applied empirical work in the country. Despite the caveats, it is worth emphasizing that these two weaknesses should not be overused. There could be a real dearth of applied empirical research in India; particularly since the extent of research was quantitatively similar in India and China in 1985, but has steadily increased in China for the last 20 years while it has stagnated in India.

Finally, it is also noteworthy that the total numbers of papers produced are very different across countries at similar levels of income. Much more is produced, for instance, on India than on Mexico.

8
This suggests that for the _total_ publications both GDP per-capita and population matter since India is poorer but significantly larger than Mexico. Deflating the number of publications by country population could yield a clearer relationship between research and wealth, to which we now turn.

Figure 3 plots the log of total publications against the log of GDP averaged over the 20-year period for all countries (left), and separated by countries in Latin America, Sub-Saharan African and the Middle East and North Africa (right).<sup>11</sup> As is clear from the figure, the richer the country, the more research there is on it. The relationship between the logarithm of the number of publications and logarithm of GDP is roughly linear with a slope close to 1. There is a hint that the US is above this line, but the deviation does not stand out, unlike in Figure 2. Distinguishing countries by region suggests that the research-wealth relationship within geographical regions is similar to that across the world. In most regions—with the exception of the Middle East and North Africa—countries are evenly spread out on the worldwide fitted line relating research to wealth. Countries in the MENA region stand out for their significant downward deviation. _All_ countries in the MENA region report significantly less research than what is expected for their levels of income and within the region, there appears to be little relationship between the research produced on a country and its wealth, measured by GDP.

Table 2 presents the regression (OLS) analog of this figure. Publication measures are totals for the period 1985-2004, whereas GDP is averaged over the same period for the 175 countries. We estimate the following equation: _Log(Publications)i = a + b.log(GDP per capita)i + c.log(Population)i + ei_ (1) The description illustrated in Figure 3 is confirmed from the results in Columns 1 and 2. Publications and GDP are positively correlated. The coefficient of log(GDP per capita) is 0.62, and it increases to 0.71 once region fixed-effects are included. There are no scale economies arising from the country’s population; a hypothesis confirmed with the formal χ<sup>2</sup> -tests (χ<sup>2</sup> = 1.215, P-value =

> 11 We plot (log) Publications against (log) GDP. This is identical to deflating both variables by population and assuming that research is scale invariant—an assumption explicitly tested in the next section.

9
0.55). Also remarkable is the high explanatory power of this simple specification—GDP per capita and population alone account for 74.2 percent of the variation in the data.

In Columns 3-6, we introduce a number of control variables to further understand the relationship between research and wealth. Column 3 introduces a press freedom index as a measure of governance (we also tried other indices such as an index of democracy and a political rights index; the high correlations among these indices imply that the results remain the same); Column 4 introduces a variable for the availability and quality of data which takes the average of the countries scores on data collection, data availability and data/statistical practices compiled by The World Bank; Column 5 introduces a dummy variable for whether the country has a Muslim majority, and Columns 6 and 7 introduce respectively a dummy variable for whether English is an official language and the enrollment in tertiary education as a proxy for country research capacity.

The estimated coefficients of wealth and population increase somewhat with the inclusion of regional fixed-effects, but drop when either our measure of governance (press freedom) or data availability/quality are included. Although the “Muslim” dummy and the use of English as an official language are important predictors, they do not mediate the research-wealth relationship of Column 1. Including all the variables together (Column 8) leads to two further differences. First, the “Muslim” dummy reverses signs and is no longer significant, suggesting the low research on Muslim countries is largely due to the correlation between being a Muslim country and poor governance levels or poor data availability. Second, there is a further drop in the GDP coefficient to 0.37, suggesting that the different explanatory variables (in particular, governance and data) have independent effects on the relationship between research and GDP. The inclusion of alternate governance measures and separate measures of data availability and quality had no further effect on this coefficient. Finally, of note is the far lower research produced on Middle East and North Africa (MENA), East Asia and Pacific (EAP) and Eastern Europe and Central Asia (ECA) regions relative to their wealth. In contrast, the coefficients are smaller for both South Asia and Sub-Saharan Africa (SSA) and are not statistically significant in either case. Although the South Asia result could be anticipated, the SSA result comes as a surprise given our priors on the low level of research on African countries. These results suggest that overall publication numbers for the SSA region are 10
small largely because the countries are poor, and not because these countries have received less attention from the research community once we control for income.

Of some interest is the fact that statistically, the US is _not_ an outlier in the volume of research that is produced on it. We examined two possibilities. First, we checked whether the US lay outside the predicted confidence interval (accounting for both the variance of the error and the estimates themselves) from a regression that excluded the US from the estimating equation. Second, we checked for the leverage that the US exerted on the estimated coefficients, by statistically comparing the coefficient vector in regressions with and without the US. Both tests showed that US was no different from other countries once our set of observable characteristics was controlled for; the volume of research for the US lies well within the predicted confidence interval and excluding the US leads to the same coefficient estimates as its inclusion. In other words, a lot more is produced on the US because it is rich and it is big; the natural comparator for the US is all of Europe and here, the volume of research is very similar.

Given that the inclusion of data availability/quality reduces the estimated coefficient on GDP, is it likely that better measures of data would drive the coefficient even lower? A priori this could certainly be true: data collection might not be a priority for poorer countries and might also be difficult to carry out given local conditions (infrastructure, local capacity, etc.) To the extent that data alone, instead of GDP, drive publications, we should find some evidence that the release of _new high quality_ data in low-income countries has some impact on the volume of research.

We look at the correlation between publications and the release of two highly influential and high quality household surveys—the Living Standards Measurement Survey (LSMS) and the Demographic and Health Survey (DHS)—in countries that reported one such survey sometime between 1985 and 2004. Figure 4 plots average publications against the date of release for the first such survey in the country. Taking an event-studies approach, we construct a variable called “Time to Data Release” that is country and time specific, and defined as the difference between calendar year and the year of data release (the event). The variable takes negative values for dates prior to release, is 0 in the year the data were released and takes positive values thereafter. On the vertical axis we plot the average number of publications, against “Time to Event”. As is clear, publications 11
start picking up on the country 3-4 years _prior_ to the release of data. Visually, there is no change in the slope around the time of release. Publications keep increasing till around 9 years after data release and then drop-off (although the drop-off is based on data for fewer countries). Between the very low numbers to the peak, the difference is approximately 5-6 publications a year on the country.

We also conducted a formal event study analysis (see appendix for details). The quantitative results from this analysis are consistent with the hypothesis suggested by Figure 4. While it is certainly possible to find small effects (and different specifications could also yield significance) of data release on publications, the formal analysis confirms a strong pre-release trend without a substantive increase in publications after release. Although the lag structure relating publications to data release is not clear and it may be that more sophisticated modeling yields different results, at first glance more data, do not lead to a sustained increase in research on the country.

# _3.2._ **Stylized Fact 2: Likelihood of Publication in the Top-5 Journals**

Our second stylized fact looks at whether research on the US is more likely to be published in a high-quality journal relative to research on other countries. We specifically look at the geographic determinants of publication in one of the top-5 journals in economics, namely _Econometrica_ , the _American Economic Review_ , the _Quarterly Journal of Economics_ , the _Journal of Political Economy_ , and the _Review of Economic Studies_ .

The basic fact that the top-5 journals are more likely to be accepting of empirical papers written about the US is shown in Table 3. Column 2 shows that of the 3083 papers published in the top-5 journals between 1985 and 2004, a staggering 2,383 focused on the US. Since around 50 percent of overall research volume is on the US, this represents a substantial premium over what would be expected from an equal probability of selection. Column 2 shows just how large the differences are. Across all countries, roughly 1 out of every 25 papers is accepted into the top-5 journals. The difference between the US and the rest of the world is enormous—6.5 percent of all papers published on the US are in the top-5 journals relative to 1.8 percent of papers from other countries. Of particular interest is that once we move outside the US, there is no difference in the likelihood of publication across the different regions—papers from the UK and other OECD countries (some of 12
which are incredibly data rich) have just as low a likelihood of publication in the top-5 as papers from Sub-Saharan Africa or East Asia and the Pacific. To put this in perspective, assume that a researcher’s output is a fairly large 5 papers a year. A researcher who works on the US will get a top5 journal publication every 3 years; one who works on non-US countries will have to wait _10 years_ for such an event to occur. If regular tenure clocks are between 6 to 8 years, this difference could essentially drive the entire evaluation process!

Columns 3-4 show some further characteristics of publications in the top-5 by considering the UKbased journals ( _Econometrica_ and _Review of Economic Studies_ ) separately from the US-based journals ( _American Economic Review_ , _Quarterly Journal of Economics_ and _Journal of Political Economy_ ). Immediately obvious is that the UK based journals place equal emphasis on papers focused on the UK and the US, but the favor is not returned among the US-based journals. For _Econometrica_ and the _Review of Economic Studies_ , a paper from the US or UK is twice as likely to be accepted as papers written about other countries. For the US-based journals, 6 percent of all papers from the US are accepted relative to 1.6 percent of papers from other countries; given the large volume of papers published in these journals compared to the UK-based ones as well as the summaries in Column 2, it comes as no surprise that OECD and UK-focused papers have as low a likelihood of publication in these journals as papers from Africa or South Asia.

To examine this relationship within a regression context, we estimated the following equation: where _Top5i_ is an indicator variable that takes the value 1 if article i is published in a top-5 journal; _USi = 1_ if the article focuses on the US; _UKi = 1_ if the article focuses on the UK and _Xi_ are the other controls used in the regression relating the volume of research to country characteristics. A causal interpretation of the coefficients b (or c) is equivalent to acknowledging a US (or UK) bias in the publication process. Clearly, the key omitted variable from this equation is the quality of the article. If US-focused papers are generally of higher quality, this would imply greater acceptance in the top-5 journals without any bias on the part of these journals. In particular, if different research institutions have different geographic foci, a positive value for b or c might just reflect the fact that 13
researchers of top research institutions are more prone to work on the US or the UK. The data do indicate such a process; for instance, among the Top-5 ranked economics departments in the world (i.e. Harvard, MIT, Stanford, Princeton and the University of Chicago), 74 percent of all papers published have a US focus, while the figure is only 47 percent for all institutions taken together (Table 3 columns 5-6).

To examine this issue further, we propose including a set of dummy variables for the institutions that the authors were affiliated with at the time of publication. The idea is that the institutional affiliations of different authors represent a plausible measure of the quality of the article. To account for co-authors, we introduce the institutional fixed effects in two different ways. Suppose that the quality of the article, _Qi = f(IAj,IAk)_ where _IAj_ is the institutional affiliation of author _j_ and _IAk_ is affiliation of author _k_ . If _f(.)_ is linear, introducing fixed effects for every institutional affiliation will provide coefficient estimates on _USi_ purged of the institutional effects. To account for the possibility that _f(.)_ is non-linear, we also introduce fixed-effects for every institutional affiliation _combination_ in the dataset as an additional check. That is, the co-author pair of Harvard and MIT is treated as an entirely separate institutional combination relative to, say, Harvard and UCLA or UCLA and MIT.

Table 4 presents the results from these specifications. There are several noteworthy patterns. Column 1 suggests that (a) there is no relationship between the likelihood of publication in the Top5 and the GDP of the country beyond the effect of wealth on the total number of papers published; (b) that there is a small positive association of the likelihood of publication in the Top-5 with the size of the country and; (c) that there is also a (small) positive association with the total volume of research on the country. The lack of a relationship with GDP and the association with population is robust across all specifications. The US and the UK are included as additional explanatory variables in Column 2. Immediately obvious is the large coefficient estimate for the US. With a coefficient estimate of 3.9 percentage points the US effect is large both in terms of its size and its statistical significance. The inclusion of the US also shows that the relationship between the likelihood of being published in the top-5 and the total number of publications is driven entirely by the large volume of research on the US and the much higher likelihood of acceptance for US-focused papers (Column 2). With the inclusion of the US, the effect of aggregate research on the likelihood of 14
publication in a Top-5 journal becomes insignificant and of the wrong sign. Column 3 shows that part of the US effect is indeed driven by the different research focuses of higher ranked economics departments. Introducing institutional affiliation fixed-effects in a linear fashion reduces the coefficient on the US to 2.5 percentage points and substantially increases the explanatory power of the regression by 4 times. Columns 3 and 4 show that introducing institutional combination fixedeffects doubles the explanatory power of the regression but _has no further effect on the estimated coefficient for the US_ . That is, we are now able to proxy for quality in a better way but this further improvement in the quality measure appears to be uncorrelated to the relative preference for US-based journals. Finally, there is no correlation between country governance measures and our measure of data quality and availability—as with the coefficient on the UK, these are precisely estimated zeroes. Columns 5 and 6 show the relative differences between the American and the UK-based journals. We confirm that (a) among the UK-based journals, the premium for UK and US focused articles is identical (and suggests a predicted 150 percent increase in the likelihood of publication) and (b) that among the US-based journals, only articles written about on the US are differentially rewarded in the publication process. If anything, there appears to be a small discount for papers from the UK.

Finally, in Table 4 columns 7 and 8, we repeat these specifications for the publishing in the Top-10 journals and find the same patterns.<sup>12</sup> There is a 8.8 percentage point increase in likelihood of publication for papers written about the US (relative to a base probability of 2.8 percent for non-US countries) and this coefficient decreases to a still large 5.7 percentage points once institutional fixed effects are introduced. Once again, the explanatory power of the regression increases 4-fold when including the linear-form; combination fixed-effects again increase the explanatory power substantially without any effect on the estimated coefficient.

How large is this US effect? Predicting the probability of publication in the top-5 journals after replacing all variables to the means of their sample values shows that 2.8 percent of all non-US papers are accepted in these journals relative to 5.3 percent for the US. These numbers are somewhat more positive than the raw differences but still suggest that with a research output of 5

> 12 The top-10 journals include the _Journal of Economic Theory_ , the _Journal of Econometrics_ , the _Journal of Finance_ , the _Journal of Financial Economics_ , and the _Review of Financial Studies_ .

15
papers a year, it would take under 4 years for a researcher working on the US to produce a top-5 article relative to above 6 years for those working on non-US countries.

How we _interpret_ these results depends on our priors. Those who believe that there is no discrimination in the publication process could well argue that the US effect is an upper-bound because further controls for quality would decrease the coefficient---perhaps to zero. Note though, that the opposite may also be true. Suppose that researchers are placed in the top ranked economics departments only if they have a minimum number of top-5 publications. Then, if our prior is that there _is_ discrimination in the top-5 (and top-10) journals, it must be the case that researchers working on non-US countries in the top-ranked institutes produce _higher quality_ papers than their colleagues who work on the US. In this case, the institutional fixed effects produce a lower-bound of the US difference relative to the raw correlations. Regardless of our judgments of what these coefficients imply, these associations and correlations may be helpful in understanding the publication process in economics and the potentially differential rewards for academic research around the world.

# **4. Conclusions**

All we have done is present some facts. The correlations with GDP may be useful because they present country-specific applied economic research in a framework familiar to many. The differences in the volume of research were eye-opening for the authors and we may well wonder what the basis of economic policies will be in the 20 poorest countries, where 3 papers per country are written every 2 years (and 1 paper per year per country if we exclude Ethiopia and Tanzania). That these differences are driven to a large extent by the income level of the country is particularly problematic since a lot of the research on low-income countries is done by researchers outside the country, rather than in local institutions. Therefore while it is understandable that other outcomes, such as health and education, have a demonstrated association with country income (largely as a function of the country’s own systems); there is no good explanation for why research, which is driven at least partially by non-local institutions, should suffer similarly. If we believe that there is some link between good economic policy and country-specific research, the low volume of research on poor countries is a cause for concern. Disciplines such as anthropology have been through an epistemological debate on the relative benefits of area-specific versus “general” research, and have 16
come down strongly on the importance of the former. To our knowledge, there has been less such debate in the economics discipline and these results (and the accompanying database) present one starting point for such a conversation.

The results on the relative likelihood of publication in the top-5 journals may also have efficiency implications. The results confirm the high premium that US-focused papers command in such journals. They also suggest that this bias favors the US in particular rather than “rich countries” in general. Papers from the UK and other OECD countries, excluding the US, have precisely the same likelihood of publication as those from India or Vietnam. Nevertheless, the potential bias against non-US countries may have more pernicious implications for low-income countries. This is because, relative to OECD countries, it is likely that a larger fraction of research on low-income countries is conducted in the _US_ rather than in the country itself. Therefore, researchers who would like to work on Vietnam but are based in the US have to choose their country carefully before starting work. As a consequence of potential bias, they may change the country focus of their research (to the US, if they are rational) _or_ decrease the marginal investment in quality and aim for lower ranked journals. As one professor from a top-ranked department told us, after umpteen rejections for work on a lowincome country (and ultimately 2 top journal publications) he decided to switch to US-focused research. Following the switch, both the papers that he has written since were accepted in the first top journal that he submitted them to! For OECD countries, local researchers may be less sensitive to bias in the top-5 journals and therefore the efficiency implications could be less severe.

We hope that there will be further additions to this database from interested researchers (e.g. looking at networks of authors or the effect of money for research on outcomes). An open question is how knowledge that has high marginal value for a particular country (the first poverty estimate for Togo where none existed) but low marginal value for the discipline (been there, done that for many other countries) should be disseminated. Equally importantly, the question of whether there is bias in the top journals is critical for the discipline. Hiring decisions, visibility and the quality of subsequent work all depend on this crucial acceptability. A bias as large as the one we document would be unacceptable.

17