DC Development Forum BBL, Monday October 17, 2005
How should Japan respond to CDI2005?
Points from on-line discussion (Aug –Sept 2005)
Disclaimer
This memo is a collection of various views of members of DC Development Forum. As such, eachcomment is not necessarily shared by majority of the members.
1. General Comments
Purpose of the Index
The CDI does not seem to answer what is the effective development assistance, though it
successfully draws broad attention to the issue. Indeed, the CDI appears to be constructed
as provocative as possible. However, which objective—provocativeness or evaluation—
are you interested in?
David Roodman
I don’t think anyone has the full answer to what is effective development assistance.
The CDI does what it can based on current thinking. Most would agree that tying aid
reduces its value, that giving aid to a very rich country like Israel or a very corrupt one
like Cameroon generally does not help development very much, that there are too
many small aid projects in Tanzania and Uganda. That said, one can think of exceptions
to these generalizations. It is true that the CDI does sometimes go a bit beyond solid
evidence in order to make people aware that certain issues, such as migration or aid
project proliferation, are important, even if we do not understand them well. But when
we do this, we still think hard about the design choices we make.
The CDI is based on “inputs” of policies but not on “outcomes.” As a reasonableindicator of commitment to development, shouldn’t it incorporate outcome measures
such as poverty reduction and growth?
David Roodman
I don’t think so. The mission of CGD is to focus on rich-country polices, and the CDI
helps us do that. The Human Development Index focuses on outcomes, and that is finetoo. But it is very hard in general to figure out which donor policies contributed to or
prevented a given outcome, so their relevance as indicators for the CDI is low.

Neutrality of the Index
Isn’t the selection of assessment criteria and weights biased to favor certain donors andhinder others, perhaps to produce an outcome in line with a pre-conceived view? For
example, it is difficult to believe that the adverse impact of Japan’s protection of rice on
developing countries is 70 times more than that of Europe’s protection of sugar. In other1words, is the tariff equivalence of import protection a good measure of impact on growthin developing countries? How about the choice of production share, instead of importshare?
David Roodman
Does someone have evidence that the Japanese rice tariff is not 70 times more
harmful? Huge numbers of poor people grow rice in countries such as Indonesia,
Thailand, and Vietnam. They could benefit if it were easy to export to Japan. The
number of people in sugar production is far smaller. The problem with weighting tariffs
by imports is that where protection is highest, imports are lowest, so that high tariffs
are automatically underweighted. This is the well-known endogeneity bias problem, andis why trade economists work so hard to avoid it. I think Vietnam’s rice production is abetter indicator of its ability to export to Japan than is its actual exports to Japan.
Of course, the CDI involves a lot of judgment, and there are always biases. That said, I
had no idea when we started this project that Japan would come out at the bottom.
Upon reflection, I realized that the reason for this result is that the CDI is strongly
shaped by a belief that international openness in rich countries is good for poorer ones.
I think it is clear that Japan is more closed, more inward-oriented, than the U.S. andEurope. One can argue that our belief in the value in openness is wrong. That would bea legitimate and serious debate. But I do wonder: Could Japan have grown as fast as it
did without access to U.S. markets?
Is the Index backed by sufficiently sound academic research? For example, is there acredible research that concludes that net resource transfer is an appropriate measure of
the impact of aid on growth and poverty reduction? In this regard, the CDI does not seem
to take adequate account of previous research on aid effectiveness, for example, Burnsideand Dollar (2000), Easterly, Levine and Roodman (2004), and Alesina and Dollar (2000).
David Roodman
I agree that the evidence on whether total aid correlates with overall economic growthis weak. (I’m not sure why Burnside and Dollar is mentioned, since it argues the
opposite, or why Alesina and Dollar is mentioned, since it is not about aid impact.) But I
do think that aid has helped and can help in important ways. Aid helped eradicate small
pox. Aid contributed to the huge fall in infant mortality over the last 50 years in poor
countries, the rise in life expectancy, and perhaps the fall in fertility too, which have all
occurred faster than they ever did in the West. Aid fostered the green revolution, which
ended famine in India. This is why I am still comfortable putting aid in the CDI. Leaving
it tells people that aid matters and that we should think hard about how to make itbetter.
Most of the background papers are unpublished. Are the results robust? Have theestimates that underpin the CDI gone though external review processes?
David Roodman
2

The background papers are on the web site (www.cgdev.org/cdi, click on“Methodology.”) Perhaps what is meant is that they are not published in academic
journals, which is true. The original trade paper was published as part of William Cline’s
scholarly book for CGD, Trade Policy and Global Poverty. There is a new trade paper
this year, which I have submitted to a journal. I may submit the aid paper to a journal
as well. But I think it is important to remember that the background papers are notexactly pieces of research. They are documentation for a set of design choices, and so
are not necessarily appropriate for the academic literature.
All of the papers do go through an external review process, which I run. I send thepapers to experts in the U.S. and other countries for review, then discuss the commentswith the authors and ask them to revise.
There is an endogeneity problem stemmed from the donor-recipient relationship. For
example, the Doing Business Indicators, developed by the World Bank, traditionallywork unfavorable for countries based on French legal system (including Francophone
African countries as well as France itself) just because the way the indicators are set
up. However, their improvement in the time-series would be more appealing from thedonors’ perspective. Hence, I would suggest that you should look at an index along thetime-series.
David Roodman
It looks like there are two issues here. The first seems to be about how when donors
evaluate recipients, they bring their own biases. But the CDI of course does notevaluate recipients, so I don’t see how the “donor-recipient” relationship is so relevant
for the CDI. Perhaps what is meant is that when an American evaluates other richcountries, he too is biased. Certainly, I am ready to listen to this criticism, which is one
reason I value this kind of discussion. Last fall I was in Tokyo and visited FASID. There,
I learned about the arms exports database of the Stockholm International Peace
Research Institute, which made it possible for me to add a penalty for arms exports thisyear, which many Japanese had argued for. I faced a lot of skepticism here, especiallyfrom the Michael O’Hanlon, who led most of the work on the CDI security component,
but I persisted and finally reached a consensus with him, with the help of another
American expert.
Still, I think the bottom line is that the CDI results are robust. Japan clearly is different
from the other 20 countries. It is more inward oriented, which shows up in low aid,
trade, migration, and security scores. The CDI is based on the belief that openness todeveloping countries is good. Perhaps that is a function of my American bias. I think if
you are going to question the CDI, this belief is what you should challenge.
The second point here is about looking at changes over time. I agree this is important,so as the years go by, we will always back-calculate the latest index methodology topast years, to allow comparisons. (And I think the methodology will not change so3much in the future.) On the CDI web site, you can see three years of data now. But I
still think comparing countries in a given year is effective too, so we will do both.
As ownership (local partnership) is at the forefront of the donors’ agenda these days, itwould be better to somehow reflect the degree of ownership in the CDI. Relatedly, would
there be any way to estimate some sort of best fit in demand-supply of development
assistance? Also, how could we measure efforts of a donor country to assist othercountries to become new effective donors? (e.g., tri-lateral cooperation of Japan)?
David Roodman
Ownership might be interesting to include, but I think it is extremely difficult tomeasure, and also you have to be very clear about who is “owning” the policies.
Indonesia’s growth success was based on policies owned by a technocratic elite. So
maybe that is the important kind of ownership. Or is the key PRSP-style broad-basedownership?
I’m sorry I don’t understand what it would mean for the CDI to “estimate some sort ofbest fit in demand-supply of development assistance.” Is the idea that aid would be
rewarded more if it goes to countries that demand it more? How would you measuredemand? It seems like most countries take all the aid they can get, even when theirleaders say they cannot handle all the aid.
Trilateral cooperation is a new idea to me. It sounds like a good thing, but it also
sounds small. Maybe it is already counted as ODA?

Difficulty with cross- donor analysis
Cross-donor comparison is not as straight forward as it may seem. Citizens in eachcountry have different preference in helping others domestically (e.g., charity, income
distribution policies, etc). The preference is deeply rooted in historically determinedcharitable culture and religious norms. We can control this fixed effect by measuring aid
to poor people abroad relative to transfers to poor people in the donor country.
David Roodman
I agree it is not straightforward! It is very hard. I think it would be interesting tocompare charity abroad to charity at home. But I am also comfortable with the CDI notdoing that. It is called the “Commitment to Development Index” but it is really a“development policy index.” It is not about intentions, but actions. I think it best to
make it good at measuring actions. I don’t want to make a country look good justbecause it is even more stingy at home than abroad.
Should the countries that earn foreign currency from their abundant natural resources(e.g., Norway) be equally compared with the countries without any natural resources? In
Japan, the inheritance tax rate are higher than the income tax rate, because Japanese as a4whole agree that the inherited wealth should be used more to redistribute economic
powers among people.
David Roodman
I think the question about Norway is a special case of a broader question: shouldcountries that are richer, for whatever reason, be compared to countries that are not asrich? I think that within certain bounds, the answer is yes, often by dividing by GDP. I
am comfortable comparing Japan to Greece, but not Ethiopia. I think a softwarebillionaire and an oil billionaire have an equal obligation to help—which is why boththeir heirs should pay the same inheritance tax. It is important to remember that
Norway is an exception to broad pattern known as the “resource curse.” Countries withnatural resource wealth are usually poorer. Natural resources appear to generate huge
profits for a small minority which then corrupts politics. From this point of view, Japanand many other Asian countries are the fortunate ones, because they are so resourcepoor. This may have made Asian countries more equitable, which led to broader-based
investment in education, and thus to economic growth.
The weights attached to each assessment category should be substantially different
among donors and/or recipient countries as they should reflect the importance of eachcategory in terms of its impact.
David Roodman
I think it would be very interesting to do different versions of the index for individual
recipient countries, or at least major regions such as Africa, Asia, Latin America, and wemight do this some day.
2. Specific Comments
Measurement of contribution of aid to development
We believe that the quality-adjusted Aid/GDP is a more meaningful indicator ofcommitment. We realize that you have made quality adjustment on aid data, but suchadjustments may be considered rather minor.
David Roodman
I agree the quality adjustments don’t have a major impact. This may be because I am
unable to measure aid quality very well. But it could also be that aid quantity/GDP really
does vary far more from country to country (by a factor of 7) than aid quality does, inwhich case this result is accurate: aid quantity dominates in comparing donors.
Can ODA promote economic growth of recipient countries unconditionally? According
to Burnside and Dollar, only when the policies of the recipients are appropriate, aid
works. Your paper with Easterly even questions the Burnside and Dollar results, findingthat there is NO growth effects of aid even if we control for governance. If this is the case,5

how can the inclusion of aid variables be justified as one of important indicators of thecommitment to development? Isn’t there an inconsistency in your arguments?
David Roodman
Good question. I discussed aid effectiveness a bit already. I would only add thatabsence of proof in one study is not proof of absence.
It is unconvincing to treat aid as a pure resource transfer, ignoring the different modalities
of assistance, for example, grants (apples) and loans (oranges). Also, contribution tomultilateral institutions should be valued higher than bilateral aid, since the former isshown to be apolitical (see Cassen et al’s book, Burnside-Dollar, final section, and othernumerous academic papers.) Relatedly, shouldn’t strategic aspects of the ODA becontrolled, given research results such as in r Alesina and Dollar (J. of Economic
Growth)?
David Roodman
I agree that aid is extremely heterogeneous, that different kinds of aid should havedifferent impacts in different contexts. However, I wouldn’t pick the grant-loandistinction as the most important one. I would expect that a $50 million World Bank
education loan for a broad program of school building and teacher training would haveabout the same effect as an equivalent $50 million grant—except of course that theloan has to be paid back eventually, which is like negative aid. One might argue thatloans make governments act more responsibly, but I really doubt it. Just look at mycountry. Or look at the HIPCs, which are having their debt cancelled because the money
is gone. I agree that multilateral aid can be expected to be better than bilateral aid. In
the index, we do not directly favor multilateral aid, but it comes out as being higherquality, so the more that bilateral donors send their aid through multilaterals, the better
they do.
What is the compelling reason to standardize the amount of ODA by GDP of donor
countries (not by recipients’ aggregated GDP)? An alternative variable to consider may
be the ODA per capita of each donor country because this variable can capture
contributions (or burden sharing) per person more accurately. Moreover, if there is aminimum required quantity of ODA to be effective, there will be a nonlinear relationshipbetween the amount of ODA and outcomes. If this is the case, the absolute amount of
ODA provides important information.
Alternatively, you can employ the following variables as an indicator of quality-,
modality-, and scale-adjusted aid from donor d:
n

wr
r 1

BI _ AIDrd
GDPr

n

wr
r 1

MULTI _ AIDrd,
GDPr

where and β are weights attached to bilateral aid and multilateral aid, respectively.
BI_AIDrd is the amount of aid from a donor d to a recipient t. GDPr is a recipient’s GDP.

6

MULTI_AIDrd is the amount of multilateral aid financed by donor d then disbursed tocountry r, e.g., the amount of Japan’s contribution to the World Bank which lead to aid to
Indonesia. Of course, this variable should be estimated but we can easily accomplish it
under reasonable assumptions.
David Roodman
There are several proposals here. If we took aid/(total recipient GDP) instead ofaid/(each donor’s GDP), then since total recipient GDP is the same for every donor, thiswould be equivalent to just using absolute aid (dividing each donor’s total aid by thesame number does not change the comparison). The CDI does not use absolute aid
because its purpose is to measure the extent to which donors are living up to there
potential to help.
ODA per capita is worth considering. But it has two disadvantages. First, if Japan isroughly twice as rich as Greece (in GDP/capita terms), it is reasonably to argue that itshould give twice as much aid. Put another way, if Greece gives the same amount perperson as Japan, than it is doing twice as good a job of realizing its potential to help.
Second, there are currency issues. Japan’s ODA/capita is in yen and America’s is indollars. Exchange rate movements can make countries suddenly appear more or lessgenerous. The DAC has developed deflators to address this problem, but there is noperfect solution for it.
If there is some kind of nonlinear relationship between aid and impact (and I am
extremely doubtful of the ability of econometrics to detect this effect reliably), then one
has to think about which donor is the marginal donor in each country. The first donor’s
aid may have no impact, but still help the country reach the minimum aid thresholdneeded for impact. Which donors then should get the credit? Large donors can be smallin given recipient countries and vice versa. I think this approach would become very
complicated and conjectural.
I actually don’t think it would be so easy to estimate and in a sound way. Resultsfrom regressions with aid split into multilateral and bilateral components are fragile
because the samples are small and the two variables are collinear. Moreover, thisformula would penalize giving aid to large countries. One dollar of aid to India would
show up as much smaller in the aid/(recipient GDP) fraction in the formula than would
one dollar of aid to Nauru. So a country could raise its aid score by moving its aid tosmall countries. Mark McGillivray proposed an aid index with this problem in 1989 in thejournal World Development. Then Howard White pointed out the problem in acommentary, and McGillivray fixed the problem in a subsequent article. I think it doesn’t
make sense.
Legitimacy of the seven categories

7

How is the specification and weight of each assessment category justified? For example,
how important would Migration be, in terms of contributing to other countries’
development?
David Roodman
Modeling by Walmsley and Winters (2003) suggests that if rich countries increased theirtemporary migrant worker stocks by an amount equal to just 3% of their labor forces,
global income would increase $150 billion, with most of that going to the temporary
workers themselves. That is a huge number. So it is not obvious that the impact of
migration is small. One could also argue that investment, environment, technology,
security, etc., are hugely important. But we don’t really know which matters most. Thisis one reason I give all the components the same weight in the final average. Moreover,“equal weighting” does not mean “equal importance in the index,” as I discuss below.
For Security, in the Index, the positive contribution of the financial and personnel
contributions to peacekeeping operations and humanitarian interventions has a heavier
weight than the negative one for penalizing some arms exports to undemocratic nations
that spend heavily on weapons. What are the reasons behind this?
David Roodman
I agree this is a point you can legitimately argue. My thinking was that sales of
weapons to Saudi Arabia probably had less total impact than NATO going into Kosovo
or Australia going into East Timor. But keep in mind, as I argue in the index technicalpaper (on the web site), equal weighting is not a well-defined concept. If there are two
indicators that are constructed so that on one, there is a very wide spread around theaverage, while on the other every country is essentially average, then if you “equal
weight” the two indicators, the one with the wide spread will determine the overall
results. So “equal weighting” is very different from “equal importance.” For an example,
note that there is a wider spread on the CDI aid component than on the CDI investment
component, so that the aid component has much more effect on the overall results.
The arms exports indicator is one with a very wide spread. Japan and some othercountries get a perfect 10 on it. At the other extreme, the U.K. gets a –17 (negative
seventeen), which is why the U.K. comes in second-to-last on security. In 2004, before
I added the arms exports penalty, the U.K. was #3 on security. So from the point ofview of the U.K. (and also France and the U.S.) the arms export penalty has a very big
impact.
For Security, contributions to the special trust funds for post-conflict reconstruction
under the World Bank and/or the UN should be taken into account.
I would think these are already counted as ODA in the aid component.

8

To compute Trade protection, the production-based weight is used this year, replacing
the import-based weight. As a result, the weights for rice, wheat and sugar (the items
which have relatively high tariff in Japan) increased, making Japan’s trade appear evenmore protective. The explanation does not seem objective: production weight is usedbecause otherwise trade protection turns lower for countries such as Japan, Norway andSweden (those who are known to be protective) than Austria.
David Roodman
I think I responded to these questions above. I use production weights because importweights are known to be endogenous and bias results. We made the switch this year
because a new data set became available that allowed it—the Market Access Map(MAcMap) data set of the Centre d’Etudes Prospectives et d’Informations
Internationales (CEPII). I actually considered using value added rather than production,
but this would have made Japan look even worse.
Also for Trade, while the weights for agricultural goods rise, the weights for industrial
goods tend to shrink. This overlooks the importance of trade in industrial goods, whenmany developing and emerging countries strive to grow through exporting their industrial
goods.
David Roodman
Actually, the weights on manufacture are much higher than those on agriculture. If you
open the index spreadsheet (http://www.cgdev.org/doc/CDI/Index2005.xls), go to the“Trade 2005” sheet, and look at cells D59 and E59, you’ll see that the weights on
agriculture total $1.7 trillion while those on everything else total $7.2 trillion. The
reason that agricultural tariffs dominate the overall results is that protection in
manufactures is fairly low in all rich countries. The major differences in protection are in
agriculture, so they dominate the overall trade results—another example of thedifference between weights and importance.
Japan's contribution to Environment is not sufficiently reflected in the CDI. The Index
should reflect not only gas emission, but also the energy usage/GDP ratio (which is abroader indicator) as well as the development and export of energy saving technology. It
is my view that inefficient energy consumption is hurting developing countries not only
through higher oil prices but also various adverse effects of global warming. This - andaging - is the biggest challenge that the world economy face in the long run.
David Roodman
The greenhouse gas indicators in the index capture fossil fuel use, which accounts forthe majority of commercial energy use. Left out are hydropower, nuclear, wind, solar,
and geothermal. I would need to have it explained to me how using these energy
sources hurts developing countries. I think the strongest argument would be forpenalizing nuclear energy use. Is this what you are suggesting?

9

As for exporting efficient technologies, we try to capture some of this through thetechnology component, and also through the investment component since a lot oftechnology transfer occurs through FDI—but we may not capture it well. One key
question is what is the policy that is to be rewarded. Exports themselves are not apolicy.
3. Future Prospects
The index offers a good opportunity for Japanese people to undertake discussions of thecommitment to development not only by the government but also by the public at large.
Historically, viable dialogue among researchers, policy makers, and field workers has
been weak in Japan. A stronger cooperation among them is needed to better communicate
with foreign parties and to strengthen the effectiveness of Japan’s ODA. In addition, we
hope to see more intellectual interactions between leading think tanks, such as CGD, andJapanese counterparts to share views and knowledge on economic development policies.
In particular, there are many lessons to learn from CGD, for example, effective
publication of research results.
David Roodman
Well said! Thank you so much for your interest, your comments, and your constructive
spirit. I am grateful for the opportunity to discuss these issues with you, and lookforward to the continuing conversation. Domo arigato.

10