'Do the data in fact deceive'? An analysis of the roles of evaluation and the production of aid effectiveness at the World Bank
Keywords: Development Institutions
Evaluation

This is a dissertation about the organisational structure of the World Bank, the professional practice of evaluation, and the meaning of the concept of aid effectiveness in practice. In international development, evaluation is a professional activity that determines and then reports on the impacts of aid projects and programmes to the clients of such efforts and to the public. 'Aid effectiveness' is a concept that refers to a standard of how aid projects and organisations should operate and the results, such as economic growth and poverty alleviation, which these efforts should deliver in order to work. The concept of 'aid effectiveness' has also been used in the debate about international development as a system and its reform. Given that aid policymakers and academic researchers often use the data contained in development organisations evaluations to determine the extent to which aid projects and programmes are 'effective', it is critical to analyse what these evaluations measure and what influences their ratings and judgments.
Based on a case study of the World Bank, the analysis is primarily qualitative and draws on both interviews with evaluation professionals in the World Bank and content analyses of the logical framework, indicators, and language in the World Bank's evaluations at the project- and country-level. Building on the previous theoretical work in post-structuralism that considers how international development organisations 'produce' their work through certain terms and processes (Escobar, 1995; Crush, 1995), I assess how the professional practice of evaluation in the Bank 'produces' the results of aid at the project- and country-level, specifically in the evaluation reports that it makes publically available. The World Banks data and evaluation reports are a window through which to understand the impact of aid, and several factors that influence this 'window' are assessed, including the institutional role of evaluation, the professional practice of evaluation, and the required evaluation processes within the World Bank.
The study has important implications for practitioners of international development, academic researchers, and evaluation professionals who endeavour to improve the aid system and often rely on the results of the World Bank's evaluations to inform their understanding of the impact of particular development efforts. By reshaping the discussion from one which considers if aid 'works' to one about the data and the process of making a judgment about the success of aid projects and programmes, I articulate what the role of evaluation is in practice and what the World Bank's resulting evaluative data do and do not reflect about the World Bank's work. The relationship between the 'scales' of aid is also analysed by comparing and contrasting the evaluation processes at the project-level and the country-level. I challenge the notion of a 'micro-macro paradox' (Mosley, 1986) between the successful results of the World Banks projects and the economic development in its client countries by articulating the actual meaning of this data in context, the unseen institutional forces that shape this data, and the difficulty of asserting a linear relationship between the results of projects and programmes on different scales of aid.