# Critical Realism and ICT4D Research

Richard Heeks, P. J. Wall

► To cite this version:

Richard Heeks, P. J. Wall. Critical Realism and ICT4D Research. 14th International Conference on Social Implications of Computers in Developing Countries (ICT4D), May 2017, Yogyakarta, Indonesia. pp.159-170, 10.1007/978-3-319-59111-7_14 . hal-01650045

HAL Id: hal-01650045

https://inria.hal.science/hal-01650045

Submitted on 28 Nov 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Critical Realism and ICT4D Research

Richard Heeks¹ and PJ Wall²

¹ Centre for Development Informatics, University of Manchester, UK
Email: richard.heeks@manchester.ac.uk

² School of Computer Science, Trinity College Dublin, Ireland
Email: wallp2@tcd.ie

**Abstract.** There is little overt engagement with research paradigms in ICT4D research but what there is shows a dominance of positivism and interpretivism. In this paper we explore the value of a “third way” research paradigm: critical realism. We concisely review the main features of critical realism: its ontological realism combined with epistemological relativism; its iterative, pluralist and reflexive methodology; and its emancipatory values. Alongside the general value of explicit use of any research paradigm, we argue two particular types of value of critical realism for ICT4D research. First, generic values including exposure of context, a contingent causality that reflects real-world ICT4D experiences, legitimisation of different stakeholder views and reduction of research bias, and support for ICT4D’s interventionist approach and its goal of delivering international development. Second, specific value in addressing current trends in ICT4D research: the growing search for causal links between “ICT” and “D”, and the political and ethical turns in ICT4D that are spurring researchers to engage more with issues of power, rights and justice. We conclude that delivery of critical realism’s utility will require the ICT4D research community to take actions that enable this emergent research paradigm to flourish.

**Keywords:** ICT4D, critical realism, philosophy, methodology

# 1 Introduction

From the turn of the 21st century, there has been an ever-growing body of research and publication examining the role of information and communication technologies in socio-economic development (ICT4D).¹ Within this body of work, explicit consideration of research philosophy generally and of specific research paradigms is rare (Gomez & Day 2013). However, analysis has been undertaken to infer the paradigms being used which finds that interpretivist and positivist approaches dominate the cur-

¹ 299 publications during 2001-2005; 1,840 during 2006-2010; and 4,150 during 2011-2015 (Google Scholar: English language only; excluding patents and citations).
rent body of ICT4D research (Walsham & Sahay 2006, Gomez & Day 2013)<fnref n="2" />. Alongside generic concerns about the limitations imposed by this philosophical duopoly, each of these two paradigms individually has a number of limitations which constrain ICT4D (and other) research. Recognition of these limitations many years ago within social science overall resulted in a revisiting of the realist paradigm, and its development into a particular body of philosophical thought that has come to be known as “critical realism” (e.g. Bhaskar 1975, Bhaskar 1979). From these philosophical origins, critical realism has spread into use in a number of academic disciplines including one of ICT4D’s main cognate disciplines, information systems (e.g. Mingers 2004a, Mingers 2004b).<fnref n="3" />

Given the relative absence of explicit critical realism in ICT4D research, and the lack of discussion about research philosophy, we perceived a knowledge gap. In this paper our aim is therefore to explore the relevance of critical realism as a philosophical paradigm for ICT4D research. In Section 2 of the paper, we outline the main features of critical realism. We then compare these to features of ICT4D research looking for both the generic, enduring value of critical realism in ICT4D research, and also for specific fit with current trends in the field. After noting challenges of applying critical realism, we draw final conclusions and thoughts about possible actions.

## 2 The Features of Critical Realism

It is possible to trace a historical trajectory within social science that is dominated first by positivism which is then joined by interpretivism. Both paradigms have been subject to various criticisms (e.g. Kanellis & Papadopoulos 2009, Bevir & Rhodes 2005, Smith 2005). These criticisms of interpretivist and positivist approaches have led a number of scholars to search for a “third way” (Allen et al. 2013:835). Critical realism has been put forward as one such, and in the following sections we will give a very brief overview based on the four main differentiators of research paradigms (Cresswell 2013):

* Ontology: what the paradigm understands to be the nature of reality.

* Epistemology: what the paradigm understands about how we construct and evaluate knowledge about that reality.

* Methodology: what research strategy, methods and techniques the paradigm uses in order to gather and analyse data.

* Axiology: what the paradigm does and does not value in research.

<footnote n="2">² With some sub-domain differences: e.g. IFIP WG9.4 conferences tend more towards interpretive work; ICTD conferences tend more towards positivist work (Gallivan & Tao 2013, Gomez & Day 2013).</footnote>

<footnote n="3">³ There is much less discussion of research philosophy in the other main cognate discipline – development studies – and little or no explicit discussion of critical realism.</footnote>
## 2.1 The Ontology of Critical Realism

Critical realism adopts a three-level “stratified ontology”, as summarised in Fig. 1 (Mingers 2004a).

<figure><img src="image_2.png" /><figcaption>Fig. 1. Stratified Ontology of Critical Realism</figcaption></figure>

Within the domain of the real lie generative mechanisms: “causal structures that generate observable events” (Henfridsson & Bygstad 2013:911). These mechanisms have an “intransitive” objective reality independent of human thought or belief: they are not merely social constructions. An example of an ICT4D-related mechanism would be an information infrastructure of technology and people in a country that attracts digital service providers, who create new services and thus attract more users, thereby strengthening the information infrastructure and creating a virtuous circle (Bygstad & Munkvold 2011).

Within the domain of the actual lie events: “specific happenings resulting from causal mechanisms being enacted in some social and physical structure within a particular ... context” (Williams & Karahanna 2013:939). An example of ICT4D-related events might be appointment of an ICT4D champion, formation of an ICT4D strategy group, or design of an ICT4D app. However, critical realism rejects linear notions of causality between mechanisms and events. It takes an open systems view of the world in which multiple mechanisms intersect, thus creating a “contingent causality” that is context-dependent (Smith 2010).

Social structures underlie and create the mechanisms within the domain of the real. There is therefore a danger that critical realism might be seen as structuralist: focusing on social structures as explanations and ignoring or downplaying the role of human agency. But this is inherently not so (Njihia & Merali 2013). One of the most explicit explanations has come from Archer (e.g. 1995) and the notion of morphogenesis; a cycle of three phases: “(a) the existing structure that shapes but does not determine actions that are about to take place; (b) social interaction, which in turn leads to; (c) structural elaboration that either changes [“morphogenesis”] or reproduces [“mor-
phostasis"] the social structure" (Ram et al. 2014:465). We can understand this as a
rolling iteration between structure and agency; between the real and the actual.

## 2.2 The Epistemology of Critical Realism

Within the domain of the empirical lie human experiences and observations of the events generated within the actual. The underlying, intransitive structures and mechanisms of the real domain cannot therefore be directly experienced; hence they cannot be directly measured by research (Danermark et al. 2002). But, as Fig. 1 indicates, the empirical is contained within the actual and the real. Thus, any experience is shaped by the context of that experience: it is not objective but is contingent and transient. Different observers will give different accounts of events depending on, for example, their own historical experiences and their own position within social structures (Dobson 2001).

## 2.3 The Methodology of Critical Realism

Three methodological features of critical realism will be identified here: iterative
retroduction, pluralism, and reflexivity. Retroduction – literally meaning leading
backwards – is a “...mode of inference in which events are explained by postulating
(and identifying) mechanisms which are capable of producing them…” (Sayer
1992:107). It therefore means moving back across the domains from the empirical via
the actual to the real, and represents the way by which the domains are connected
within active research. Though potentially a one-time movement, in practice, retro-
duction is more generally understood as part of an iterative cycle in which mecha-
nisms are postulated from existing data, evidenced or otherwise through gathering of
new data, and supported or revised or rejected iteratively during the analysis of that
data (Easton 2010).

Given the transitive relation between the empirical and the actual, critical realism requires pluralism of methods in order to improve the validity of insights into events (and, hence, into underlying mechanisms) (Downward & Mearman 2007). This is typically understood in terms of two types of triangulation. Data triangulation is most often operationalised by gathering data from different stakeholders, thus allowing for multiple perspectives and inter-subjective insights into the events of the actual. Method triangulation means critical realism is associated with mixed-methods research: combining qualitative and quantitative methods (Zachariadis et al. 2013: see Fig. 2).

Because the empirical is subject to the influence of context then data gathered will be value-laden. And this will also be true of the research process itself. Critical realism therefore asks of its researchers that they and their research participants be reflexive: “a dynamic process of interaction within and between ourselves and our participants, and the data that inform decisions, actions and interpretations at all stages of research” (Etherington 2004:36). This process can also be retroductive: seeking to expose the underlying mechanisms that can explain the events of the research process.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Methodology of Critical Realism</figcaption></figure>

## 2.4 The Axiology of Critical Realism

So far our explanation has been more realist than critical. The extent and nature of the critical within critical realism is debated (Klein 2004): for example, it could just mean critique of prior research paradigms. But there is a central thread within many writings on critical realism that associate it with critical theory (Bhaskar 1989). From this outlook, critical realism contrasts itself to positivism's notions of value-free research in two ways. Like interpretivism its epistemology recognises observation and research to be value-laden: shaped by experience and context. But additionally, it seeks research to be values-driven: specifically driven by the values of emancipation. This means recognising the way in which the social structures and mechanisms of the real domain can sometimes serve to generate events and processes that are oppressive and outcomes that are unequal. But beyond merely understanding the world, the critical of critical realism inspires changing the world through engagement with practice: “developing ways of working with practitioners to help them understand their situation, identify barriers and opportunities for change and implement solutions” (Ram et al. 2014:465).

# 3 Critical Realism and ICT4D Research

In this section, we will look at the potential value that critical realism can bring to ICT4D research. We will also acknowledge some challenges.
**3.1 Generic Value of Research Paradigms in ICT4D Research**

We have previously noted that explicit consideration of research paradigms is rare in ICT4D publications. Is this a problem? There are certainly arguments, beyond use of any specific approach, about the general value of such consideration. Beyond the direct contribution to a particular ICT4D research project or publication, “explicit recognition of research philosophies can help researchers’ self-development, their capacity to analyze the work of themselves and others, and the academic credibility of a research field” (Heeks & Bailur 2007:252). There is also a current value to ICT4D as a research sub-discipline. If it is to achieve greater maturity and academic recognition then there will need to be greater attention to, and use of, research paradigms.

**3.2 Ontological Value of Critical Realism in ICT4D Research**

There is widespread agreement in ICT4D research on the importance of context; for instance, the way in which the outcome of ICT4D projects is influenced by the interests of stakeholder groups (Bailur 2006), by developmental goals (Prakash & De 2007), by local language and culture (Sinha & Hyma 2013), etc. Conversely, lack of engagement with context is a criticism of some ICT4D research and practice (Dodson et al. 2013, Turpin & Alexander 2013). Critical realism can help address these contextual issues: it requires an investigation of context because context is represented by the domain of the real. Other research paradigms can encompass context but only within critical realism is it an integral and required component (Ram et al. 2014). It forces that involvement with ICT4D context to focus on what is present – relations, systems, ideas, resources – rather than conceiving development contexts solely in terms of lack or absence (Njihia & Merali 2013).

We can identify two further values that the contextualist ontology of critical realism brings to the current state of ICT4D research. The first is its incorporation of causality. The open systems view taken by critical realism is one that does not provide for causal mechanisms that operate in the same way at all times and in all contexts; but it does develop an understanding of causality. As Njihia and Merali (2013:866) explain, critical realism “should tell us with good reason why things are as they are now and where they could be heading, based on the causal tendencies of identified generative mechanisms”. This is especially relevant in ICT4D today. The focus of ICT4D research has been shifting over time from issues of readiness and availability through adoption to development impact (Heeks 2014b). But the current interest in development impact has been hampered by lack of research that investigates or demonstrates a causal connection between technology and development (Andersson & Hatakka 2013). Critical realism helps engage with that causal connection.

The second timely value of critical realism is its relevance to what we might call the “political turn” in ICT4D. This has arisen to reflect the growing role of ICTs in politics in developing countries (e.g. Breuer & Welp 2014), the length and depth of diffusion of ICTs such that they are starting to impact structures of power in developing countries (Heeks 2016), and the greater salience of politics within international development and development studies (Hickey 2013). It is reflected in conference
tracks and journal special issues dealing with political perspectives on ICT4D (Andrade & Urquhart 2012a, Heeks et al. 2016) and by calls for more political analysis in ICT4D research and practice (Andrade & Urquhart 2012b, Roberts 2015).

Critical realism, unlike other research paradigms, has particular features that facilitate political research. Political theory often conceives power in terms of underlying structures and mechanisms that shape but do not determine (e.g. Hearn 2012). This is, of course, precisely the ontological perspective of critical realism. It contrasts with the determinism of positivism (and its logical impossibility of denying the politics of research and observation while researching politics) and interpretivism's struggles to recognise social structure or the way in which power constitutes beliefs (Torgerson 1986, Bevir & Rhodes 2005). In addition, the critical aspect of critical realism means it has a central concern with the ways in which power structures society. Hence we find examples of explicitly critical realist research on issues of power and politics (e.g. Patomaki 2002).

## 3.3 Epistemological Value of Critical Realism in ICT4D Research

Critical realism's understanding of the empirical domain allows for different perceptions of common events within an ICT4D project. This legitimises an observed reality of such projects: that different individuals and groups will express different views (Chib et al. 2012); something which positivism at least can labour to encompass. Further, of course, through reference to the mechanisms of the real domain, critical realism allows for an explanation of why those differences occur. Alongside the methodological requirement for triangulation of multiple perspectives, critical realism therefore facilitates use of stakeholder theory in ICT4D research (and use of stakeholder analysis in ICT4D practice); something which has been advocated as a means to provide greater insights into the trajectories of ICT4D projects (Bailur 2006).

## 3.4 Methodological Value of Critical Realism in ICT4D Research

Concerns about lack of rigour in research can be found in every academic discipline. As already discussed, ICT4D research is seen to suffer from a lack of credible investigation into causality (May & Diga 2015), something which undermines internal research validity. Both positivism and critical realism address this but critical realism has an arguable additional value because its iterative retroduction forces ongoing contemplation and critique of the relation between causes and effects. And, as again noted, its contingent approach to causality provides a better reflection of the varied cause-effect patterns seen in ICT4D in practice.

There are also frequent concerns about bias in ICT4D research and the way in which it can undermine both reliability and validity. These include biases of case and respondent selection (Burrell & Toyama 2009), biases of the researchers themselves (Krauss 2012), and biases of individual research methods (Dearden 2013). Interpretivism seeks to address respondent and researcher bias by embracing them as integral to its worldview, but it struggles to deal with the other biases. By contrast, critical realism's methodological pluralism and triangulation force multiple viewpoints and
data sources and methods to be incorporated. More generally, critical realism's mandated reflexivity forces ongoing introspection about the nature of the research process and its overall rigour including biases of context, respondents and researcher. In so-doing, it may be able to improve rigour and mitigate biases. Critical realism also forces reflection on the value of ICT4D research; something, again, that is a concern from those seeking to develop the field (ibid., Krauss & Turpin 2013).

## 3.5 Axiological Value of Critical Realism in ICT4D Research

Some academic disciplines lend themselves to theorisation and abstraction. ICT4D is not such a field: from its very definition it is oriented to practice and it centres around ICT-based interventions in developing countries (Marathe et al. 2016). In a general way, critical realism's concern with engagement and change is supportive of practice-oriented disciplines.

But the role of ICT4D goes beyond the unspecific notion of practice. As the “4D” element directly indicates, it is seeking to achieve progressive social change; seeking to deliver specific development goals. While individual goals vary, foundational orientations of the Sustainable Development Goals are towards transformation: changing underlying systems of development; and towards inclusion: addressing both symptoms and causes of inequality (Heeks 2014a). The congruence of critical realism can therefore be argued given its fit with at least these main goals of development. As discussed above, critical realism is values-driven and orients not only towards understanding how structures and mechanisms constrain development, but also towards interventions that bring about emancipatory change. Dodson et al. (2013:27) argue that the additional concerns of ICT4D, relating to both practice and change, create a problem: “the [ICT4D] research community is not unified on how to harmonize the difficult and sometimes competing goals of conducting experiments, producing social change, and studying the phenomena of ICT use in developing countries”. Critical realism emerges as a basis for this harmonisation given that it encompasses research, practice, and developmental social change.

This is particularly timely because of its relevance to what we might call the “ethical turn” in ICT4D. This has been advanced by the growing use of Amartya Sen’s work – with its focus on justice (Sen 2009) – within ICT4D, and by the growing interest in ethics and social justice within wider development (Oosterlaken 2015). It is reflected by literature on ethics in ICT4D research and practice (Traxler 2012, Mthoko & Pade-Khene 2013). The exact relation of critical realism to ethics and justice is debated (Norrie 2010). At a basic level, the emancipatory impulses of critical realism resonate with the ideas of ethics, and critical realism would thus be supportive of work on ICT4D and ethics. But at a deeper level, one might argue that the foundations of rights, ethics and justice in the ICT4D field lie within the structures of society: that these all derive from and are largely determined by social structures (Heeks & Renken 2016). If that argument is accepted, then critical realism becomes uniquely appropriate for work on ICT4D and ethics given its combined desire to both understand and progressively change the social structures that envelop ICT4D.
## 3.6 Challenges of Using Critical Realism in ICT4D Research

Critical realism faces challenges:

* Philosophical challenges: from a positivist perspective, critical realism is criticised due to its lack of objectivity; the limitations it places on the generalisation of findings; and the "provisional, fallible, incomplete, and extendable" nature of its explanations (Dobson 2009:808). From an interpretive perspective, it is the realism of critical realism that is misguided.

* Practical challenges: critical realism has been criticised as hard to understand and "difficult, time-consuming and resource-intensive" to operationalise (Reed 2009:436).

Well-supported ICT4D doctoral researchers may have the time and training to develop the knowledge and skills necessary to address these challenges, but for most ICT4D researchers, they will present important barriers.

These are issues facing any user of critical realism but we can also reflect on challenges specific to the ICT4D research domain: the lack of ICT4D academic culture and capabilities around critical realism means a lack of both drivers and enablers to greater use of critical realism. In practical terms, for example, there are few supervisors and reviewers who demand use of critical realism, and few supervisors and reviewers who are able to guide and comment on critical realism. This in turn perpetuates the lack of culture and capabilities. It will require express intervention, say of ICT4D journal editors, to encourage greater engagement with research paradigms; developing a more fertile ground for critical realism – among other paradigms – to flourish.

## 4 Conclusions

There is general value of direct engagement with any research paradigm in enhancing ICT4D researchers' analytical capacity and the overall credibility of themselves and our sub-discipline. But our specific interest was one particular paradigm – critical realism – and its potential value in ICT4D.

There is a generic ability of critical realism to address issues seen as concerns for ICT4D research. It engages with underlying structure: helping to expose causal mechanisms and, for example, facilitating use of theoretical frames that connect ICTs to development impact. It encompasses difference: reflecting the contingent and contextualised link between cause and effect seen in ICT4D practice, and legitimising the views of different stakeholders on ICT4D phenomena. It triangulates: reducing the bias of individual ICT4D respondents, researchers or methods. It asks for reflexivity: pressing the ICT4D researcher for deeper insights into their work. And it seeks progressive social change: supporting ICT4D's intervention-orientation and its goal of delivering international development.

We also identified particular value of critical realism to current trends in ICT4D research. It supports the recent and growing search for causality within ICT4D. It
supports the “political turn” in ICT4D: exposing the structures and mechanisms of power that underpin application of ICTs in development contexts, but still allowing space for consideration of human agency. And it supports the “ethical turn” in ICT4D: seeking the outcome of a more just and equitable society, and necessitating investigation of the social structures that underpin rights, ethics and justice.

There are both general and domain-specific challenges to further use of critical realism within ICT4D research; not least the orientation of many ICT4D researchers towards alternative paradigms. But for those who share recognition of critical realism's value there can be a number of enabling actions. These could be critical realism-specific or relate more generally to research paradigms with specific incorporation of critical realism. Examples include: commissioning of special issues and conference tracks; training and development activities for researchers such as workshops; and pressure from conference chairs, editors and reviewers for more explicit incorporation of research paradigms within the methodology sections of papers.

## References

1. Allen, D. K., Brown, A., Karanasios, S. & Norman, A. (2013) How should technology-mediated organizational change be explained?, *MIS Quarterly*, 37(3), 835-854

2. Andersson, A. & Hatakka, M. (2013) What are we doing?, in: *Into the Future*, N. Hayes & R.L. La Rovere (eds), IFIP WG9.4, Ocho Rios, Jamaica, 282-300

3. Andrade, A.D. & Urquhart, C. (2012a) Special issue on the politics of ICT for development, *International Journal of E-Politics*, 3(3), i-iii

4. Andrade, A.D. & Urquhart, C. (2012b) Unveiling the modernity bias: a critical examination of the politics of ICT4D, *Information Technology for Development*, 18(4), 281-292

5. Archer, M.S. (1995) *Realist Social Theory*, Cambridge University Press, Cambridge, UK

6. Bailur, S. (2006) Using stakeholder theory to analyze telecenter projects, *Information Technologies & International Development*, 3(3), 61-80

7. Bevir, M. & Rhodes, R.A. (2005) Interpretation and its others, *Australian Journal of Political Science*, 40(2), 169-187

8. Bhaskar, R. (1975) *A Realist Theory of Science*, Harvester Press, Brighton, UK

9. Bhaskar, R. (1979) *The Possibility of Naturalism*, Harvester Press, Brighton, UK

10. Bhaskar, R. (1989) *Reclaiming Reality*, Verso, London

11. Breuer. A. & Welp, Y. (eds) (2014) *Digital Technologies for Democratic Governance in Latin America*, Routledge, Abingdon, UK

12. Burrell, J. & Toyama, K. (2009) What constitutes good ICTD research?, *Information Technologies & International Development*, 5(3), 82-94

13. Bygstad, B. & Munkvold, B.E. (2011) In search of mechanisms, paper presented at *ICIS2011*, Shanghai, 4-7 Dec

14. Chib, A., Ale, K. & Lim., M.-A. (2012) Multi-stakeholder perspectives influencing policy-research-practice, in *Linking Research to Practice*, A. Chib & R. Harris (eds), IDRC, Otawa, 95-106

15. Cresswell, J.W. (2013) *Qualitative Inquiry and Research Design*, Sage Publications, Thousand Oaks, CA

16. Danermark, B., Ekstrom, M., Jakobsen, L. & Karlsson, J.C. (2002) *Explaining Society: Critical Realism in the Social Sciences*, Routledge, London
17. Dearden, A. (2013) See no evil? Ethics in an interventionist ICTD, *Information Technologies & International Development*, 9(2), 1-17

18. Dobson, P.J. (2001) The philosophy of critical realism: an opportunity for information systems research, *Information Systems Frontiers*, 3(2), 199-210

19. Dobson, P.J. (2009) Critical realism as an underlying philosophy for IS research, in: *Encyclopaedia of Information Science and Technology*, 2nd edn, M. Khosrowpour (ed), IGI Global, Hershey, PA, 805-810

20. Dodson, L.L., Sterling, S. & Bennett, J.K. (2013) Considering failure: eight years of ITID research, *Information Technologies & International Development*, 9(2), 19-34

21. Downward, P. & Mearman, A. (2007) Retroduction as mixed-methods triangulation in economic research, *Cambridge Journal of Economics*, 31(1), 77-99

22. Easton, G. (2010) Critical realism in case study research, *Industrial Marketing Management*, 39(1), 118-128

23. Etherington, K. (2004) *Becoming a Reflexive Researcher*, Jessica Kingsley, London

24. Gallivan, M. & Tao, Y. (2013) A scientometric analysis of research appearing in post-millennial IFIP 9.4 conferences, in: *Into the Future*, N. Hayes & R.L. La Rovere (eds), IFIP WG9.4, Ocho Rios, Jamaica, 144-165

25. Gomez, R. & Day, S.A. (2013) Research questions, paradigms and methods in ICT for development, in: *Into the Future*, N. Hayes & R.L. La Rovere (eds), IFIP WG9.4, Ocho Rios, Jamaica, 301-317

26. Hearn, J. (2012) *Theorizing Power*, Palgrave Macmillan, Basingstoke, UK

27. Heeks, R. (2014a) *From the MDGs to the Post-2015 Agenda*, IDPM Development Informatics Working Paper no.56, University of Manchester, UK

28. Heeks, R. (2014b) *Future Priorities for Development Informatics Research from the Post-2015 Development Agenda*, IDPM Development Informatics Working Paper no.57, University of Manchester, UK

29. Heeks, R. (2016) *Examining “Digital Development”: The Shape of Things to Come?*, GDI Development Informatics Working Paper no.64, University of Manchester, UK

30. Heeks, R., Graham, M. & Ramalingam, B. (2016) “Power, Politics and Digital Development” track, *UK Development Studies Association*, Oxford, UK, 12-14 Sep

31. Heeks, R. & Bailur, S. (2007) Analyzing e-government research: perspectives, philosophies, theories, methods, and practice, *Government Information Quarterly*, 24(2), 243-265

32. Heeks, R. & Renken, J. (2016) Data justice for development: what would it mean?, *Information Development*, prepublished, 8 Nov

33. Henfridsson, O. & Bygstad, B. (2013) The generative mechanisms of digital infrastructure evolution, *MIS Quarterly*, 37(3), 907-931

34. Hickey, S. (2013) *Thinking about the Politics of Inclusive Development*, ESID Working Paper no.1, SEED, University of Manchester, UK

35. Kanellis, P. & Papadopoulos, P. (2009) Conducting information systems research: an epistemological journey, in: *Information Systems Research Methods, Epistemology, and Applications*, A. Cater-Steel & L. Al-Hakim (eds), Information Science Reference, Hershey, PA, 1-34

36. Klein, H.K. (2004) Seeking the new and the critical in critical realism: déjà vu?, *Information and organization*, 14(2), 123-144

37. Krauss, K. (2012) Towards self-emancipation in ICT for development research, *The African Journal of Information Systems*, 4(2), 1

38. Krauss, K. & Turpin, M. (2013) The emancipation of the researcher as part of Information and Communication Technology for Development work in deep rural South Africa, *The Electronic Journal of Information Systems in Developing Countries*, 59(2), 1-21
39. Marathe, M., Chandra, P., Kameswaran, V., Kano, T. & Ahmed, S.I. (2016) In search of missing pieces: a re-examination of trends in ICTD research, in *Proceedings of the 8th International Conference on ICTD*, S. Wyche & J. Srinivasan (eds), ACM, New York, NY

40. May, J. & Diga, K. (2015) Progress towards resolving the measurement link between ICT and poverty reduction, in: *Impact of Information Society Research in the Global South*, A. Chib, J. May & R. Barrantes (eds), Springer, Singapore, 83-104

41. Mingers, J. (2004a) Re-establishing the real: critical realism and information systems, in: *Social Theory and Philosophy for Information Systems*, J. Mingers & L. Willcocks (eds), John Wiley, Chichester, UK, 372-406

42. Mingers, J. (2004b) Real-izing information systems: critical realism as an underpinning philosophy for information systems, *Information and Organization*, 14(2), 87-103

43. Mthoko, H.L. & Pade-Khene, C. (2013) Towards a theoretical framework on ethical practice in ICT4D programmes, *Information Development*, 29(1), 36-53

44. Njihia, J.M. & Merali, Y. (2013) The broader context for ICT4D projects: a morphogenetic analysis, *MIS Quarterly*, 37(3), 881-905

45. Norrie, A. (2010) *Dialectic and Difference*, Routledge, Abingdon, UK

46. Oosterlaken, I. (2015) ICT4D and ethics, in: *The International Encyclopedia of Digital Communication and Society*, Vol 1, R. Mansell & P.H. Ang (eds), Wiley Blackwell, Chichester, UK, 353-361

47. Patomaki, H. (2002) *After International Relations*, Routledge, London

48. Prakash, A. & De, R. (2007) Importance of development context in ICT4D projects, *Information Technology & People*, 20(3), 262-281

49. Ram, M., Edwards, P., Jones, T., Kiselinchev, A. & Muchenje, L. (2014) Getting your hands dirty, *Work, Employment & Society*, 29(3), 462-478

50. Reed, M.I. (2009) Critical realism, in: *The Sage Handbook of Organizational Research*, D.A. Buchanan & A. Bryman (eds), Sage, London, 430-448

51. Roberts, T. (2015) Critical intent & practice in ICT4D, in: *Openness in ICT4D*, P. Nielsen (ed.), Department of Informatics, University of Oslo, Norway, 622-633

52. Sayer, A. (1992) *Method in Social Science*, 2nd edn, Routledge, London

53. Sen, A. (2009) *The Idea of Justice*, Penguin, London

54. Sinha, C. & Hyma, R. (2013) ICTs and social inclusion, in: *Connecting ICTs to Development*, L. Elder, H. Emdon, R. Fuchs & B. Petrazzini (eds), Anthem Press, London, 91-116

55. Smith, M.L. (2005) Reconsidering ICT for development research: critical realism, empowerment, and the limitations of current research, in: *Enhancing Human Resource Development through ICT*, A.O. Bada & A. Okunoye (eds), IFIP WG9.4, Abuja, Nigeria, 30-40

56. Smith, M.L. (2010) Testable theory development for small-n studies, *International Journal of Information Technologies and Systems Approach*, 3(1), 41-56

57. Torgerson, D. (1986) Between knowledge and politics, *Policy Sciences*, 19(1), 33-59

58. Traxler, J. (2012) Ethics and ICTD research, in: *Linking Research to Practice*, A. Chib & R. Harris (eds), IDRC, Ottawa, 68-81

59. Turpin, M. & Alexander, T. (2013) Desperately seeking systems thinking in ICT4D, in: *In-to the Future*, N. Hayes & R.L. La Rovere (eds), IFIP WG9.4, Ocho Rios, Jamaica, 166-178

60. Walsham, G. & Sahay, S. (2006) Research on information systems in developing countries, *Information Technology for Development*, 12(1), 7-24

61. Williams, C.K. & Karahanna, E. (2013) Causal explanation in the coordinating process, *MIS Quarterly*, 37(3), 933-964

62. Zachariadis, M., Scott, S.V. & Barrett, M.I. (2013) Methodological implications of critical realism for mixed-methods research, *MIS Quarterly*, 37(3), 855-879