# Critical Realist Information Systems Research in Action

Sven A. Carlsson

► To cite this version:

Sven A. Carlsson. Critical Realist Information Systems Research in Action. 1st Working Conference Researching the Future (Future-IS), Jun 2011, Turku, Finland. pp.269-284, 10.1007/978-3-642-21364-9_17. hal-01567993

HAL Id: hal-01567993

https://inria.hal.science/hal-01567993

Submitted on 24 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Critical Realist Information Systems Research in Action

Sven A. Carlsson

Informatics, School of Economics and Management, Lund University,
Ole Römers väg 6, 223 63 Lund, Sweden sven.carlsson@ics.lu.se

**Abstract.** There is a growing interest in critical realism (CR) in Information Systems (IS) research. This paper presents and discusses how critical realism can be an alternative philosophical underpinning for IS research. It briefly presents critical realism and how it can be used in IS research. Contemporary examples of how CR have been used in IS research are presented and discussed. The future use of CR in IS research is also discussed.

**Keywords:** Critical realism, empirical research, theory development, design science research, theory testing, evaluation research

## 1 Introduction

There is a growing interest in critical realism (CR) in Information Systems (IS) research. Scholars have pointed out major weaknesses and limitations in positivism and constructivism, see, for example, Archer et al. (1998) and López and Potter (2001). CR is an alternative to the different strands of positivism and constructivism currently dominating IS research. CR argues that social reality is not simply composed of agents' meanings, but that there exist structural factors influencing agents' lived experiences. CR starts from an ontology that identifies structures and mechanisms, through which events and discourses are generated, as being fundamental to the constitution of our natural and social reality.

This paper presents and discusses how critical realism can be an alternative philosophical underpinning for IS research. It briefly presents critical realism and how it can be used in IS research. Contemporary examples of how CR have been used in IS research are presented and discussed. The future use of CR in IS research is also discussed.

## 2 Critical Realism

Different philosophies of science have different ontological views. Idealists have the view that reality is not mind-independent. Idealism comes in different forms,
reflecting different views on what is man-created and how it is created. Realists have the view that reality exists independently of our beliefs, thoughts, perceptions, discourses, etc. As for idealism, realism comes in different forms. Bhaskar says that it is not a question of being a realist or not, but what type of realist (Bhaskar 2000).

Critical realism was developed as an alternative to traditional positivist models of social science as well as an alternative to constructivism. The most influential writer on CR is Roy Bhaskar. Good summaries of CR are available in Archer et al. (1998), Dean et al. (2005), Sayer (2000), and Chapter 1 in Bhaskar (2002); key concepts and main developments are presented in Hartwig (2007). In Archer et al. (1998) and López and Potter (2001), chapters focus on different aspects of CR, ranging from fundamental philosophical discussions to how statistical analysis can be used in CR-based research.

CR was primarily developed as an answer to the positivist crisis. In 1975, Bhaskar's work *A Realist Theory of Science*, with "transcendental realism," was published. In *Possibility of Naturalism* (1998), Bhaskar focused on the social sciences and developed "critical naturalism." These two major works present a thorough philosophy of science project and later critical realism and critical naturalism were merged to "critical realism," a concept also used by Bhaskar. Through the 1980s, Bhaskar primarily developed his position through sharpening arguments. The late 1970s and early 1980s saw a number of other CR scholars publishing influential works, for example, Margaret Archer's *Social Origins of Educational Systems* (1979) and Andrew Sayer's *Method in Social Science* (1992). Most early critiques of CR targeted positivism, but later critique is targeting alternatives to positivism, for example, constructivism and structuration theory.

Critical realism can be seen as a specific form of realism:

> To be a realist is to assert the existence of some disputed kind of entities such as gravitons, equilibria, utility, class relations and so on. To be a scientific realist is to assert that these entities exist independently of our investigation of them. Such entities, *contra* the post modernism of rhetoricians, are not something generated in the discourse used in their investigation. Neither are such entities, *contra* empiricists, restricted to the realm of the observable. To be a *critical* realist is to extend these views into social science (Fleetwood 2002, p. 29).

The manifesto of CR is to recognize the reality of the natural order and the events and discourses of the social world. It holds that

> we will only be able to understand—and so change—the social world if we identify the structures at work that generate those events or discourses.... These structures are not spontaneously apparent in the observable pattern of
events; they can only be identified through the practical and theoretical work of the social sciences (Bhaskar 1989, pp. 2-3).

Bhaskar (1978) outlines what he calls three domains: the *real*, the *actual*, and the *empirical* (see Table 1).

<table><caption>**Table 1.** Ontological Assumptions of the Critical Realist View of Science (from Bhaskar 1978. X's indicate the domain of reality in which mechanisms, events, and experiences, respectively reside, as well as the domains involved for such a residence to be possible).</caption><thead><tr><th></th><th>Domain of Real</th><th>Domain of Actual</th><th>Domain of Empirical</th></tr></thead><tbody><tr><th>Mechanism</th><td>X</td><td></td><td></td></tr><tr><th>Events</th><td>X</td><td>X</td><td></td></tr><tr><th>Experiences</th><td>X</td><td>X</td><td>X</td></tr></tbody></table>

The *real domain* consists of underlying structures, mechanisms, and relations; events and behavior; and experiences. The generative mechanisms residing in the real domain exist independently of, but capable of producing, patterns of events. Relations generate behaviors in the social world. The *domain of the actual* consists of these events and behaviors. Hence, the actual domain is the domain in which observed events or observed patterns of events occur. The *domain of the empirical* consists of what we experience; hence, it is the domain of experienced events.

Bhaskar (1978, p. 13) argues that

real structures exist independently of and are often out of phase with the actual patterns of events. Indeed it is only because of the latter we need to perform experiments and only because of the former that we can make sense of our performances of them. Similarly it can be shown to be a condition of the intelligibility of perception that events occur independently of experiences. And experiences are often (epistemically speaking) "out of phase" with events—e.g., when they are misidentified. It is partly because of this possibility that the scientist needs a scientific education or training. Thus I [Bhaskar] will argue that what I call the domains of the real, the actual and the empirical are distinct.

CR also argues that the real world is ontologically stratified and differentiated. The real world consists of a plurality of structures and generative mechanisms that generate the events that occur and do not occur. From an epistemological stance, concerning the nature of knowledge claim, the realist approach is non-positivistic, which means that values and facts are intertwined and hard to disentangle.
Critical realism is a well-developed philosophy of science, but on the method-
ological level, it is less well-developed. The writings of Kazi (2003), Layder (1993,
1998), Pawson (2006), Pawson and Tilley (1997), and Robson (2002), as well as some
of the chapters in Ackroyd and Fleetwood (2000) and Fleetwood and Ackroyd (2004),
can serve as guidelines for doing critical realist research. Unfortunately, from an IS
perspective, the writings on CR have not focused on artifacts like IS/IT and it has been
only in the behavioral science paradigm.

CR has influenced a number of disciplines and fields, for example, economics, management, and organization studies. There is a growing interest in CR in the IS field. Scholars have argued for the use of CR in IS research (Carlson 2004; Dobson 2001; Mingers 2004; Mutch 2002). CR-based empirical research can be found in Dobson et al. (2007), and DeVaujany (2008), Morton (2010), Strong and Volkoff (2010)m and Volkoff et al. (2007). CR has also critical and emancipatory compo- nents (Bhaskar 2000). Wilson and Greenhill (2004) and Smith (2005) address how CR in IS research can work critically and be emancipatory. CR's potential for IS design science research has been argued by Carlsson (2010) and Lyytinen (2008). CR-based IS design science can be found in Clarson et al. (2011) and Hrastinski et al. (2010). A sign of the growing interest and importance of CR-based research is a recent call for paper for an *MIS Quarterly* special issue on "Critical Realism in Infor- mation Systems Research." Having briefly presented CR, we will now focus on how it has been used in IS research and discuss strengths and weaknesses in the presented studies.

3 Using Critical Realism in IS Research

Gregor (2006) argues that five interrelated types of IS theories can be distinguished: (1) theory for analyzing, (2) theory for explaining, (3) theory for predicting, (4) theory for explaining and predicting, and (5) theory for design and action. The five types can be clustered into two main types: traditional natural/social research (first four types) and design science research (fifth type). Next, we examine how CR can be used in both main types of research. Theory developing studies (3.1), evaluation research (3.2), and design science research (3.3) are presented.

3.1 CR-Based Development of IS Theories

Two theory-generating IS studies are briefly presented. The first is the development of a CR-based theory of technology-enabled organizational change. The second study develops an explanation of the causes of the outcomes of attempts to develop and implement strategic IS plans in organizations.
Volkoff et al. (2007) addressed the classical concern: How does technology mediate organizational change? They did this by developing a new theory based on the philosophy of CR. They longitudinally studied the implementation and use of an enterprise resource planning (ERP) system. Their specific view on technology-mediated organizational change is based on Archer's (1995) morphogenetic approach, which is conceptualized as a cycle consisting of three phases: (1) structural conditioning, (2) social interaction, and (3) structural elaboration. The "concept *morphogenesis* indicates that society has no preferred form...but is shaped and re-shaped by the interplay between STRUCTURE and AGENCY" (Archer 2007, p. 319). The middle phase of social interaction is the one which may appear to be where human agency has its greatest role, but this is not the case, as human agency is implicated in and embedded within all phases of the cycle. The structural conditioning phase incorporates the critical realist assumption that structure predates action(s) which transforms it. Structural elaboration postdates those actions. The structural elaboration phase of the model, which flows out of the social interactions in phase two can have one of two characteristics: structural elaboration/morphogenesis, where people and structures are transformed, and structural reproduction/morphostasis, where people and structures are largely reproduced. As Archer states, "Although, all three lines [phases] are in fact continuous, the analytical element consists in breaking up the flows into intervals determined by the problem in hand" (2007, p. 319).

Volkoff et al. (2007, p. 835) view ERP as

a source of structural conditioning that is relatively independent and enduring, existing materially in the real domain, rather than primarily as a malleable structure, existing only in the empirical domain at the moment of instantiation. Using a critical realist perspective, we can discuss the interplay between structures and human agents, and examine the generative mechanisms or mediators through which agents affect structure and, of greater importance for this study, how structures shape agency.

Volkoff et al. based their study on Pentland and Feldman's (2005) work on organizational routines and especially their distinction between ostensive (structural) aspects of routines and the performative (agentic) aspects of routines. Volkoff et al. found that "in addition to ostensive and performative aspects, routines also have a material aspect that is embedded in the technology....[and] material aspect of routines plays a critical and direct role in the change process" (p. 833). Their study resulted in a new technology-mediated organizational change theory.

The study by Volkoff et al. illustrates how their theory overcomes some of the problems with previous theories. Previous theories can be categorized as having a deterministic view or an emergent view. They can also be classified as having an
agency view or a structural view (including a technology view). In line with CR, Volkoff et al.'s study and theory focuses both on structure and agency and has an emergent view. It separates structure and agency, whereas some other theories either ignore agency (i.e., theories based on institutional theory), ignore structure (i.e., theories based on behavioral approaches), or conflate the two (i.e., theories based on structuration theory and ANT). A similar study is Strong and Volkoff (2010); it is a CR-based longitudinal study of an ERP system focusing on organization-ERP system fit. Strong and Volkoff and Volkoff et al. use a qualitative approach and their analyses are based on grounded theory (their coding process generally followed a grounded theory approach). It should be noted that retroduction, which is adopted by critical realism and refers to the inference from a description of some phenomenon to a description of something that produces it or is a condition for it, is not used or mentioned.

The second CR-based theory generating study by Morton (2010) addresses the research question: What are the causes of the outcomes of attempts to develop and implement strategic IS plans in organizations? Achieving the vision of the strategic IS plan is difficult and Morton's literature review shows that both private and public sector organizations having used SISP had problematic experiences. This motivates the research question for his thesis.

Morton argues that CR is useful for establishing causal explanations of phenomenon where closed system conditions (controlled experiments) are difficult to apply, such as in organizational settings where complex interactions occur between people and technology and outcomes are not predictable. The methodology used by Morton is consistent with CR and he uses a multiple case study research design.

Morton reviewed the SISP literature and an immanent critique was undertaken to establish a conceptualization of SISP consistent with the open systems view of social reality set out by CR. An immanent critique is an examination of existing theories about the phenomenon of interest using CR as a meta-theory to critique the theories' ontological presuppositions (Hesketh et al. 2006). The two dominating models of SISP, the comprehensive, formal, and systematic model and the incremental, informal, and opportunistic model, were the targets of the immanent critique. The immanent critique finds significant theory-practice inconsistencies in relation to both the comprehensive and informal models of SISP. On the basis of the immanent critique, a new reconceptualization of the phenomenon was offered: SISP as an intervention into the open systems social reality of the organization with unpredictable outcomes. In this view, SISP is itself a collection of causal mechanisms that activate other causal mechanisms in the setting which originate in the agency of participants, the prevailing social structures, and the use by agents of a range of ideational elements from the cultural system. Morton used the reconceptualization as a framework to analyze each of four case studies to identify a set of causal mechanisms responsible for the outcomes of SISP projects contingent on organizational conditions.
Morton's research design follows Robson's (2002) four-part structure for CR-based research comprising (1) purpose, (2) methods, (3) theory, and (4) sampling strategy. Morton's aim was not to test specific propositions or hypotheses about SISP; rather it was to generate theory, by a process of reintroducing the form and constitution of causal mechanisms, that could have produced and hence explain the empirical effects in the case studies. The theorization of causal mechanisms was supported by reference to the literature for corroboration of their existence. Four case studies, drawn from organizations either developing or implementing a strategic IS plan, were used as the primary source of empirical data for. A multiple case study design allows for the corroboration of findings across case studies via a comparative approach (Bergene 2007), although the effects of causal mechanism are always subject to contingent conditions. Four methods of data collection were used: (1) participant observation (case one), (2) non-participant observation (case two), (3) interviewing (all cases), and (4) document review (all cases).

Comparative analysis of the 4 case studies revealed a set of 13 causal mechanisms that had a significant impact on the outcomes in each case and 9 contingent factors that affected their activation and interaction. The mechanisms complete the CR-based reconceptualization of SISP and explain the varied outcomes of the SISP projects by showing how they result from the contingent interaction of these mechanisms.

Morton's study is primarily a qualitative study. Morton's work is interesting because of his use of immanent critique (Hesketh et al. 2006) and his use of Bergene's (2007) comparative approach. The use of immanent critique can be viewed as one way to address that CR rejects judgmental relativism (the inability to judge the merits of theories). An alternative approach would have been to use the three theories on the four cases and compared how they were performing in terms of describing and explaining the phenomenon that are the causes of the outcomes of attempts to develop and implement SISP in organizations.

## 3.2 CR-Based Information Systems Evaluation Research

IS evaluation and IS evaluation research have been stressed as critical means in advancing the IS field. Generally, IS evaluation is concerned with the evaluation of different aspects of real-life interventions in the social life where information systems are critical means in achieving the anticipated goals of the intervention. IS evaluation research can be considered a special case of evaluation research.

Driving CR-based IS evaluation research, which here is called *realistic IS evaluation*, is the aim to produce ever more detailed answers to the question of why an IS initiative-IS, types of IS, or IS implementation-works for whom, in what circumstances, and why. This means that evaluation researchers attend to how and why an IS initiative has the potential to cause (desired) changes.
A realistic evaluation researcher works as an experimental scientist, but not according to the logic of the traditional experimental research. According to Bhaskar (1978, p. 53),

> The experimental scientist must perform two essential functions in an experiment. First, he must trigger the mechanism under study to ensure that it is active; and secondly, he must prevent any interference with the operation of the mechanism. These activities could be designated as "experimental production" and "experimental control."

Realistic evaluation researchers do not conceive that IS initiatives “work.” It is the action of stakeholders that makes them work, and the causal potential of an IS initiative takes the form of providing reasons and resources to enable different stakeholders and participants to “make” changes. This means that a realistic evaluation researcher seeks to understand why an IS initiative works through an understanding of the action mechanisms. It also means that a realistic evaluation researcher seeks to understand for whom and in what circumstances (contexts) an IS initiative works through the study of contextual conditioning.

Realistic evaluation researchers orient their thinking to context-mechanism-outcome pattern configurations—called CMO configurations (Pawson and Tilley 1997). In IS evaluation research, a CMO configuration is a proposition stating what it is about an IS initiative (IS implementation) that works for whom in what circumstances (Carlsson 2003). A refined CMO configuration is the finding of IS evaluation research—the output of a realistic evaluation study.

Realistic IS evaluation based on the above may be implemented through a realistic effectiveness cycle. The starting point is theory. Theory includes propositions on how the mechanisms introduced by an IS invention into pre-existing contexts can generate outcomes. This entails theoretical analysis of mechanisms, contexts, and expected outcomes. The second step consists of generating hypotheses. Typically the following questions would be addressed in the hypotheses: (1) What changes or outcomes will be brought about by an IS intervention? (2) What contexts impinge on this? (3) What mechanisms (social, cultural, and others) would enable these changes, and which one may disable the intervention? The third step is the selection of appropriate data collection methods. In this step, it might be possible to provide evidence of the ability of the IS intervention to change reality. Based on the result from the third step, one may return to the program (the IS intervention) to make it more specific as an intervention of practice. Next, but not finally, one returns to theory. The theory may be developed, the hypotheses refined, the data collection methods enhanced, etc.

To illustrate realistic IS evaluation, two studies are presented. The first is an evaluation study of enterprise systems implementations. The second is a study of the impact of a computer-based IS on clinical practice.
The first study focused automated performance measurement systems (APMS). APMS are a fairly recent evolution within the context of enterprise information systems. APMS deliver information (values for key performance indicators) to senior managers through automatically collecting operational data from integrated information systems. APMS are a consequence of the Sarbanes-Oxley Act (SOX) and similar legislation. The study investigated a number of APMS implementations with varying degree of success. In reflecting on their research, Dobson et al. (2007, p. 143) say,

> In the context of the APMS research, it became evident that contextual issues were paramount in explaining the success and failure of the implementations....This emphasis on context impacted the underlying research focus. The critical realist focus on retroductive prepositional-type questioning led to a contextual basis for the study seeking to answer "Under what conditions might APMS implementation prove successful?" rather than "What are the (predictive) critical success factors for an APMS implementation?" A simplistic critical success factors approach tends to deny the heavy contextuality and complexity of large-scale systems implementation.

Dobson et al.'s study had six major phases. The first phase was a literature review. The purpose of the review was to develop an APMS success model. The literature review followed Pawson's (2006) suggestions on how to conduct systematic reviews to make sense of a heterogeneous body of literature. The review was driven by a focus on CMO and how outcomes can be "produced." Using Pawson's approach means that it is possible to move away from the many one-off studies and instead learn from fields such as medicine and policy studies on how to develop evidence-based IS knowledge and theory. In order to guide practitioners, researchers should analyze previous research based on the assumption that one can draw more powerful conclusions from the collective wisdom of previous research.

In the second phase, the developed success model was used as the basis for generating questions for semi-structured, qualitative interviews. In the third phase, the generated questions were used in a focus group interview. The focus group was composed of IS industry experts, active in the performance measurement system area. The generated data and the first developed model were analyzed and a revised model (Model 1) was developed. In the fourth phase, this model was be tested against a case study with further refinements to the model resulting in Model 2. In the fifth phase, a number of reviews and case interviews were done. These led to the refinements of the model (Model 3 and 4). A final model was synthesized and presented by Myles (2008).

Dobson et al.'s study illustrates the realistic IS evaluation research well. The study has a focus on unpacking the mechanisms of how complex IS implementations
work in particular settings and contexts. The focus is on what is it about this IS
implementation that works for whom in what circumstances and why. With the
study's emphasis on deep understanding of contexts, settings, and mechanisms, it
shows how realistic IS evaluation research can be a very good alternative to simplistic
critical success factors studies.

It is also possible to use a quantitative approach for CR-based IS evaluation.
Oroviogoicoechea's (2007) study is an example of a primarily quantitative CR-based
evaluation study (see also Oroviogoicoechea and Watson 2009). The aim of
Oroviogoicoechea's study was to explore nurses' perceptions of the impact on clinical
practice of the use of a hospital information system. Computer-based information
systems are increasingly being introduced in clinical practice.

Oroviogoicoechea used a realistic IS evaluation design based on Pawson and Tilley 1997) and Carlsson (2003). It was used across all phases of the study. Oroviogoicoechea used a questionnaire containing both closed and open-ended questions. Descriptive statistics were used for an overall overview of nurses' perceptions. Inferential analysis, including both bivariate and multivariate methods (path analysis), was used for cross-tabulation of variables searching for CMO relationships. Content analysis of open-ended questions was used to identify major themes in nurses' responses. Overall satisfaction with the IS was positive. Mechanisms and outcomes are highly correlated. Comparisons with context variables show how users' characteristics, with the exception of attitude toward the introduction of technology, did not have a significant influence on perceptions while the nursing unit context had greater influence. Path analysis illustrated that the influence of unit context variables are on outcomes and not on mechanisms. Six main themes emerged from open-ended questions. Some differences in relation to the unit context were observed.

The two studies are different in that one is primarily a qualitative study and the
other is primarily a quantitative study. Although most CR-based IS evaluation
studies are qualitative studies, the two studies exemplify how CR is committed
methodological pluralism. At the same time it should be noted that few studies have
really taken advantage of CR's methodological pluralism.

Both studies are driven by the equation context + mechanism = outcome. As pointed out by Houston (2010), time is also relevant in shaping outcome and human agency certainly can play a critical role in shaping outcome. Hence, the extended equation is context + mechanism + time + human agency = outcome. Future IS evaluation studies could explore the use of the extended equation as the driver for IS evaluation research.

3.3 CR-Based Information Systems Design Science

This section addresses Gregor's (2006) fifth type of theory: theory for design and
action. Research generating this type of theory is solution-driven and called IS
design science research. Two major IS design science research schools have emerged (El Sawy 2006): (1) information systems design theory (see Walls et al. 2004), and (2) design science research (Hevner et al. 2004). The schools share a focus on the IT artifact. Some scholars argue for a third “school” based on IS as a socio-technical discipline and that

design science and the research that builds that body of knowledge must
acknowledge that IS is fundamentally about human activity systems which
are usually technologically enabled, implying that *the context of design and*
*use* is critical, and that research paradigms, practices and activities must
embrace such a worldview (McKay and Marshall 2005, p. 5).

A number of projects have been developed in the past several years on IS design knowledge for “IS use and management” and an approach developed for socio-technical IS design science research (STISD) (Carlsson et al. 2011; Hrastinski et al. 2010). The approach is underpinned by CR and has four major research activities (Figure 1): (1) identify problem situations and desired outcomes, (2) review extant theories, knowledge, and data, (3) propose/refine design theory and knowledge, and (4) test design theory and knowledge. The figure reveals that IS design science research is not only about doing or designing. An important part of this research approach is to continuously test design theories.

<figure><img src="image_2.png" /><figcaption>**Figure 1.** IS design knowledge development (from Carlsson et al. 2011; Hrastinski et al. 2010)</figcaption></figure>

**Research Activity: Identify Problem Situations and Desired Outcomes.**

Design theories and design knowledge aim to support solving practical problems in such a way that desired outcomes are reached. Hence, such theories and knowledge are goal- and outcome-oriented, which means that, when used, they should increase the likelihood of reaching desired outcomes.
**Research Activity: Review Extant Theories, Knowledge, and Data.** Design theories and design knowledge should be enhanced through grounding in previous research. A design theory should be enhanced by continuously interacting with what is currently known, that is, grounding in extant theories, knowledge, and data. Gregor (2006) argues that other types of theories can inform design theory and that design theory and explanatory and predictive theories are strongly interrelated. Van Aken (2005) maintains that design knowledge in the form of design propositions can be developed through cross-case analyses of previous case studies (see also Carlsson et al. 2003; Gregor 2009; and Hrastinski et al. 2010). This means that design knowledge is abstracted from cases. Van Aken refers (2004) to this as "extracting case studies" and shows how it has led to a number of useful and actionable design propositions, for example, Kanban systems and just-in-time.

In general, design theories and design knowledge can be enhanced through systematic reviews of previous research—see, for example, Pawson’s (2006) CR-based approach for conducting systematic reviews. Such reviews should be driven by PIMCO configurations and should have a specific focus on outcome(s) and how outcome(s) can be produced or enhanced (in the next activity, PIMCO configurations are explained). Using this approach for review of relevant literature means that it is possible to move away from the many one-off studies and instead learn from fields such as medicine and policy studies on how to develop evidence-informed IS design knowledge.

**Research Activity: Propose/Refine Design Theory.** Researchers orient their thinking to problem situation (P), IS initiative (I), mechanisms (M), context (C), outcome (O) pattern configurations (PIMCO configurations). A PIMCO configuration is a proposition stating what it is about an IS initiative that works for whom in what circumstances. When proposing a design theory, for example, in the form of design propositions, it is important to provide thick descriptions to aid the reader in understanding the theory, which may support practitioners in translating a theory to specific contexts and situations (van Aken 2005). A design proposition can be expressed as: In problem situation (P) and context (C), to achieve outcome (O), then design and implement IS initiative (I). The "design and implement IS initiative I" includes three different types of designs: (1) object-design, (2) realization design, and (3) a process design.

A field-tested and grounded design proposition has been tested empirically and is grounded in science. The latter means primarily grounding in results and theories from the behavioral science paradigm. Field-tested and grounded design propositions will in most cases be in the form of heuristics. This is consistent with critical realism’s view on causality and means that the indeterminate nature of a heuristic design proposition makes it impossible to prove its effects conclusively, but it can be tested in context, which in turn can lead to sufficient supporting evidence.

**Research Activity: Test Design Theory.** After having formulated an initial design theory, the next step is empirical tests, which include the selection of appropriate data collection methods (Carlsson 2010; Carlson et al. 2011). In doing this, it
can be examined whether the design theory may be used as support when trying to change reality. Based on the results, the outcome may be reflected on and the design theory may be refined. Through multiple studies, one can accumulate supporting evidence iteratively and continuously move toward evidence saturation. To strengthen the validity of design theories, test triangulation may be beneficial (i.e., to combine two or more complementing ways of conducting tests, such as focus groups and field experiments).

The approach has been used in a number of projects for developing socio-technical design theory and knowledge, for example, for the development of a design theory for synchronous e-learning (Hrastinski et al. 2010) and a design theory for IS integration management in mergers and acquisitions (Carlsson et al. 2011; Henningsson and Carlsson forthcoming).

# 4 Discussion

Above, some CR-based IS studies have been presented. Most of CR-based IS studies identified have been done as “traditional” behavioral science research and there is a domination of qualitative theory generation studies and qualitative evaluation studies. Given this picture, a few areas where CR-based research can move the IS-field forward and how CR-based studies can be improved are pointed out.

As noted above, CR rejects judgmental relativism (the inability to judge the merits of theories). This means that it should be possible to compare different theories. This will also open up for possibilities for theory testing. Currently, there are few, if any, CR-based theories testing IS studies. Miller and Tsang (2010) present a four-step approach for theory testing. Their approach is based on CR and provides guidance for deploying established research methods to test theories. Their approach is worthwhile to explore in CR-based IS research where the aim is to test theories—in behavioral science as well as in design science research. Their approach calls for the use of a diversity of research methods. I agree with their call for using a diversity of research methods in CR-based studies, not only for testing theories but also in research aiming at generating theories (see also Downward and Mearman 2007; Easton 2010; Zachariadis et al. 2010). Miller and Tsang's approach would, in most cases, require collaboration between researchers trained in different fields. In terms of comparing theories, Mole and Mole (2010) compare the use of Gidden's (1984) structuration theory and Archers (1995) morphogenesis theory in the entrepreneurship field and conclude that a CR perspective, like Archer's morphogenesis, may be more appropriate. Similar types of studies are needed in the IS field.

There is also a need for researchers using CR to be more cautious in their use of methods and techniques, for example, some studies use induction or deduction instead of critical realism's retroduction.
Most CR-based IS studies focus only one or two of Layder's (1993, 1998) domains of social life (self, situated activity, setting, and context). Future CR-based IS studies could address a larger number of domains and also incorporate that the domains are interlocking and mutually dependent on each other—on this issue, see, Houston (2010).

CR-based studies could also have critical and emancipatory aims. Bhaskar does not view the role of the social scientist as one that is value free and he views "facts" as having moral implications. This means that there is certainly a need for CR-based studies aimed at discovering mechanisms that led to harmful effects. If researchers find such mechanisms, then there is an obligation to expose the mechanisms and facilitate the conditions or contexts whereby emancipatory mechanisms can be activated. As Houston (2010, p. 76), states, "It is only by understanding the deep causes of oppression that we can develop ways of dismantling it."

## 5 Conclusion

Although CR has influenced a number of disciplines and fields, there has until recently been very little CR-based research. CR's potential for IS research has been argued by a number of scholars. In this paper, we have seen that CR can be useful as an underpinning philosophy for behavioral IS research as well for IS design science research. The examples presented in the paper can function as exemplars for how to do different types of IS research underpinned by CR. It should be noted that doing CR-based IS research is not without problems. For example, due to its open system view and that it recognizes social systems' complexity, CR-based IS research will generate theories that are provisional, fallible, incomplete, and extendable. In other words, CR-based IS research will not produce simple theories and "quick fix" results. Some weaknesses in current CR-based IS research have also been pointed out and ways to move forward suggested.

## References

* Ackroyd, S., and Fleetwood, S. (eds.): Realist Perspectives on Management and Organisations. London: Routledge (2000)

* Archer, M.S.: Social Origins of Educational Systems. London: Sage (1979)

* Archer, M.S.: Realist Social Theory: The Morpohogenetic Approach. Cambridge, UK: Cambridge University Press (1995)

* Archer, M.S.: Morphogenesis/Morphostatis. In: M. Hartwig (ed.), Dictionary of Critical Realism. London: Routledge, 319 (2007)

* Archer, M., Bhaskar, R., Collier, A., Lawson, T., and Norrie, A. (eds.): Critical Realism: Essential Readings. London: Routledge (1998)
Bhaskar, R.: A Realist Theory of Science. Sussex, England: Harvester Press (1978)

Bhaskar, R.: Reclaiming Reality. London: Verso (1989)

Bhaskar, R.: Philosophy and the Idea of Freedom. Oxford, UK: Basil Blackwell (1991)

Bhaskar, R.: The Possibility of Naturalism, 3rd ed. London: Routledge (1998)

Bhaskar, R.: Reflections on Meta-Reality. London: Sage (2002)

Bergene, A.C.: Towards a Critical Realist Comparative Methodology: Context-Sensitive Theoretical Comparison. J. of Critical Realism, 6, 1, 5–27 (2007)

Carlsson, S.A.: Advancing Information Systems Evaluation (Research): A Critical Realist Approach. Electronic J. of Information Systems Evaluation, 6, 2, 11–20 (2003)

Carlsson, S.A.: Using Critical Realism in IS Research. In: M. E. Whitman and A. B. Woszczynski (eds.), The Handbook of Information Systems Research. Hershey, PA: Idea Group Publishing, 323–338 (2004)

Carlsson, S.A.: Design Science Research in Information Systems: A Critical Realist Approach. In: A. Hevner and S. Chatterjee, Design Research in Information Systems. New York: Springer, 209–233 (2010)

Carlsson, S.A., Henningsson, S., Hrastinski, S., and Keller, C.: Socio-technical IS Design Science Research: Developing Design Theory for IS Integration Management. Information Systems and e-Business Management, 9, 1, 109–131 (2011)

Dean, K., Joseph J., and Norrie, A.: Editorial: New Essays in Critical Realism. New Formations, 56, 7–26 (2005)

De Vaujany F.-C.: Capturing Reflexivity Modes in IS: A Critical Realist Approach. Information and Organization, 18, 51–71 (2008)

Dobson, P.J.: The Philosophy of Critical Realism—An Opportunity for Information Systems Research. Information Systems Frontier, 3, 2, 199–201 (2001)

Dobson, P., Myles, J., Jackson, P.: Making the Case for Critical Realism: Examining the Implementation of Automated Performance Management Systems. Information Resources Management Journal, 20, 2, 138–152 (2007)

Downward, P., and Mearman, A.: Retroduction as Mixed-methods Triangulation in Economic Research: Reorienting Economics into Social Science. Cambridge Journal of Economics, 31, 77–99 (2007)

Easton, G.: Critical Realism in Case Study Research. Industrial Marketing Management, 39, 118–128 (2010)

El Sawy, O.A.: Personal communication, August (2006)

Fleetwood, S.: Boylan and O'Gorman's Causal Holism: A Critical Realist Evaluation. Cambridge Journal of Economics, 26, 27–45 (2002)

Fleetwood, S., and Ackroyd, S. (eds.): Critical Realist Applications in Organization and Management Studies. London: Routledge (2004)

Giddens, A.: The Constitution of Society. Cambridge, UK: Polity Press (1984)

Gregor, S.: The Nature of Theory in Information Systems. MIS Quarterly, 30, 3, 611–642 (2006)

Gregor, S.: Building Theory in the Sciences of the Artificial. In: DESRIST'09, May 7–8, Malvern, PA (2009)

Hartwig, M. (ed.): Dictionary of Critical Realism. London: Routledge (2007)

Henningsson, S., and Carlsson, S.A.: The DySIIM Model for Managing IS Integration in Mergers & Acquisitions. Information Systems Journal (forthcoming)

Hesketh, A., and Fleetwood, S.: Beyond Measuring the Human Resources Management Organizational Performance Link: Applying Critical Realist Meta-Theory. Organization, 13, 5, 677–699 (2006)
Hevner, A. R., March, S. T., Park, J., and Ram, S.: Design Science in Information Systems Research. MIS Quarterly, 28, 1, 75–105 (2004)

Hrastinski, S., Keller C., and Carlsson, S.A.: Design Exemplars for Synchronous e-Learning: A Design Theory Approach. Computers & Education, 55, 652–662 (2010)

Houston, S.: Prising Open the Black Box: Critical Realism, Action Research and Social Work. Qualitative Social Work, 9, 1, 73–91 (2010)

Kazi, M.A.F.: Realist Evaluation in Practice. London: Sage (2003)

Layder, D.: New Strategies in Social Research. Cambridge, UK: Polity Press (1993)

Layder, D.: Sociological Practice: Linking Theory and Social Research. London: Sage (1998)

López, J., and Potter, G. (eds.): After Postmodernism: An Introduction to Critical Realism. London: Athlone (2001)

Lyytinen, K.: Design: Shaping in the Wild. Keynote speech at the Third International Conference on Design Science Research in Information Systems & Technology (DESRIST 2008), May 7-9, Atlanta, GA (2008)

McKay, J., Marshall, P.: A Review of Design Science in Information Systems. In: Proceedings of the 16th Australasian Conference on Information Systems, Sydney, November 29-December 2 (2005)

Miller, K.D., Tsang, E.W.K.: Testing Management Theories: Critical Realist Philosophy and Research Methods. Strategic Management J., 32, 139–158 (2010)

Mingers, J.: Re-establishing the Real: Critical Realism and Information Systems. In: J. Mingers and L. Willcocks (eds.), Social Theory and Philosophy for Information Systems. Chichester, England: Wiley, 372–406 (2004)

Mole, K.F., and Mole, M.: Entrepreneurship as the Structuration of Individual and Opportunity: A Response Using Critical Realism. J. of Business Venturing, 25, 230–237 (2010)

Morton P.A.: Explaining Outcomes of Strategic Information Systems Planning Using a Critical Realist Approach. Unpublished Ph.D. Thesis, School of Business IT and Logistics, RMIT University, Melbourne (2010)

Mutch, A.: Actors and Networks or Agents and Structures: Towards a Realist View of Information Systems. Organizations, 9, 3, 477–496 (2002)

Myles, J.F.: Discovering Critical Success Factors for Implementing an Automated Measurement System: A Case Study Approach. Unpublished Ph.D. Thesis, Faculty of Business and Law, School of Management, Edith Cowan University (2008)

Oroviogoicoechea, C.: A Realistic Evaluation of the Impact of a Computerized Information System on Clinical Practice: The Nurses' Perspective. Unpublished Ph.D. Thesis, School of Nursing and Midwifery, University of Sheffield (2007)

Oroviogoicoechea, C., and Watson, R.: A Quantitative Analysis of the Impact of a Computerized Information System on Nurses' Clinical Practice Using a Realistic Evaluation Framework. International J. of Medical Informatics, 78, 839–849 (2009)

Pawson, R.: Evidence-Based Policy: A Realist Perspective. London: Sage (2006)

Pawson, R., and Tilley, N.: Realistic Evaluation. London: Sage (1997)

Pentland, B. T., and Feldman, M. S.: Organizational Routines as a Unit of Analysis. Industrial and Corporate Change, 14, 793–815 (2005)

Robson, C.: Real World Research, 2nd ed. Oxford, UK: Blackwell (2002)

Sayer, A.: Method in Social Science: A Realist Approach, 2nd ed. London: Routledge (1992)

Sayer, A.: Realism and Social Science. London: Sage (2000)
Smith, M.L.: Overcoming Theory-Practice Inconsistencies: Critical Realism and Information Systems Research. Working Paper 134, Department of Information Systems, London School of Economics (2005)

Strong, D.M., Volkoff, O.: Understanding Organization-Enterprise System Fit: A Path to Theorizing the Information Technology Artifact. MIS Quarterly, 34, 4, 731-756 (2010)

van Aken, J.E.: Management Research as a Design Science: Articulating the Research Products of Mode 2 Knowledge Production in Management. British J. of Management, 16, 1, 19-36 (2005)

van Aken, J.E.: Management Research Based on the Paradigm of Design Sciences: The Quest for Field-tested and Grounded Technological Rules. J. of Management Studies, 41, 2, 219-246 (2004)

Volkoff, O., Strong, D.M., and Elmes, M.B.: Technological Embeddedness and Organizational Change. Organization Science, 18, 5, 832-848 (2007)

Walls, J.G., Widmeyer, G.R., and El Sawy, O.A.: Building an Information Systems Design Theory for Vigilant EIS. Information Systems Research, 3, 1, 36-59 (1992)

Wilson, M., and Greenhill, A.: Theory and Action for Emancipation: Elements of a Critical Realist Approach. In: B. Kaplan, D. Truex III, D. Wastell, T. Wood-Harper, and J. DeGross (eds.), Information Systems Research: Relevant Theory and Informed Practice. Amsterdam: Kluwer, 667-675 (2004)

Zachariadis, M., Scott, S., and Barrett, M.: Designing Mixed-Method Research Inspired by a Critical Realism Philosophy: A Tale from the Field of IS Innovation. In: ICIS Proceedings, paper 265 (2010)

## About the Author

**Sven Carlsson** is Professor of Informatics at Lund University School of Economics and Management (LUSEM). His current research interests include the use of IS to support management processes, knowledge management, enterprise systems, techno-change, enterprise 2.0, and the use of social media in business processes. He has published more than 125 peer-reviewed journal articles, book chapters, and conference papers and his work has appeared in journals such as *Journal of Management Information Systems*, *Decision Sciences*, *Information Systems Journal*, *Information & Management*, *Journal of Decision Systems*, *International Journal of Technology Management*, *Knowledge and Process Management*, *Knowledge Management Research & Practice*, *Information Systems and e-Business Management*, and *Scandinavian Journal of Information Systems*. Sven has been a visiting scholar/professor at University of Arizona, Tucson, National University of Singapore, University College Cork, University of Southern California, Monash University, Melbourne, and Università della Calabria, Rende. He is a regional editor for *Knowledge Management Research & Practice*.