# Performing Research Validity: A “Mangle of Practice” Approach

Robert B. Johnston, Kai Reimers, Stefan Klein

► To cite this version:

Robert B. Johnston, Kai Reimers, Stefan Klein. Performing Research Validity: A “Mangle of Practice” Approach. Working Conference on Information Systems and Organizations (ISO), Dec 2016, Dublin, Ireland. pp.201-214, 10.1007/978-3-319-49733-4_12 . hal-01619195

HAL Id: hal-01619195

https://inria.hal.science/hal-01619195

Submitted on 19 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Performing Research Validity:
A “Mangle of Practice” Approach

Robert B. Johnston¹, Kai Reimers² and Stefan Klein³

¹University of Sydney and Monash University, Australia
robert.johnston@ucd.ie

²RWTH Aachen University, Aachen Germany
reimers@wi.rwth-aachen.de

³University of Muenster, Muenster, Germany
Stefan.Klein@wi.uni-muenster.de

**Abstract.** Mainstream discussions of research validity (truth, significance, objectivity) draw heavily on a certain “representational idiom” of science [1] that assumes a knowledge–reality correspondence. However, for research on practices, rather than nature, such a knowledge–reality distinction is neither feasible nor desirable, as it is at odds with the very notion of a “practice”. Drawing on Pickering’s alternative “performative idiom” for science, and extending it to participatory forms of social research, we propose alternative validity claims for practice-oriented research. Using the example of information infrastructuring practices, we show that the three aspects of validity thus reinterpreted become quite closely related to each other and also to the process of information infrastructuring itself. In so doing, we demonstrate the importance of extending the notion of “material agency” to embrace the *dual* agencies of the practice studied *and* the researcher’s own disciplinary practice.

**Keywords:** Validity · performativity · truth · objectivity · significance · rigour · relevance · mangle of practice · infrastructuring.

## 1 Introduction

Mainstream discussions of research validity (truth, significance, objectivity) draw heavily on a certain “representational idiom” of science [1] that assumes a knowledge–reality correspondence. Within this representational idiom, notions of research validity come to be formulated as dimensions of this putative knowledge–reality correspondence. In this paper, we ask: what should be the validity claims for practice-oriented research where it is neither feasible nor desired to adopt the assumptions that support the correspondence principle of the representational idiom?

We will argue that to answer this question we must shift the discussion of validity away from definitions that assume an observer–reality split and consider how validity is actually *performed* in the real-time, situated struggle of the researcher to bring forth and understand a phenomenon. We will seek to define validity in terms of attributes of the actual *interaction* between the researcher and the studied world. For this we use
Pickering's performative "mangle of practice" framework [1], extending its application to the case of participatory practice-oriented research.

As a concrete example of practice-oriented research we use an approach of our own design [2] for researching information infrastructuring practices. We will argue validity in this case requires an account of research practice that fully acknowledges the real-time, emergent, social and material nature of *both* the infrastructuring practices studied *and* the researchers own disciplinary practice. We show that the three aspects of validity (truth, significance and objectivity) need to be radically reconsidered: when understood performatively they become quite closely related to each other and also, in a particular sense, to the phenomenon that is studied.

In so doing, we address the theme of this conference by showing how an issue normally framed within an epistemology of separation of knower and known obtains a richer solution when approached from a performative and interactive perspective. Specifically, we show that in the context of strongly participatory social research Pickering's notion of material resistance must be extended, and this extension has novel consequences for claims of research validity in this context. More generally, we reaffirm that Pickering's performative approach, by emphasizing that all scientific knowledge is acquired through ongoing real-time interaction between knower and known, negates any schism between natural and social science on the question of validity.

The flow of the argument is as follows. We begin by describing Pickering's critique of the traditional "representational idiom" for science and his alternative "performative idiom". Using the latter, we present an alternative formulation of validity that is applicable even when a knowledge-world separation is not feasible. This we apply to our example case of information infrastructuring practices, analysing what Pickering's "mangle of practice" would entail in this context and what novel insights a performative analysis of validity yields. We conclude by reflecting on the generality of our arguments.

## 2 The "Representational Idiom" of Science and its Problems

Pickering [1] (p. 5) argues that there is a dominant representational idiom of science that "casts science as, above all, an activity that seeks to represent nature, to produce knowledge that maps, mirrors, or corresponds to how the world really is". Within this representational idiom, notions of research validity come to be formulated as dimensions of this putative knowledge–reality correspondence.

*Truth*: How well does the representation correspond to its real-world object?

*Objectivity*: How independent is the representation of human interests and prejudices?

*Significance*: How broad is the class of real-world phenomena that the representation covers?

The disciplines of Organizational and Management Studies (OMS) and Information Systems (IS) have largely taken over this accepted view of research validity as part of their allegiance to science as a privileged basis for knowledge and as a basis for
disciplinary respectability. This view is so embedded in research culture through the
influential scientific realist philosophy (e.g. [3]), and also through the subject–object
split implicit in everyday folk-theories of knowledge [4], that research more generally
tends to be held accountable against these dimensions of validity [5].<fnref n="1" /> This is the case
even for interpretive and practice-oriented research that does not by and large accept
the ontological and epistemological assumptions underpinning this correspondence
theory of knowledge, leading to complaints by such researchers that their work is often
judged against standards they reject [7] (p. 84).

However, for research of a participatory flavour in these disciplines, because the
object of study is “practices” (of organizing and IT use) it is not so straightforward to
separate knowledge and reality in a way required by the representational idiom.

The first problem is the degree to which knowledge and research object can be
separated in principle. If one takes seriously the notion consistent with the “practice
turn” in OMS and IS [8, 9] that practices are *interpreted* material and social
performances, then what is researched in practice-oriented research includes how the
practitioners themselves make sense of their practices – the very same practices the
researcher is studying. This makes a separation of knowledge and observed reality to
the poles of a correspondence highly problematic.

The second issue is the degree to which researcher knowledge and practitioner
knowledge are entangled in practice. Far from being a passive object of study, the
practitioners studied are members of the broader community that commissions socio-
technical research and are thus potential customers for the knowledge produced.
Knowledge gained and made explicit by the research has the potential to become one
of the drivers of change in the focal practices. This is particularly the case for practice-
oriented research that has strong participatory or ethnographic flavour, such as our
example. Furthermore, this recursive relation between knowledge and its object
introduces an extra normative requirement that knowledge should be (at least
potentially) useful and relevant to the practices studied. This additional validity
requirement of *relevance* is frequently viewed to be in direct conflict with, or
demanding a trade-off with, the standard validity attributes above [10, 11].

The attraction of the representational idiom of science is that it promises to keep the messy human process of knowledge production, as documented in numerous studies of the sociology of scientific knowledge [12, 13, 14, 15], separate from its knowledge “product”. This knowledge product, a representation, then takes on the kind of independence of humans that its object – nature – is assumed in science to have, and validity is sterilized from any trace of the human agency and work that *produced* knowledge in the first place. However, the entanglement of researcher knowledge-seeking practices with practitioner knowledge-using practice outlined above threatens to contaminate the knowledge–reality separation underpinning the standard account of

<footnote n="1">¹ Here we make a claim about what is experienced routinely within the OMS and IS disciplines. We recognize that there are other approaches to truth, such as coherence, pragmatic and consensus theories [6] but the correspondence theory appears to be the working folk-theory of truth in these disciplines.</footnote>
validity. Must we therefore accept that the validity of practice-oriented research can
never match the “gold standard” provided by science?

3 An Alternative Account of Scientific Practice

We will argue that to answer this question we must shift the discussion of validity away
from definitions that assume an observer–reality split and consider how validity is
actually *performed* in the real-time, situated struggle of the researcher to understand
phenomena. We will seek to define validity in terms of attributes of the actual
interaction between the researcher and the observed world. Once it is understood that
*all* scientific knowledge is acquired through such ongoing real-time interaction, any
schism between natural and social science on the question of validity disappears.

For this purpose, we draw on Pickering’s alternate performative account of scientific
practice. Although it was introduced largely in the context of natural science, we will
show that it can be used also to conceptualize validity claims for practice-oriented
research. This is possible because this formulation of scientific practice does not depend
on the usual realist assumption of an independent, separable research object to which
knowledge must correspond, but rather focuses on “how, in practice, connections
between knowledge and the world are made” [1] (p. 182).

3.1 Pickering’s “Performative Idiom” for Scientific Practice

Pickering [1] points out that the representational idiom of science encourages a
particular account of scientific progress as a gradual and inevitable revelation of a
reality “out there”, which he calls the “scientist’s account”. Importantly, the scientist’s
account takes reality to be described by our current understanding and then reconstructs
breakthroughs that led to this understanding as an inevitable triumph of reality over
earlier false theory (see also [13]). He criticizes this account for failing to acknowledge
that scientists (and actors in practices, more generally) actively make unforced choices
and so exercise *agency*. The actual work that scientists do in achieving a successful
capture of natural phenomena through tuning mechanical apparatus and routinizing
operating procedures, in interpreting data, and in revising theories and beliefs, is *deleted
from the “scientist’s account”*. The effort and temporality of this work is replaced –
through hindsight – with a story of unproblematic comparison between reality and
representation in which both humans and machines are largely absent. Thus, nature is
the only “agent” in the scientist’s account.

By contrast, Pickering draws on extensive archival material and interviews on actual
practices of scientists ([1], [16], see also [14]) to show that both choices between rival
theories and evaluation of data from experiment can only be understood as involving
“human agency” – intentional human choices that are conditioned by historical and
cultural context. Thus, Pickering seeks to provide an account of science that makes
central the agentic real-time, performative nature of scientific practice.

The basic assertion is that practices are extended from one moment to the next
through the coming together of human and material agency, a process that he calls “the
mangle of practice”, consisting of alternating episodes of resistance and accommodation. His exemplar case is high-energy particle physics, which employs increasingly complex and large machinery to capture certain physical phenomena and allow comparison with theory. Viewed in the real-time of actual scientific practice, the capture of phenomena is a time-extended struggle between the scientist’s intentions and choices, and recalcitrant machinery. Machinery offers resistance as it produces unexpected or awkward results: the scientist tunes and retunes the machinery and/or develops new interpretations and concepts that would align with the results in an on-going iterated process. To capture the dynamism of this process, Pickering proposes to view the resistance of the machinery as a “material agency” that struggles with the human agency of the scientist. A phenomenon is understood if and when the iterated process of resistance and accommodation converges toward a stabilized agreement between explanations and experimental outcomes. Any such stabilized understanding must be viewed as an outcome (a collaboration) of both human agency and material agency. Such a stabilization is neither guaranteed nor determined by initial human intentions nor material configurations. The process is thus open and emergent: there is no purely mechanical procedure that will inevitably direct the scientist towards “the truth”. Any stabilization is hard won and fragile, at risk of coming apart again through further human or material agency. This idea is depicted in Figure 1.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** The “mangle” according to [1]. Researcher’s intentions encounter resistance from machinery, resulting iteratively in an accommodation of human and material agency.</figcaption></figure>

## 3.2 Validity under the Performative Idiom

Pickering’s performative account of science replaces the notion of a representation–reality correspondence with a dynamic process of achieving a stabilized capture of natural phenomena in machinery (apparatus). Pickering [1] (chapter 6) shows that traditional notions of validity based on correspondence must be revised and linked to the *stabilization* of the dynamic mangling:

*Truth*: Knowledge of a phenomenon is acquired if, and to the extent that, mangling of the researcher’s agency with material agency becomes stabilized. This is not a correspondence of the scientist’s abstract theoretical account with an external reality, but an *agreement* between a scientist’s hard-won understanding of a natural phenomenon and how it is manifested in the performance of actual machinery. This then is a performative claim for truth.

*Objectivity*: As joint mangling approaches stabilization the researcher’s original intentions get “mangled” by the resistance put up by machinery. The process of
mangling thus enacts a performative kind of objectivity: human agency develops and evolves as a series of responses to the agency of a material “other”, rather than from independently willed moves that might force initial human plans, interests and prejudices upon the outcome. The resulting stabilization might depend on the historical and cultural context of human agency but no longer on individual initial desires, intentions or free-will of the researcher. This then is a performative claim for objectivity.

*Significance*: Pickering did not explicitly consider research significance. It is reasonable, though, to associate significance with some measure of the breadth of the phenomenon that is captured in the stabilization. This associates significance with universality of the phenomenon studied, rather than generality of a theoretical assertion. We will return to a performative claim for significance later, in the more concrete setting of our example.<fnref n="2" />

## 3.3 Moving Beyond Natural Science: Generalizing Material Agency

Our argument is that these alternate conceptions of validity developed by Pickering for the case of natural science can also be applied to forms of social science where close engagement of researchers with practitioners is involved.

Firstly, these performative formulations of truth, objectivity and significance make no reference to a putative independent “real world”. They therefore do not depend on a commitment to a realist ontology or an epistemology of separation between knower and known. They require only a genuine iterative interaction (mangling) of researcher agency and the resistive agency of the “machinery” that captures a natural phenomenon. Secondly, even though they were derived for the case where the phenomenon is captured in the machinic performance of experimental apparatus, they do not depend on this setting. Therefore, we can apply this analysis even when the phenomena investigated are social in nature and are captured in human social performance (human practices) where a realist separation of knowledge and world is untenable. Thirdly, the performative idiom captures how knowledge is actually accumulated, refined and tested in the face of, and actually *as a result of*, the resistance offered by the world to the researcher’s knowledge-seeking intentions. This means that strong involvement of researchers with their object of study, characteristic of engaged forms of practitioner-oriented research (including our example), does not preclude strong claims for research validity in social science.

However, any application of the mangle of practice beyond its natural science origins requires the notion of material agency to be generalized beyond the resistive agency of apparatus. Some extensions have already been considered by Pickering [1] (see chapters 4 and 5) and in the next section we will extend the idea to the resistive agency of socio-material practices. First we will describe the general principle involved and then Pickering’s own initial extensions.

<footnote n="2">² In the language of dynamic systems theory [17], truth is an attractor of the dynamic mangling process, objectivity is the insensitivity to initial conditions of the attractor, and significance is the breadth of the basin of attraction.</footnote>
It needs to be stressed that, even in the original experimental science setting, Pickering's notion of material agency is not referring to autonomous or cybernetic behaviour on the part of physical mechanisms. Rather, material agency is a product of the researcher-centric view he takes: seen from the experimentalist's situated real-time view, an apparatus that mediates access to nature presents itself as a combative "agent" enacting resistive "moves" in response to the researcher's intentional actions to validate proposed theory and concepts. The source of this agency is not any intrinsic intentionality of physical artefacts, but the researcher's situated involvement in the world.<fnref n="3" /> Accordingly, Pickering has already extended the concept to cover forms of resistance other than from physical apparatus [1].

The first extension is to resistance that is not primarily from the research object but rather from culturally entrenched notions of proper procedure within the scientist's own disciplinary practices, which he terms "disciplinary agency". The second extension ("cyborg agency") deals with resistance from a socio-technical assemblage (in his case, of numerically controlled machines and their human operators). In each case, it is shown that the "mangle" framework of alternating episodes of human accommodation to resistive agency, progressing towards stabilisation, adequately describes the case dynamics.

We can think of these two extensions as elaborating nuances of a broader concept of "materiality" defined as "that which resists human intentional agency" and which manifests as a "material agency" from the situated, performative view of an actor in a practice. It is this generalised notion of material agency that we will employ when we use the mangle of practice notion to discuss validity for our example case.

# 4 Example: Researching Information Infrastructuring

In this section we will illustrate how the performative analysis of research validity can be applied in a research project that would raise validity concerns under the representational understanding of science. Our example is a novel research intervention we have pioneered to study the process of industry-wide information infrastructure creation.

## 4.1 Information Infrastructuring

The study of information systems as information infrastructures is a relatively novel concern among OMS and IS scholars [20, 21]. Increasingly, scholars speak of "infrastructuring" rather than "infrastructure development" to emphasize the processual character of infrastructure creation [22, 23, 24, 25]. Our own work [2], [26] has followed this processual turn. We view information infrastructuring as the ongoing becoming of infrastructure as diverse interacting practices become increasingly aligned in their pursuit of a shared enterprise, through on-going sense-making that may disclose

<footnote n="3">³ Thus he does not see material agency as totally symmetric with human agency, as do some "post-humanists" [18, 19].</footnote>
new shared uses of information technology. This approach asserts that extension and deepening of the understanding of the possibilities for coordination among diverse practices drives an uptake of technology, rather than design and adoption of technological artefacts driving greater coordination.

To study infrastructuring we have devised and employed [2] a novel research intervention called a Learning Community. In a Learning Community a diverse group of interested practitioners within an industry meet regularly over an extended period to engage in mutual and expansive learning about each other's practices.<fnref n="4" /> The intention is that through such interaction, facilitated by the researcher, participants will come to understand the practices they necessarily depend on for coordinated activity, and consequently gain and diffuse back into their practice a deeper understanding of their own practice – including the potential of information technology as equipment for their practice. According to our hypothesis, this deepening and expanding understanding of interacting practices constitutes the process of infrastructuring. By facilitating and participating in the process, the researcher is well placed to gain an understanding of the infrastructuring process as it occurs. However, this understanding does not take the form of distanced observation: an important aspect of the researcher's role in the Learning Community is to discover by trial and error how to cultivate and nurture the community in such a way that the expansive learning that constitutes infrastructuring actually takes place. Thus in a very real sense, the researcher participates with the practitioners in the very infrastructuring process as it occurs. However, where the practitioners are motivated to enhance knowledge within their own practice by better understanding the practices they need to coordinate with, the researcher's motive is to increase scientific knowledge of the infrastructuring process being facilitated.

While this form of participatory practice research holds out the promise of authentic access to its research “object”, it is also open to typical scientific critiques of its validity, implicitly derived from the representational idiom of science. For example, since infrastructuring is an enacted process rather than an external, existent object how can any account of it claim to be true? Since the researcher sacrifices distance to participate in the process being studied how can an account claim to be objective? If a particular instance is studied how can the account claim to have any level of significance?

## 4.2 Applying the Mangle to Research on Infrastructuring

To apply the mangle of practice framework here we note that the researcher engages in a time-extended, two-way interaction with the Learning Community, comprising workshops, interviews and other forms of conversation and dialogue. The researcher will put up for review by the Learning Community a certain understanding of information infrastructuring that informs the researcher's interactions at a given time, which may encounter resistance in the form of objections or other negative reactions of

<footnote n="4">⁴ In this paper we will not present results of this research. Here we are using the particular arrangement of this research intervention as an example to which to apply the performative analysis of validity.</footnote>
the participants. Thus, just as in all other cases of mangling described by Pickering, the researcher will have to iteratively modify and accommodate intentions, understandings, procedures and instruments to a form of “resistance” until an agreement between researcher understanding and the reactions of the Learning Community is achieved. But it is important to recognise that this resistance derives from the understanding that each participant has by virtue of that participant’s enrolment in their own particular practice, for instance, understandings within the practice of how work must be done, what are worthy aims and what is appropriate equipment use. Thus we style this kind of resistance as a “practice resistance”, which is another form of generalised material resistance and appears to the situated researcher as a material “practice agency”, *not to be confused with the human agency of any particular participant*. In fact, this resistance has the same nature and source as the resistance that participants face in their own attempts to adjust their own understanding of adjacent practices in the course of their exposure to these practice in the Learning Community.

This process is quite analogous to how the physical scientist refines his/her understanding of a natural phenomenon by adjusting to how a machine that captures that phenomenon – their apparatus – responds to active “interrogation”. In the present example, the phenomenon of infrastructuring is captured in the expanding understanding of the community of interacting practices. So again we can conceptualise research in this example as a time-extended struggle between the researcher’s “human agency” and the resistive “practice agency” of the Learning Community as apparatus. This situation is depicted in Figure 2.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Researcher's intentions encounter resistance from information infrastructuring practices, resulting iteratively in an accommodation of human and practice agency.</figcaption></figure>

However, the researcher not only faces resistance from the information infrastructuring practices, but also from his/her own research practice. The researcher is expected not only to apply “rigorous” scientific methods in his/her work, but also to extend scientific knowledge by doing sanctioned forms of research. Thus, the researcher must present to the research community new theoretical, empirical or methodological insights derived from the study of information infrastructuring in practice. These may challenge existing forms of knowledge and sanctioned methods, and will thus rouse “disciplinary resistance”. The researcher is therefore necessarily involved in an additional mangling with disciplinary agency that must be stabilized if the researcher wants to validly claim to have extended disciplinary knowledge of information infrastructuring. This situation is depicted in Figure 3.
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Research accounts of information infrastructuring practices are an accommodation of human agency with both practice agency and research disciplinary agency.</figcaption></figure>

These two forms of resistance interact with each other since some of the researcher's
explanations that have been successfully aligned with the information infrastructuring
practices may be resisted by the research practice and vice versa. The barriers to
successful accommodation to these two intertwined forms of resistance are therefore
quite high. In what follows we will demonstrate that such a call for dual accommodation
is justified and, indeed, necessary for science genuinely to do its job in the realm of
social phenomena.

4.3 Performative Validity Claims for Research on Information
Infrastructuring

We now apply and extend Pickering's analysis of validity within the performative
idiom of science to our example case of researching information infrastructuring
practices.

Truth: An understanding of information infrastructuring is acquired if, and to the extent that, mangling of the researcher's agency with *both* the agency of the information infrastructuring practices studied *and* disciplinary agency of the researchers own research practice (Fig. 3) becomes stabilized. Such an understanding will be a hard-won, stabilized but fragile, agreement between researcher explanations and practitioner understandings. The researcher will have repeatedly and stubbornly presented their explanations of “what is going on” to the practitioners *and* within their own discipline to solicit responses. In this process, the researcher will have modified some explanations, dropped others and discovered alternative ones until *both* practice resistance and disciplinary resistance become sufficiently muted to assume that an accommodation has been achieved. The agreement is *fragile* because multiple practices are involved and articulation of that understanding can change the practices themselves. The agreement is *hard-won* because it resulted from the actual human work and acceptance of risk characterized by the mutual mangling process. The researcher will have put on the line their very identity as a social scientist, welcomed practitioners to challenge the researcher's grasp of the nature of their practices, and also accepted the challenge to defend that understanding to peers.
If stabilization is achieved, it can be said that the researcher has “true” knowledge of the information infrastructuring process. It is worth digging deeper into the character of such a “truth claim”. First, it cannot be a claim to have uncovered “mechanisms” that operate “behind the backs” of the actors in the practices, which are only recognized because of the special theoretical sensitivity afforded to the researcher by the research discipline. But equally, it is not a claim for special local practical knowledge by a researcher who has “gone native” and obtained tacit knowledge beyond the explicit reach of the discipline. Rather, it is a claim that an extended interaction between multiple parties and practices has created the *ground* upon which an explicit and truthful account of the infrastructuring process could be articulated, agreed upon and shared. On that ground of effortful and risky interaction, a knowledge–reality correspondence has been *performed* rather than passively *observed*.

*Objectivity:* According to Pickering’s analysis, a performative interpretation of objectivity refers to a kind of “washing out” of individual interests, pet theories and normative orientations through the gradual and on-going mangling of human and material agency. Applied to research on information infrastructuring, the focal and the research disciplinary practices will have put up dual resistance to the researcher’s intentions and will have forced the researcher to defend or modify theoretical concepts and methodological approaches to the phenomenon through several iterations. As this iterated process goes on, increasingly ideas and concepts will have become detached from their origins to the extent that the researcher can no longer claim to “own” them: nor can practitioners completely. They are now the “public property” of the collaboration. Consequently, these ideas and concepts can no longer be traced to particular interests, motives and preferences that researcher or practitioners might harbour. This process of *freeing* knowledge from individual intentionality is enhanced by the phenomenon studied that involves multiple practices and sources of resistance.

At the same time this new kind of “objectivity claim” can no longer serve to justify normative proposals to “reengineer” the practices to overcome problems, discovered by the researcher through objective causal analysis, that are not apparent to the practitioners themselves. However, it does support a different kind of normativity because the understanding so obtained includes and embeds the vision towards which the practices themselves are already heading and working as part of the becoming of the information infrastructure. Any advice that the researcher (or any practitioner) gives has *already become* at once both relevant and scientific, in a way that a privileged “interventionist” normativity cannot be.

*Significance:* Above we argued that significance, within a performative idiom, should be associated with the breadth of the interaction that becomes stabilised, rather than with the generality of some theoretical assertion. Such broadening can only be achieved through further mangling. For research on information infrastructuring, two possibilities might be entertained. In the first, styled on the traditional notion of generality, researchers might attempt to involve more practices in their efforts. However, such efforts would quickly become limited by material means and the difficulty of aligning ever more practices. The second is a possibility only revealed within the performative approach. An agreement already achieved with a particular infrastructuring process may be broadened through the mangling that occurs with
research disciplinary agency, resulting in changes of understanding of information
infrastructuring processes within the research practice itself. Other researchers,
drawing upon this changed disciplinary understanding, may then encounter changed
resistance when mangling their own understandings with further independent instances
of infrastructuring practice, and so on, in an iterated form of Figure 3. Consequently,
the multiple independent information infrastructuring processes studied by independent
researchers will become indirectly involved in a broader mangling process. To the
extent that the various understandings brought to this larger mangling process converge
and achieve stabilization, the resulting alignment of understandings becomes broader
and thus more significant.

Two things should be noted about the character of this performative significance
claim. First, this catalytic form of broadened scope depends on the researcher
submitting to both forms of resistance depicted in Figure 3. Second, this spreading and
alignment of understanding within diverse infrastructures extends the reach of “public
knowledge” among the practices, and is therefore *the very process of information
infrastructuring itself*! Thus, the significance of research on information
infrastructuring is measured by the extent to which the researcher ultimately
*contributes* to the phenomenon studied.

5 Discussion and Conclusion

We began this paper by asking “what should be the validity claims (truth, significance, objectivity) for practice-oriented research, where it is neither feasible nor desired to adopt the assumptions that support the correspondence principle of the representational idiom?”. Noting that Pickering’s performative analysis of validity, developed largely in the context of natural science, does not require any commitment to scientific realism, we applied this analysis to our own context of research on information infrastructuring practices.

The answers we arrive at are both interesting and challenging. The three aspects of validity become quite closely related to each other and also, in a particular sense, to the phenomenon that is studied. A research claim is *true* if a genuine interaction between the researcher and practitioner has created the ground for an understanding shared between their practices; the claim is *objective* if it is not subservient to any party’s particular intentions and interests and becomes “public property”; and it is *significant* if it has contributed to extending the reach of the information infrastructuring process. In a deep sense, these are also the requirements of the kind of shared understanding that constitutes the process of information infrastructuring itself. Thus for our example, performing valid research on information infrastructuring is largely indistinguishable from its practice object.

These specific conclusions about validity ultimately derive from our analysis of the *dual resistance* faced by a researcher from both the focal research practices and their own disciplinary practice. This dual resistance was not anticipated in Pickering’s discussions of the mangle of practice because his cases (experimental, conceptual and sociotechnical) all still focussed on a researcher/actor studying a world they were not
actually part of. However, the context of genuine participatory research motivated by
the desire to create widely shared public practices by means of infrastructure, brings to
the fore the need for an account of science as an activity of studying a world, or rather
disclosing a new world [4], that researchers and their research subjects are equally part
of. This requires a more symmetrical treatment of scientific and practitioner knowledge
production in which valid research is performed by meeting the challenge of this dual
resistance and accepting the risk this entails. We argue this is the basis of genuinely
engaged and relevant scholarship.

Thus, it is arguable that our conclusions also apply more widely, to a large class of
social scientific research where the issue is contributing to public knowledge and
broader practice. A reason for believing this is that all practices must of necessity
coordinate with other intersecting practices and so all practice knowledge development
is essentially infrastructuring. However, defining the class of research interventions to
which our specific conclusions apply is for future work.

More broadly, the approach described here can potentially be applied to any research
setting to determine specific conditions for research validity. The great advantage of
this framework is that it does not require “special pleading” on the part of social science
for validity, because the performative idiom is derived from, and already captures, the
actual processes of knowledge creation in the natural sciences that are usually held to
be the “gold standard” of rigour. An important implication is that research rigour need
not be exclusively associated with observational distance. The issue is not for the
researcher to retreat to the “high ground” of detached observation of the research object,
but to create through effortful interaction with the research object a “middle ground”
where a desire to account for the world and the world’s resistance to being accounted
for, can collaborate in performing rigour.

We also note a close relationship of our arguments to Gadamer's [27] conception of
"fusion of horizons" (between information infrastructuring practices and the research
practice) and to Heidegger's [28] notion of truth as disclosure (aletheia) as the
condition for the possibility of any knowledge-object correspondence. We leave these
theoretical threads to be explored in future work.

References

1. Pickering, A.: The Mangle of Practice. University of Chicago Press, Chicago, IL (1995).

2. Reimers, K., Johnston, R.B., Guo, X., Klein, S., Xie, B., Li, M.: Novice-based Data Collection Methods for the Study of IOIS: Practice Probes and Learning Communities. Electronic Markets. 23, 285-293 (2013)

3. Weber, R.: Evaluating and Developing Theories in the Information Systems Discipline. Journal of the AIS. 13, 1-30 (2012)

4. Spinosa, C., Flores, F., Dreyfus, H.L.: Disclosing New Worlds: Entrepreneurship, Democratic Action, and the Cultivation of Solidarity. MIT Press, Cambridge, MA (1997)

5. Bryman, A., Bell, E.: Business Research Methods. Oxford University Press, Oxford (2011)

6. Mingers, J.: Realising Systems Thinking: Knowledge and Action in Management Science. Springer Verlag, New York (2006)
7. Klein, H.K., Myers, M.D.: A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems. MIS Quarterly. 23, 67–88 (1999)

8. Orlikowski, W.J.: Using Technology and Constituting Structures: A Practice Lens for Studying Technology in Organizations. Org. Sci. 11, 404–428 (2000)

9. Schatzki, T.R., Knorr Cetina, K.D., Savigny, E.: The Practice Turn in Contemporary Theory. Routledge, London (2001)

10. Schön, D.: Knowing-in-Action: The New Scholarship Requires a New Epistemology. Change. Nov/Dec, 27–34 (1995)

11. Dubé, L., Paré, G.: Rigor in Information Systems Positivist Case Research: Current Practices, Trends, and Recommendations. MIS Quarterly. 27, 597–635 (2003)

12. Fleck, L.: Genesis and Development of a Scientific Fact. University of Chicago Press, Chicago (1937)

13. Latour, B., Woolgar, S.: Laboratory Life: The Construction of Scientific Facts. Princeton University Press, Princeton (1986)

14. Galison, P.: How Experiments End. University of Chicago Press, Chicago (1987)

15. Knorr Cetina, K.D.: Objectual Practice. In: Schatzki, T.R., Knorr Cetina, K.D., Savigny, E. (eds) The Practice Turn in Contemporary Theory, pp. 184–197. Routledge, London (2001)

16. Pickering, A.: Constructing Quarks: A Sociological History of Particle Physics. Chicago University Press, Chicago (1984)

17. Gleick, J.: Chaos: Making a New Science. Viking Press (Penguin), New York (1987)

18. Collon, M.: The Sociology of an Actor-Network: The Case of the Electric Vehicle. In: Callon, M., Law, J., Rip, A. (eds) Mapping the Dynamics of Science and Technology: Sociology of Science in the Real World. MacMillan Press, London (1986)

19. Barad, K.: Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Duke University Press, Durham, NC (2007)

20. Tilson, D., Lyytinen, K., Sorensen, C.: Digital Infrastructures: The Missing IS Research Agenda. Inform. Sys. Res. 21, 748–759 (1996)

21. Bowker, G.C., Baker, K., Millerand, F., Ribes, D.: Toward Information Infrastructure Studies: Ways of Knowing in a Networked Environment. In: Hunsinger, J., Klastrup, L., Allen, M. (eds) International Handbook of Internet Research, pp. 97–117. Springer, Dordrecht (2010)

22. Star, S.L., Ruhleder, K.: Steps Toward an Ecology of Infrastructure: Designs and Access for Large Information Spces. Inform. Sys. Res. 7, 111–134 (1996)

23. Star, S.L., Bowker, G.C.: How to Infrastructure. In: Lievrouw, L.A., Livingstone, S. (eds) Handbook of New Media: Social Shaping and Consequences of ICTs, pp. 230–245. Sage, London (2002)

24. Pipek, V., Wulf, V.: Infrastructuring: Toward an Integrated Perspective on the Design and Use of Information Technology. Journal of the AIS. 10, 447–473 (2009)

25. Aanestad, M., Mukherjee, A., Jolliffe, B., Sahay, S.: Infrastructuring Work: Building a State-Wide Hospital Information Infrastructure in India. Inform. Sys. Res. 25, 834–845 (2014)

26. Reimers, K., Johnston, R.B., Klein, S.: An Empirical Evaluation of Existing IS Change Theories for the Case of IOIS Evolution. Euro. J. Info. Sys. 23, 373–399 (2014)

27. Gadamer, H.-G.: Truth and Method. Continuum International Publishing, New York (1975)

28. Heidegger, M.: Being and Time. Harper & Row, New York (1961)