Thoughts on Relevance of IS Research

The discussion of whether research is relevant should include attention to a frame of reference for the meaning of relevance. This paper argues that three elements belong in such a frame. One is a clear identification of what interests or stakes are involved in the research, and whose interests are served. Another element is whether the research is intended to produce knowledge for its own sake or is intended to solve a particular practical problem. Lastly, different time frames can lead to completely different judgements about relevance and value. This complexity suggests considerable caution should apply to evaluating the relevance of IS or any other body of research. Relevance is far too slippery a concept to just be tossed about in a way that's disconnected from a frame of reference. So, to be useful, the discussion of whether any particular body of research is relevant must include some careful attention to that frame. This principle was reflected in the ISWORLD discussion of relevance in good measure, especially in the discussion thread that dealt with one important element of such a frame, i.e., the matter of "relevance to whom?" As I read that thread, however, it seemed to me that some important distinctions, which are part of such a frame, were getting a bit muddled. One rather critical distinction is between relevance to and serves the interests of or is of value to. So I’d like to address these comments to that distinction. The link between relevance and interests can be complicated. To say that research is relevant to some party does not make clear whether the research has positive, neutral, or negative implications for that party's interests. Research that leads to improved security in wireless networking, for example, is relevant to both developers seeking to protect wireless systems and hackers seeking to invade them. Much of the discussion in the thread, by contrast, seemed to equate research’s relevance to some business or industry with serves the interests of. The latter is an adequate touchstone of relevance only from the point of view of that business or industry or those that have shared interests. Many of us doing research in an academic/university environment may be quite unconcerned with relevance to business or industry when relevance means serves the interests of. My current work in inter-organizational information sharing is a ready and apt example. I hope the results of this work will be useful and therefore relevant to many kinds of organizations such as businesses, government agencies, and non-profits. I believe it has the potential to be so. But we did not choose this work to serve the interests of business. We are not employed by a business nor beholden in any way directly to the private sector. The work is supported by NSF, so is intended above all to advance knowledge generally, with practical contributions being secondary. For this particular work, the primary touchstone of relevance is the acceptance by the research community, as evidenced by publication of the results in peer-reviewed outlets. Establishing relevance or value through peer review is a foundation of NSF operations, applied to both 38 Communications of the Association for Information Systems (Volume 6, 2001) 37-39 Thoughts on Relevance of IS Research by A. M. Cresswell awarding research grants and evaluating results. This policy is natural for an organization whose mission is to promote scientific research. As academic researchers we share the same interests, since the reward system in this environment is based largely on scientific publishing and getting grants. Thus the sponsor’s goals or interests are well aligned with ours. That establishes the major touchstone of relevance for this work. He who pays the piper may call the tune, but this particular chorus would be singing the same tune anyway. However, that does not settle the issue of relevance to the larger society. Linking relevance to this particular piper’s tune (NSF), raises another problem: to what should scientific inquiry be relevant? The mission of the NSF contains a bit of a contradiction in this regard. The mission statement is, "To promote the progress of science; to advance the national health, prosperity, and welfare; and to secure the national defense." This mission combines the unqualified pursuit of science with a distinctly practical agenda. The statement embodies the conflict between valuing science for its own sake and valuing it in terms of tangible benefits to the nation. I have neither the ability nor the desire to settle that debate here. But it is important to point out that judgments about research relevance do hinge in part on this conflict. From the "science for its own sake" position I can legitimately claim that if my research is relevant to other’s doing similar work (at whatever level of abstraction), that is enough. Producing practical outcomes is not necessary. Others could just as legitimately claim that my work is irrelevant without such outcomes; science should serve the public good, preferably in direct, tangible, and immediate ways. One or the other of these value propositions is an essential part of the relevance frame. Then there’s the matter of who isspeaking for the public good and thereby establishing what is relevant. We, as academic or university-based researchers, would not consider the views of those who speak for the interests of the business community to be a sufficient basis for determining the relevance of our work. An observer working for a corporation and speaking for the interests of that corporation could easily say our work is irrelevant. However a great deal of what business does could be considered irrelevant to the good of the larger society; what's good for General Motors is not necessarily what's good for the country. So part of the issue of relevance should be linked to the age-old political question, "Qui bono?" One Senator, William Proxmire (D., Wisconsin), got great political mileage out of making himself an arbiter of the relevance and value of scientific research. He created what was called "The Golden Fleece Award" in 1975, and continued to confer it until 1988. Each month he would identify a Federal activity, quite often an NSF-funded research project, which sounded ridiculous or patently wasteful. The resulting press coverage would highlight him as a champion of cutting waste in government spending. NSF projects would be described in terms that made them sound frivolous or silly. There was typically no attempt to identify any scientific merit for these projects or provide the researchers an opportunity for rebuttal. In these cases, the Senator was the clear beneficiary. [GoldenFleece, 2001] But having a political interest or financial stake does not necessarily disqualify anyone from judging relevance either. The simple fact that a claim of relevance or irrelevance arises in selfinterest is not sufficient to dismiss it. As researchers we should look carefully at any charge of irrelevance, whether it comes from the business, political, or scientific communities. But looking carefully is not the same as simply accepting the validity of the charge. It could turn out that we were seen as irrelevant because our work misses or misconstrues some important facts or principles about inter-organizational interactions. That is, our work could be flawed or invalid in a scientific sense, even if the flaw was detected by someone in a business setting. It would therefore be incumbent on us to make appropriate adjustments in the work and acknowledge the relevance of the evaluation irrespective of its source. Or our review could also lead us to the conclusion that the work is sound, but seen as irrelevant to others simply because it does not serve the observer’s interests. In that case we would simply reject the charge of irrelevance as irrelevant. The key here is whether that evaluation is independent of the interests or influence of the observer. Communications of the Association for Information Systems (Volume 6, 2001) 37-39 39 Thoughts on Relevance of IS Research by A. M. Cresswell One important additional element of a frame of reference for relevance should be time. The accumulation of scientific knowledge and experience gained from its application to problems have a way of greatly affecting our notions of what’s relevant, and to whom. What is pure science to one generation can become a whole new industry to the next, or a major theoretical breakthrough, or some fruitless backwater of limited or no interest to anyone. Predicting how any particular line of research will be valued over the long term is very risky business. It seems this same assessment of risk could apply to the larger question of assessing relevance. Working through this and the other discussions on relevance has made me wary. I think it will be with much more caution and reflection that I apply "relevant" or "irrelevant" to any particular research. And that’s not a bad thing.