# Methods for Studying the Information Systems Future

Paul Gray, Anat Hovav

## ► To cite this version:

Paul Gray, Anat Hovav. Methods for Studying the Information Systems Future. 1st Working Conference Researching the Future (Future-IS), Jun 2011, Turku, Finland. pp.299-316, 10.1007/978-3-642-21364-9_21. hal-01567986

HAL Id: hal-01567986

https://inria.hal.science/hal-01567986

Submitted on 24 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Methods for Studying the Information Systems Future

Paul Grayª and Anat Hovavᵇ

ªClaremont Graduate University, School of Information Science and Technology,
Claremont, CA paulgray@cgu.edu

ᵇKorea University, Business School, Seoul, Korea anatzh@korea.ac.kr

**Abstract:** Information Systems studies involve the present and the future. Although data can be obtained on the present, the future can only be estimated. In Information Systems, a rapidly changing intellectual and technological field, simply extrapolating the present to the future is likely to be wrong. This paper presents and explains three proven research methods for studying the IS future and describes the role of Futures Research methods as a tool for academic research.

**Keywords:** Modeling the future, environmental scanning, Delphi, scenarios, role of futures in academic research

## 1 Introduction

A research method is a strategy of inquiry which moves from the underlying philosophical assumptions to research design and data collection.

Michael Myers (1977)

Historically, the study of the human and organizational aspects¹ of management information systems (MIS) adopted the methodologies of social science. This view of research permeated IS as early as the Minnesota experiments (Dickson et al. 1977). The majority of work published at the time used surveys or quasi-experiments. Over time, the field recognized the need for a “pluralism of methodologies” (Robey 1996; Walsham 1995). At the IFIP 8.2 working conference in 1997, Markus pointed out that qualitative research methods were becoming acceptable in MIS research. These methods, including case studies, action, ethnographic, and grounded research, are now

¹Although IS research is primarily concerned with systems, people, and the policies involved in managing IS, those aspects change as the technology changes. Hence IS research also needs to consider the impact of future technologies.
a routine part of the IS tool kit. These methodologies encompass positivist, and critical work. One cannot say that a given methodology is better than another (Markus 1997). The choice depends on the research question and the context being studied.

## 1.1 The Conference Theme

The theme of this conference is “Researching the Future.” The future is a dynamic environment and is fraught with uncertainty. Because the future will differ from the present, simple extrapolation of current trends into the years ahead is not enough. Policies, organizational business practices, science and technology, and tastes can and will change Information Systems and their use. The occurrence of events and the direction of trends are unknown in advance. Often the best that can be done is assigning probabilities to outcomes. As a result, conventional methods of research are no longer sufficient when considering the long-term future.

Consider, for example, the future of the Internet and its security. At this writing, the United States government proposes to allow the President to shut down domestic Internet providers in an emergency (e.g., foreign cyber-attack). What is the probability this law will pass in 2011? If passed, will the authority be used? How severe must an attack be to declare an emergency? If an emergency is declared, how long will the Internet be down in the United States? In other countries? What will be the effect on the economy as a function of shutdown period?

Another example: The conventional wisdom is that the trend of security incidents is rising. Will this trend continue? Stabilize? Decrease? Alternate between increasing and decreasing? Are there inflection points? How will security be improved by specific policies? By enforcement? By government investment? By private sector investment? For each of these alternatives, when (if ever) will they be introduced? If one of them is introduced, how much will it affect other options? Will its security be perceived so poor that significant numbers will stop using the Internet? What are the economic implications if the Internet collapses? If alternative future situations are possible, what is the probability of each? If a particular unfavorable outcome is highly likely under one set of assumptions, are there policies, actions, or technologies that can ameliorate or eliminate that outcome?

Note that in these examples the scope is much broader than the IS field's conventional studies of individual organizations or industries. The impact of IS is moving from the organizational context to society and beyond. Furthermore, the research questions being asked by the MIS discipline have expanded from the operational and strategic to societal policies and issues. To address these questions, MIS researchers need to expand their methodology toolkit.
Although futures research methodologies are used extensively in such disciplines as policy analysis, strategy, competitive intelligence, and forecasting, and are widely used by industry, government, and centers for studying the future, they are rarely used in IS research. Studies about the future of information technology itself have appeared from time to time. Herbert Simon (1969) forecast design science and artificial intelligence. Dahlbom (1997; Dahlbom and Mathiassen 1997) discussed the future of IT in conceptual terms but did not use a futures methodology to address ICT research questions.

## 1.2 What This Paper Is About

This paper introduces three well-established futures research methodologies, and shows how these methodologies can be applied to rigorous IS academic research. The goal of this paper is technology transfer; that is, to familiarize IS researchers with these methodologies and explore some of the institutionalization mechanisms needed to incorporate them into the IS research tradition. The discussion is exploratory to initiate dialogue among IS researchers.

Section 2 briefly describes three standard futures methodologies (environmental scanning, Delphi, and scenarios) and illustrates how they can be incorporated into IS research. Using scenarios as an example, Section 3 discusses futures methodology as an academic method for IS research. This section includes epistemology, research design (research questions, unit of analysis, setting propositions and criteria for interpreting results), and issues such as construct and internal validity and generalizability. The conclusions (Section 4) present our views of what can best be accomplished with conventional and futures methods. Appendices A and B expand the descriptions of the Delphi and scenarios methodologies, Appendix C suggests guidelines for futures research in information systems, and Appendix D lists 37 methods used in futures research.

# 2 Futures Research Methods

This section describes three futures research methodologies: environmental scanning, Delphi, and scenarios. These methods were selected for this paper because they are relatively simple to implement and are widely used.

Futures research methods have been used actively since the 1950s. Methods such as Delphi, cross-impact analysis, and scenarios were discussed by this paper's authors in their IS research since the 1990s (Gray and Hovav 1999, 2002, 2007, 2008). Attempts to predict the future of IS by using extrapolations were also published in the 1990s (Cule and Grover 1994; Ein-Dor and Segev 1993). Most recently, Crespo et al. (2010) reference six Delphi studies on IS topics published since 2002.
## 2.1 Assumptions of Futures Research

The three methods and their variations described in this section share a common set of assumptions. First, the future is not prewritten nor is it wholly determined by the past. Second, present conditions and trends will continue along evolutionary lines unless natural limits are approached (e.g., speed of light) and/or extreme external changes occur (e.g., PC, Internet, iPhone, iPad).

External changes are inherently uncertain. Some aspects cannot yet be explained scientifically and some aspects (e.g., policy) depend on human choice. Attempts to guide the future through policy are based on (1) assessment of present and past conditions, (2) possible future changes, and (3) the extent to which future conditions can be altered by social resources. The desirability of social conditions is a value judgment, not based on fixed criteria of economic rationality.

Futures research methods should be undertaken when<fnref n="2" />

1. Uncertainty about the future is high

2. Historical data is limited

3. The number of stakeholders (e.g., IT and communications vendors, users, companies, government) is large

4. The issue is important for decision makers

5. Major impact on IS is involved and the issue is more than just technical

6. The analysts involved are knowledgeable about the problem

## 2.2 Environmental Scanning/Collective Intelligence

Environmental scanning<fnref n="3" /> involves observing the present and using the results to infer the future. Glenn and Gordon (2009) trace the method back to the crow's nest on ancient sailing ships. Environmental scanning is also a way to gain the input needed for SWOT (strength, weaknesses, opportunities, threat) analysis.<fnref n="4" />

Environmental scanning is used to create databases that serve as the foundation for futures studies. The objective is to distinguish among constant, slow, continuous or potentially disruptive change. Scanning involves analyzing multiple sources of

<footnote n="2">²These criteria were enumerated by Olaf Helmer (personal communications).</footnote>

<footnote n="3">³Also called collective intelligence, futures scanning systems, and early warning systems.</footnote>

<footnote n="4">⁴SWOT analysis is taught in business schools as a way of assessing present and future competition.</footnote>
information (public and private) and identifying early change indicators. Such indi-
cators are used to provide managers the required lead-time necessary to respond to
future events and trends. Public sources include database searches, executive
speeches and interviews, papers at meetings, and employment advertisements. Much
analysis and skill are involved in filtering information (i.e., deciding what is important
and what is irrelevant). Of particular interest, for example, are finding disruptive
technologies and obtaining reliable estimates of when they will reach the market.
Environmental scanning is often used in business intelligence.

## 2.3 Delphi

The Delphi method (Linstone and Turoff 2002) combines the wisdom of crowds (Surowiecki 2004) to estimate, as a function of time, the probabilities of events (e.g., an invention) and the evolution of trends (ongoing phenomena) when there is no source of factual data and a basis for opinion exists. The information sought usually affects decisions and policies.

In Delphi, questions are presented to a panel of subject experts on a topic of interest. For example, what is the earliest, most likely, and latest time when the Google.com market share for businesses equals Microsoft's? How many PCs will be sold in Africa in 2015? In 2020? In 2025? (Input provided to the respondents might include 2000-2009 sales.) Respondent anonymity prevents undue influence, groupthink, or biased judgment under uncertainty (Tsohou et al. 2008). The panelists are typically dispersed in space and time, and communication is often facilitated by information and communication technologies (e.g., email, GDSS, Web 2.0 applications).

A Delphi survey is administered in loop-back rounds, usually by a facilitator. Each round's responses are tabulated and fed back to the group. Panelists who provided extreme estimates (smallest and largest values) are asked to state their reasons. Their opinions are also used as inputs to the next round. The rounds stop when either consensus or dissensus occurs. Consensus is measured with traditional statistical tools (cluster analysis or distance from the mean). Desired betas<fnref n="5" /> are predetermined. Experimental results indicate that good forecasts can be obtained and that accuracy can be improved by asking people to state their expertise for each item. Dissensus on a question indicates that expert opinion on the subject is divided. At present, Delphi is the most commonly used futures research methodology in published IS research. Appendix A explains the method further.

<footnote n="5">⁵That is, the acceptable probability of Type II error.</footnote>
The cross-impact model (e.g., Glenn and Gordon 2009) is the next step of sophistication beyond Delphi. Delphi assumes events and trends are mutually independent and all other things are equal (i.e., *ceteris paribus*). These assumptions are rarely valid in the real world. For example, the probability of occurrence of a particular innovation may be more likely if another innovation occurs or if a trend reaches a threshold level. That is, event probabilities and trends do interact with one another. An additional set of inputs is therefore required: the estimated change (i.e., the impact) in the probabilities of every other event if a particular event occurs.

Cross-impact analysis is run as a multi-period stochastic simulation. The first period (e.g., 2 years) is examined. A random number generator is used to determine which events “occurred.” That is, if the random number for an event is less than or equal to the probability of an event during that period, the event is declared occurred. For each occurred event, its impact on events that have not yet occurred is used to update the future probabilities of events with which it interacts. Given the updated probabilities, the next period is run. The process is repeated until the entire time being studied is completed. To keep the updated probabilities in the range 0 to 1, the calculations involve the use of likelihood ratios. The output of a single run is a scenario that has a low probability of occurrence. The aggregate of all the runs—that is, the complete simulation—provides expected values and variances.

If the foregoing sounds complex, it is. Cross-impact requires many more inputs and calculations than Delphi. In terms of physical effort by researchers and expert panels, it takes much longer to perform than a Delphi. However, the results obtained provide a much richer understanding of the problem studied.

## 2.4 Scenarios

Scenarios are a way of communicating about the future. They are stories that describe the future in terms that managers and other nonspecialists can understand. That does not mean that they are arbitrary. They are carefully developed to reflect the logical implications of assumptions and forecasts about the future (Gray and Hovav 2007). Scenarios are an end product of a prudent analysis, not the workings of the imagination of a novelist or a science fiction writer. Scenario building is a discipline that involves a number of stakeholders and typically describes future events and trends within a given set of assumptions and constraints.

**Why Scenarios?** Organizations use scenarios to forecast alternative futures they may face. Scenarios examine the issues to be resolved, time relations, interactions, and logical consequences. Scenarios are most powerful when several are used to present alternative views of the future as seen from the present. They are often used as a policy analysis tool (Boroush and Thomas 1992). For example, if a scenario indicates a high probability of an undesirable future, actions can be taken to
reduce that probability (e.g., lobbying, merging, changing business practices, innovating).

Scenarios are most valuable when: (1) the external environment is highly uncertain, (2) the environment could take plausible alternative trajectories, or (3) specific indicators measure the driving forces and their likely impacts if they occur.

**Scenario Development.** Six principles in creating high quality scenarios are described in Boroush and Thomas (1992) and Morrison (1994).

1. Establish assumptions and constraints in advance.

2. Identify the critical (or possible) choices likely to be faced.

3. Identify key drivers.

4. Model or create the scenario space (see below). Select the values to be associated with each variable in the scenario space. The values should lead to a range of outcomes that cover the scenario space.

5. Analyze the implications of alternative contingency policies and strategic plans that cope with each scenario.

6. Use Delphi, qualitative discussion, or cross-impact analysis as inputs.

To use scenarios for long-term strategic planning, iterate steps 3, 4, and 5 to identify the impact of various actions.

**Scenario Principles.** Scenarios must adhere to three principles. They must be possible, plausible, and internally consistent.

*Possible*: No barriers to the events described can occur (e.g., speeds faster than light)

*Plausible*: The described events are believable (e.g., isolating all corporate computers from the Internet to ensure privacy is possible but not plausible)

*Internally consistent*: All parts of the scenario are consistent with one another (e.g., that a firm relies on law enforcement for cyber protection but personnel rarely report hacking attempts is internally inconsistent)

**The Scenario Space.** Scenarios are embedded in a scenario space. The space is defined by dimensions with prespecified values. For example, the scenario space in Figure 1 is based on two dimensions: technical software security and identity theft and privacy breaches, with each assigned a high and a low. These two dimensions create four scenarios, one for each combination.

The practicalities of scenarios and an example from our previous work are presented in Appendix B.
<table><caption>**Figure 1.** Sample Scenario Space</caption><thead><tr><th></th><th></th><th></th></tr></thead><tbody><tr><td>High Technical Security Level</td><td>Trusted</td><td>Social Engineering</td></tr><tr><td>Low Technical Security Level</td><td>Status Quo</td><td>Chaos</td></tr><tr><td></td><td>Low Identity Theft Level</td><td>High Identity Theft Level</td></tr></tbody></table>

# 3 Futures as an Academic Method

The theme of this conference indicates that investigating the future of ICT and ICT-based practices is ready to become a new, viable part of mainstream IS research. Researchers who want to establish new paradigms generally overcome the inevitable gatekeeper objections by mobilizing a community of supporters to achieve the convergence of opinions required to introduce new ways of thinking. To gain mainstream acceptance, futures research methodologies should have the characteristics of a scientific methodology so they can be applied with academic rigor.

This section presents exploratory ideas for making futures research methodologies viable for IS. The section is framed in terms of a specific methodology, *scenarios*, discussed in Section 2.4 and Appendix B. It is also framed in terms of the sequence of steps usually followed in IS research. The initial step is to explore whether existing IS research designs can incorporate established futures research methods to create rigorous and relevant results.

## 3.1 Epistemology

The epistemology of futures research methods can be either positivistic or interpretive. In positivistic research, the assumptions lead to propositions which are then investigated using, for example, Delphi or cross-impact analysis. Alternatively, they could be interpretive, where subjective processes lead to various outcomes (scenarios). In both qualitative and interpretive research, the role of the researcher is instrumental. The researchers' values and beliefs help shape their investigation. It is rare for two futures studies to arrive at identical solutions. Yet, by determining the necessary dimensions and driving forces in advance, the researchers can justify their assumptions and the logical progression of their arguments.

## 3.2 Theoretical Foundations and Research Design

**Research Design Components.** A proper design for both quantitative and qualitative research should include at least the following five distinct components:<fnref n="6" />

<footnote n="6">⁶Although, this list is based on Yin (1994), who centers his research on case studies, the basic steps also apply to other methodologies.</footnote>
1. Research question

2. Propositions or hypotheses (if applicable)

3. A defined unit of analysis

4. The logic linking data to the propositions

5. Criteria for interpreting the results

Each of these components is discussed in turn for futures research methodologies and additional considerations are presented. A set of guidelines based on this section are given in Appendix C.

**Research Questions.** The first issues considered are

1. What type of research questions are Futures Research methods able to address?

2. Can futures research methods be used in conjunction with theoretical underpinnings?

It is our contention that futures research methodologies are well suited to study what Markus and Robey (1988) termed “process theories.”<sup>7</sup> Process theories are longitudinal and dynamic. Process theories are defined as “causation consists of necessary components in sequence” (Markus and Robey 1988, p. 590) where chance and random events play a role. Future methodologies can be used to incorporate interactions (e.g., through cross impact) and unexpected events (e.g., through scenarios). In process theories, the dependent variable is a set of discrete outcomes rather than a continuous variable. Outcomes depend on the nature of the process as well as the levels of the predictors. Conventional variance theory, on the other hand, assumes that a change in the independent variables will result in a predicted change to the dependent variable. In addition, process models are event-driven (Tsohou et al. 2008). Process theories assume that the dependent variable can change over time and that the sequence and duration of events matter.

Process studies aim to answer research questions such as, “How does a particular issue emerge, develop, increase/decrease/cease over time?” (Tsohou et al. 2008). Futures methodologies are well positioned to answer such questions. For example, a scenario can describe the process by which an issue will emerge and the events that could lead to its development. Cross impact analysis can be used to estimate the probability distribution that an issue becomes salient over time or ceases to exist.

**Setting Propositions and Linking Data to Propositions.** Gregor (2006) defines five types of IS theories based on their structure. Theories for predicting (Gregor's Type III), address the question “what will be” rather than why something is.

<footnote n="7"><sup>7</sup> This contention is an example and does not negate the use of futures research methodologies to answer other type of research questions.</footnote>
Such theories should be used to predict outcomes from a set of relevant factors. Predictive theories can be used to define a set of alternative propositions. Unlike in traditional research, the assumption is that the future cannot be predicted and that propositions should be formulated as discussed above, under "Research Design Components" and their formulation depends on the methodology used. For example in a study of the future of academic e-journals (Hovav and Gray 1999), the authors used Eisenhard's (1994) framework to develop two propositions: the future of academic e-journals depends on user acceptance and on economic viability. A subsequent paper developed a scenario space and created four possible scenarios (Hovav and Gray 2002). The scenarios were used to link the data collected about six types of e-journals in support of the two propositions.

Gregor also states that predictive theories, although used in econometrics and finance, are rarely used in IS. She suggests using traditional research methods designed for studying the present and the past (e.g., correlations, regression, and data mining) to address Type III problems. We disagree. Futures research methodologies are more suitable to the study of these type of problems.

**Unit of Analysis.** Most futures studies tend to be broad, looking at topics that span societies, countries, organizations, and technologies.

**Selecting Criteria for Interpreting Results.** The methods illustrated in this paper depend on the selection of a set of assumptions. In their classification of current and future information systems, Ein-Dor and Segev (1993) argued that selecting the dimensions for classifying the systems is crucial and difficult. A different set of dimensions can lead to different results. Sommer and Sommer's (1991) guideline for ethnographic research also highlights the need to separate the selection of criteria from data collection. Separation is especially important due to the involvement of the researcher and his/her possible subjectivity in data collection. These assertions also apply to futures research. By determining the dimensions along which the future is described in advance, research subjectivity can be lessened. For example, one of the critical and more difficult steps in scenario building is identifying the driving forces that shape one potential future from others (Morrison 1994).

**Quality of the Design.** Construct, internal, and external validity are important in research design. Quantitative research has well-defined mathematical methods to measure internal and external validity. Qualitative research also has a set of requirements (e.g., Yin 1994), although these guidelines are less defined and are often subjective. The following subsections explore the content, internal, and external considerations for futures research methodologies.

**Construct Validity.** Construct validity refers to the ability to ensure that the measures used correlate with the theorized construct. Quantitative research uses methods such as convergent and discriminant analysis. Qualitative research uses triangulation, multiple sources of information, chain of evidence, and participant review (Yin 1994). Futures research methodologies could use multiple sources of
data and methods, converging expert review and the "wisdom of crowds." The implementation depends on the specific method used, the question studied, and the availability of data. Consider a simple example. A research project investigating the long-term relationships between Internet security and e-commerce might use both environmental scanning (Section 2.2) and Delphi (Section 2.3). If the two methods lead to similar forecasts, the validity of the relations found between security and e-commerce are assumed to be stronger.

**Internal Validity.** Internal validity refers to the validity of the causal relationships between constructs. In traditional quantitative research, internal validity is measured statistically. In futures studies as in qualitative research, internal validity guidelines need to be developed. These guidelines are likely to vary among methodologies. For example, in scenarios, the relationships among various outcomes need to be plausible, possible, and internally consistent (Section 2).

**Generalizability.** Generalizability (i.e., external validity) to a range of situations is the hallmark of a good theory. However, generalizability has to be examined from two aspects: (1) the methods and (2) the problems. The methods of futures research are generalizable in that they can be used in most studies about the "future of..." including both the future and the impact of IS. However, the results obtained from these methods rarely extend beyond the specific problem being considered.

**Comparison of Methodologies.** Table 1 shows an initial comparison of a futures research methodology (the creation of scenarios) with conventional qualitative and quantitative methodologies. This comparison provides illustrative examples and is intended to stimulate discussion.

# 4 Conclusions

Information Systems is a field that changes rapidly. It is subject to disruptions in both technologies and social norms. If IS researchers are to do more than speculate about change, they need to ensure that their analyses about the future rest on solid ground. Fortunately, by learning about existing, well-established methodologies, such as the ones presented in this paper and in Coates (2000), Glenn and Gordon (2009), Gray and Hovav (2008), and Linstone and Turoff (2002), researchers do not need to reinvent them. Rather, they can use them to gain insight into what lies ahead.

The results of applying any of the futures research methods cannot and does not forecast the future precisely. Because they do reduce the uncertainty, they can improve decision-making at the firm, country, and societal levels.
<table><caption>**Table 1.** Comparison of Quantitative, Qualitative and Scenarios Methodologies</caption><thead><tr><th>Item</th><th>Scenarios</th><th>Quantitative</th><th>Qualitative*</th></tr></thead><tbody><tr><td></td><td>Dynamic</td><td>Static</td><td>Static</td></tr><tr><td>Time</td><td>Longitudinal</td><td>Cross sectional</td><td>Mixed</td></tr><tr><td>Theory type</td><td>Process</td><td>Variance</td><td>Variance/process</td></tr><tr><td>Research question</td><td>Prediction</td><td>Analysis<br>Explanation</td><td>Analysis<br>Description</td></tr><tr><td>Construct and<br>internal validity</td><td>Possible, plausible,<br>internally consistent</td><td>Statistical</td><td>See Tsohou et al. 2008</td></tr><tr><td>Antecedents</td><td>Past events, trends</td><td>Quantifiable dependent<br>variables</td><td>Descriptive or quantifiable<br>dependent variables</td></tr><tr><td>Consequences</td><td>Alternative or<br>unforeseen outcomes</td><td>Quantifiable<br>independent variables</td><td>Descriptive or qualitative<br>independent variables</td></tr><tr><td>Postulations</td><td>Scenario space</td><td>Hypotheses</td><td>Propositions</td></tr><tr><td>Unit of analysis</td><td>Society, policy,<br>organization</td><td>Individual, group,<br>organization</td><td>Individual, group,<br>organization</td></tr><tr><td>Generalizability</td><td>Classification, analy-<br>tical, bounded by cases</td><td>Statistical, classifica-<br>tion</td><td>Analytical but not<br>statistical</td></tr><tr><td>Role of the<br>researcher</td><td>Researcher beliefs and<br>values help shape the<br>investigation</td><td>Neutral, objective</td><td>Weak or strong<br>constructionist views</td></tr><tr><td>Role of theory</td><td>Dimensions of the<br>scenario space, driving<br>forces and possible<br>directions</td><td>Define variables and<br>quantifiable relation-<br>ships, suggest models</td><td>Define variables and<br>descriptive relationships</td></tr></tbody></table>

*Although qualitative methodologies range from positivistic case studies to critical theory building, the entries in the table refer to positivistic case studies, the most common qualitative method used in MIS research.

## References

* Boroush, M. A., and Thomas, C. W.: Scenarios for the Defense Industry after 1995. Planning Review, 20, 3, 24–29 (1992).

* Coates, J.: Scenario Planning. Technological Forecasting & Social Change, 65, 115–123 (2000).

* Crespo, A. G., Colomo-Palacios, R., Soto-Acosto, P., and Ruano-Mayoral, M.: A Qualitative Study of Hard Decision Making in Managing Global Software Development Teams. Information Systems Management, 27, 247–252 (2010).

* Cule, P. E., and Grover, V.: Prospective on the Next Twenty Five Years. Into the Next Millennium: Some Thoughts on IS Practice and Research. Data Base, 25, 15–23 (1994).
Dahlbom, B.: Going to the Future. In: J. Berleur and D. Whitehouse (eds.), An Ethical Global Information Society: Culture and Democracy Revisited. London: Chapman and Hall (1997)

Dahlbom, B., and Mathiassen, L.: The Future of Our Profession. Communications of the ACM, 40, 6, 80–90 (1997)

Dalkey, N., and Helmer, O.: An Experimental Application of the Delphi Method to the Use of Experts. Management Science, 9, 458–467 (1963)

Dickson, G. W., Senn, J. A., and Chervany, N. L.: Research in Management Information Systems: The Minnesota Experiments. Management Science, 23, 913–934 (1977)

Ein-Dor, P., and Segev, E.: A Classification of Information Systems: Analysis and Interpretation. Information System Research, 4, 166–204 (1993)

Eisenhart, D.: Publishing in the Information Age: A New Management Framework for the Digital Era. Westport, CT: Quorum Books, Westport (1994)

Glenn, J. C., and Gordon, T. J.: Futures Research Methodology, Version 3.0. Washington, DC: The Millenium Project (2009)

Gray, P., and Hovav, A.: Using Scenarios to Understand the Frontiers of IS. Information Systems Frontiers, 1, 15–24 (1999)

Gray, P., and Hovav, A.: The IS Organization of the Future: Four Scenarios for 2020. Information Systems Management, 24, 113–120 (2007)

Gray, P., and Hovav, A.: From Hindsight to Foresight: Applying Futures Research Techniques in Information Systems. Communications of the AIS, 22, 211–234 (2008)

Hovav, A., and Gray, P.: Electronic Publishing: A Framework. In: Proceedings of the 5th Americas Conference on Information Systems, Milwaukee, WI (1999)

Hovav, A., and Gray, P.: Future Penetration of Academic Electronic Journals: Four Scenarios. Information Systems Frontiers, 4, 229–244 (2002)

Gregor, S.: The Nature of Theory in Information Systems. MIS Quarterly, 30, 611–642 (2006)

Linstone, H. A., and Turoff, M.: The Delphi Method: Techniques and Applications (http://is.njit.edu/pubs/delphibook/index.html) (2002)

Markus, L.: Writing Up “Intensive” Research in the “Standard Article Format” Enablers or Constraints. Working Paper, Claremont Graduate University, Claremont, CA (1992)

Markus, L.: The Qualitative Difference in Information Systems Research and Practice. In: A. S. Lee, J. Liebenau, and J. I. DeGross (eds.), Proceedings of the IFIP TC8 WG8.2 International Conference on Information Systems and Qualitative Research. London: Chapman & Hall, 11–27 (1997)

Markus, L., and Robey, D.: Information Technology and Organizational Change: Causal Structure in Theory and Research. Management Science 34, 583–598 (1988)

Myers, M.: Qualitative Research in Information Systems. MIS Quarterly, 21, 2, 241–242 (1997)

Morrison J.: The Future Tool Kit. Across the Board, 3, 18–25 (1994)

Robey, D.: Research Commentary: Diversity in Information Systems Research: Threat, Promise, and Responsibility. Information Systems Research 7, 400–408 (1996)

Simon, H. A.: The Sciences of the Artificial. Cambridge, MA: MIT Press (1969)

Sommer, B., and Sommer, R.: A Practical Guide to Behavioral Research: Tools and Techniques. New York: Oxford University Press (1991)

Surowiecki, J.: The Wisdom of Crowds, New York: Anchor Books (2004)
Tsohou, A., Kokolakis, S., Karyda, M., and Kiountouzis, E.: Process-Variance Models in Information Security Awareness Research. Information Management & Computer Security, 16, 271–287 (2008)

Walsham, G.: The Emergence of Interpretivism in IS Research. Information Systems Research, 6, 376–394 (1995)

Yin, R. K.: Case Study Research Design and Method (2nd ed.). Thousand Oaks, CA: Sage Publications (1994)

## About the Authors

**Paul Gray** is Professor Emeritus and Founding Chair of the School of Information Systems and Technology at Claremont Graduate University. His interests in studying the future started in the 1970s when he was a professor at The University of Southern California. He was previously a professor at Stanford University, the Georgia Institute of Technology, the University of Southern California, and Southern Methodist University. Prior to his academic career, he worked for 16 years in research and development organizations including nine years at SRI International. He received his Ph.D. from Stanford University.

**Anat Hovav** is an associate professor at Korea University Business School in Seoul, South Korea. She holds a Ph.D. in Management and Information Systems from Claremont Graduate University and has over 15 years of industry experience in information systems management and strategic planning. Her research interests include the socio-technical aspects of organizational information security, studies of the future, risk assessment, Internet standards, and electronic scholarship.

## Appendix A. Delphi

In the 1950s, the Air Force regularly convened expert civilian panels to obtain technical and/or policy advice about future directions. They found the advice they received was typically from one or two dominant participants, rather than the whole group. The dominant individuals were the most respected or most vocal. The Air Force hired RAND Corporation to develop a method to overcome group-think. The RAND researchers, led by Olaf Helmer and Norm Dalkey, decided the problem was the face-to-face interaction because inevitably the dominant voices prevailed. Therefore, they recommended the panels should be anonymous and unaware of who favored which policy.

**The Approach.** Delphi is a method that uses anonymity of opinion to obtain forecasts from a group of experts. A group is needed if the scope of the problem is too broad for any one expert or so new that there are few experts. The group cross-fertilizes ideas and ensures that all aspects of the problem are covered.

Delphi involves multiple rounds. During the first round, the experts are asked to give their estimates and their reasons for making those estimates. The resulting data is analyzed. If there is consensus on a particular question, there is no need to continue on that question. However, if there is dissensus, the distribution of answers (without attribution to individual respondents) is fed back to the group as are the rationales given for the outliers. The group is asked to re-estimate. This is round 2. Usually a number of rounds is needed to reach consensus or to conclude that there is dissensus.
The relatively straight forward steps in a perfect Delphi are<fnref n="8" />

1. Specify the subject and the objective

2. Specify all outcomes you want to know about

3. Specify how the results will be used if achieved

4. Include only precisely phrased judgmental questions that cover topics of interest

5. Assemble respondents who can answer questions creatively, in depth, and on schedule

6. Provide respondents with relevant historical data so that they anchor their answers based on reality

7. Collate the group's responses consistently and promptly

8. Make sure respondents are explicit about the reasons for their estimates, interpretations, and recommendations

9. Analyze the data

10. Identify any important needed improvements in methodology and results

11. Write and publish a report

**Research Results.** Experiments were undertaken by Delphi co-inventors Dalkey and Helmer (1963). UCLA graduate students and RAND analysts were asked questions, (e.g., number of telephones in Uganda) for which correct answers were available. However, only a few subjects were likely to know the answers. In general, the subjects made reasonable "estimates" for these questions. The major findings were

1. The spread of opinion narrows after the second round and the median usually shifts toward the true answer.

2. Delphi interactions generally produce more accurate estimates than face-to-face meetings.

3. Usually, the narrower the range of opinion, the more accurate the answer.

4. Self-appraised expertise is a powerful indicator of accuracy.

**Variations on Delphi.** Variations on the standard Delphi process include

* *Round 0.* In round 0, the panel members define the subject, using open-ended questions, rather than the organizer selecting the questions. This approach expands and refines the range of topics covered (e.g., in a Delphi on computer security, a panelist introduces a new physical form of identification or a different encryption algorithm).

* *Self-rating.* Rather than accepting all responses as equal, panelists rate their own expertise on each question. These ratings become weights when tabulating responses. Self-ratings are not perfect (e.g., a small fraction rates themselves low on almost everything but their specialty whereas others claim expertise on almost everything).

* *Permanent Panel.* For running many Delphi sessions over time, recruiting panelists for each inquiry becomes a major task. Helmer (personal communication) proposed setting up a large permanent panel (called D-net) from which people are chosen for a particular Delphi. Such a panel might include broadly knowledgeable, computer-literate, retired people.

<footnote n="8">⁸Source: Personal communication with Olaf Helmer.</footnote>
* *Mini Delphi.* Distributed technologies used in real time allow the panel to work simultaneously. For each round, the participants enter their responses anonymously into the computer and give reasons for their values. For each question, two or more participants (typically those with extreme answers) are asked to discuss their views briefly. Panelists then revise their estimates anonymously. This approach is particularly useful for gaining closure.

**Delphi Advantages.** Delphi brings together knowledgeable people to work on a problem and maintains their attention on the issues. The set of questions provides a framework in which to work. Anonymity minimizes barriers to communication. Delphi provides people the opportunity to be heard as individuals rather than as labels (e.g., affiliation, field, reputation). It produces precise, documented records.

**Delphi Disadvantages.** Anonymity and separation from other panelists removes the face-to-face stimulation of interactions. When invented, the process was paper and pencil. A round took considerable time because questionnaires and results of previous rounds involved reproduction, mailing, and waiting for responses. Panel size decreased with each round because of a lack of timely response. Much changed with email and the Internet. Although rounds run more quickly, the attrition problem still exists.

**Conclusions on Delphi.** Delphi is well established. It incorporates expert opinion into forecasts, providing insights for scenarios and input to cross-impact analysis. Occasionally, Delphi is easy to discount given the history of expert opinion failure and the many pitfalls in running it. Yet, the use of the wisdom of crowds, where the crowd knows about the subject, commends it where other approaches fail.

**Appendix B. Scenarios**

**Practicalities.** The sequence of steps in scenario development are

1. Select system variables of interest

2. Forecast system parameters

3. Develop scenario space

4. Perform a consistency check

5. Develop measurements

6. Write the scenarios

7. Analyze the scenarios and develop relevant policies

Scenarios for a given time horizon can be written from three points of view (Coates 2000):

1. A *newspaper story* written on some date in the future (e.g., July 1, 2020) describing the situation that day with no history or motivation. Assumes everyone knows what happened previously.

2. A *history* written on say, July 1, 2020, that describes the events that led to the future situation.

3. A *forecast* that starts in the present and shows the evolutionary path which results from the assumptions in the scenario space. This approach leads forward into the future rather than working backward from the end-point. This evolutionary approach is the one most often used for strategic planning and is the one the authors recommend.
In creating scenarios, it is important to discuss (1) the stakeholders involved, (2) the effects on the stakeholders, and (3) the assumed values of the parameters.

The parameter values must be consistent with the scenario space element being discussed. For example, a scenario for coping with security issues by making software an outsourced service results in a different role, size, and technical composition of the IS department than one based on technological advances and preventive tools.

**Number of Scenarios.** Because each scenario takes considerable time to create, only a few can be generated in a particular study. Select a small number of scenarios (e.g., four or six) from a scenario space which together represent diverse future outcomes. Present an even number of scenarios; an odd number leads managers to select the middle scenario as most likely and avoid considering the others (Morrison 1994). With two scenarios, managers select only the more favorable one.

**Scenario Pitfalls.** Coates (2000) suggests five pitfalls scenario builders should avoid:

1. Confusing the scenario and the outcome

2. Confusing the scenario with a forecast

3. Assuming that scenarios are static

4. Relying on extrapolation as THE scenario

5. Creating scenarios where everything is maximum or minimum or average

Example: In the 1990s, companies like Digital Equipment Corporation disappeared because they assumed the mainframe or minicomputer would continue to be the only future. They did not plan for microcomputers or Intel architecture.

**Conclusions on Scenarios.** Scenarios are stories used for communicating and are a basis for policy discussions. They help in understanding the implications of different outcomes and the range of possibilities as well as in developing alternatives that can react to or influence whichever future occurs. Scenario building is both art and science. Although it involves experts that follow the processes described here, rarely do two scenario builders create the same scenarios for a given problem (Coates 2000).

Example: Figure B1 and Table B1 summarize the findings of a study published by the authors in *Information Systems Frontiers* in 1999. Four scenarios were considered in a two-dimensional scenario space. The details are shown in Table B1. The shadings show the outcomes as of 2011. Much of the status quo remains but only three of the Dystopian elements exist and many of the Utopian and Technological outcomes came true. Recognize that scenarios rarely occur as forecast.

<table><caption>**Figure B1.** Scenario Space for 1999 Study</caption><thead><tr><td>Social Acceptance</td><td>Limited Telecommunications</td><td>Ubiquitous Telecommunications</td></tr></thead><tbody><tr><td>High</td><td>Status Quo</td><td>Utopian</td></tr><tr><td>Low</td><td>Dystopic</td><td>Technology</td></tr></tbody></table>
<table><caption>Table B1. Scenarios on the Future of IS***</caption><thead><tr><th>Variable</th><th>Utopian</th><th>Dystopian</th><th>Status Quo</th><th>Technology</th></tr></thead><tbody><tr><td><strong>Technological</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>Internet diffusion*</td><td>Internet III****</td><td>Internet II****</td><td>Internet II****</td><td>Internet III****</td></tr><tr><td>Rate of technological change*</td><td>Constant</td><td>Decreasing</td><td>Constant</td><td>Increasing</td></tr><tr><td>Y2K problem*</td><td>Solved</td><td>Unsolved</td><td>Partly solved</td><td>Solved</td></tr><tr><td>Moore's law*</td><td>Continues</td><td>Ends</td><td>Continues</td><td>Speeds up</td></tr><tr><td><strong>Socioeconomic</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>Economic crisis*</td><td>No</td><td>No</td><td>No</td><td>No</td></tr><tr><td>Energy crisis*</td><td>No</td><td>No</td><td>No</td><td>No</td></tr><tr><td>Globalization*</td><td>Continues</td><td>Stops</td><td>Continues</td><td>Stops</td></tr><tr><td>Computer literacy</td><td>Everyone</td><td>Constant</td><td>Everyone</td><td>Techies only</td></tr><tr><td>Privacy*</td><td>Medium</td><td>High</td><td>Medium</td><td>Low</td></tr><tr><td>Luddite reaction</td><td>No</td><td>Yes</td><td>No</td><td>No</td></tr><tr><td><strong>Information Systems</strong></td><td></td><td></td><td></td><td></td></tr><tr><td>Systems</td><td>ERP</td><td>Systems Integration</td><td>Mix**</td><td>ERP</td></tr><tr><td>Centralization</td><td>Decentralized</td><td>Centralized</td><td>Mix**</td><td>Decentralized</td></tr><tr><td>Sourcing</td><td>Outsourcing</td><td>Insourcing</td><td>Mix**</td><td>Insourcing</td></tr><tr><td>Software</td><td>Standards</td><td>Proprietary</td><td>Proprietary</td><td>Standards</td></tr><tr><td>Software</td><td>Packages</td><td>Custom</td><td>Mix**</td><td>Package</td></tr></tbody></table>

Source: Updated from Gray and Hovav (1999)

*See Gray and Hovav (1999) for assumptions.

**Mix implies both options included.

***These scenarios were created in 1999. The shaded areas show which elements were correct as of 2011.

****The term Internet II refers to a next generation Internet infrastructure using IPv6. Internet III refers to the version after IPv6. This terminology is not to be confused with Web 2.0, which refers to existing 2011 Web-based services.

## Appendix C. Suggested Guidelines for Futures Research in Information Systems

1. **Proper process:** Define the philosophical assumptions for the study based in theory, research design, data collection, data analysis based on preset criteria, and conclusions. Each step needs to be well defined, self-contained, and justified.

2. **Research design:** Use a structured research design, following the five steps described in Section 3.

3. **Triangulation:** Use several methods or several sources of data. Triangulation is particularly difficult when data is scarce. For example, a study can incorporate and compare results from a Delphi study and environmental scanning in selecting events and trends (i.e., multiple sources of data). Similarly, researches can use cross impact and scenario analyses when estimating the probability of future outcomes (i.e., multiple methods). This process is similar to the triangulation used in case studies that involve multiple cases and multiple investigators (Yin 1994).

4. **Content validity:** The definition of content validity in futures studies differs from the one
used in other IS methodologies. However, much as in traditional research, content validity can be obtained by using a panel of experts. For example, the discussion of scenario building in Section 2.3 stresses that events described in a scenario should be possible, plausible, and internally consistent. The results of a well-designed Delphi can be used to achieve these criteria (Section 2.2).

5. **Standard formats of reporting** (Markus 1992) result in making the reviewing process easier and in engaging the reader.

6. **Consistency in the unit of analysis:** Avoid mixing units of analysis in defining events and trends. It is too easy to shift from one level of analysis (e.g., the organization) to another (e.g., society). For example, if the unit of analysis is the level of technology over time, the Delphi questions presented to the experts should first ask them to provide not only a baseline estimate of the technology over time but also estimates of the technology's impact on society and suggested policies.

7. **The set of characteristics** or dimensions should be selected prior to the data collection or analysis (Sommer and Sommer 1991). Researchers should not confuse the subject of the analysis with its attributes.

## Appendix D. Futures Research Methodologies

Table D1 lists 37 futures research methodologies described in great detail in Glenn and Gordon (2009).

<table><caption>**Table D1.** Futures Research Methodologies</caption><tr><td>1. Environmental Scanning</td><td>14. Substitution Analysis</td><td>26. Using Vision in Futures</td></tr><tr><td>2. The Delphi Method</td><td>15. Statistical Modeling</td><td>27. Normative Forecasting</td></tr><tr><td>3. Real Time Delphi</td><td>16. Technology Sequence Analysis</td><td>28. Science and Technology Road Mapping</td></tr><tr><td>4. Scenarios</td><td>17. Morphological Analysis</td><td>29. Field Anomaly Relaxation</td></tr><tr><td>5. Cross-Impact Analysis</td><td>18. Systems Perspectives</td><td>30. Agent Modeling</td></tr><tr><td>6. Trend Impact Analysis</td><td>19. Robust Decision Making</td><td>31. Multiple Perspective Concept</td></tr><tr><td>7. Interactive Scenarios</td><td>20. Participatory Methods</td><td>32. Heuristics Modeling</td></tr><tr><td>8. Scenario Planning</td><td>21. The Futures Polygon</td><td>33. Causal Layered Analysis</td></tr><tr><td>9. Wild Cards</td><td>22. The Futures Wheel</td><td>34. Personal Futures</td></tr><tr><td>10. Decision Modeling</td><td>23. Structural Analysis</td><td>35. State of the Future Index</td></tr><tr><td>11. Text Mining for Technology Foresight</td><td>24. Genius Forecasting, Intuition, and Vision</td><td>36. SOFI Software System</td></tr><tr><td>12. Relevance Trees</td><td>25. Prediction Markets</td><td>37. Chaos and Nonlinear Dynamics</td></tr><tr><td>13. Simulation and Games</td><td></td><td></td></tr></table>

Source: Glenn and Gordon (2009)