View metadata, citation and similar papers at core.ac.uk

brought to you by

CORE

provided by RERO DOC Digital Library

Science and Public Policy 39 (2012) pp. 208–221

doi:10.1093/scipol/scs017

On concepts and methods inhorizon scanning: Lessons frominitiating policy dialogues on
emerging issues
Effie Amanatidou1,*, Maurits Butter2, Vicente Carabias3,
Totti Könnölä4, Miriam Leis5, Ozcan Saritas1, Petra Schaper-Rinkel6
and Victor van Rij7
1

Manchester Institute of Innovation Research, MBS, Harold Hankins Building,
University of Manchester, Manchester, M13 9PL, UK
2
TNO, Wassenaarseweg 56, 2333 AL Leiden, The Netherlands
3
EU DG JRC-IPTS, Seville, Spain, and ZHAW Zurich University of Applied Sciences,
Institute of Sustainable Development, Postfach 805, CH-8401 Winterthur, Switzerland
4
Impetu Solutions, Vı´ctor Andre´s Belaunde, 36-4C, 28016 Madrid, Spain
5
Editage/Cactus Communications Inc., C-912 BSEL Tech Park, Sector 30A, Vashi,
New Mumbai 400 705, India
6
Austrian Institute of Technology, Donau-City-Straße 1, 1220 Vienna, Austria
7
Advisory Council for Science and Technology Policy, Javastraat 42, 2585 AP The Hague,
The Netherlands*Corresponding author. Email: amana@otenet.gr.
Future-oriented technology analysis methods can play a significant role in enabling early warning
signal detection and pro-active policy action which will help to better prepare policy- anddecision-makers in today’s complex and inter-dependent environments. This paper analyses theuse of different horizon scanning approaches and methods as applied in the Scanning for
Emerging Science and Technology Issues project. A comparative analysis is provided as well asa brief evaluation the needs of policy-makers if they are to identify areas in which policy needs
to be formulated. This paper suggests that the selection of the best scanning approaches
and methods is subject to contextual and content issues. At the same time, there are certain
issues which characterise horizon scanning processes, methods and results that should be kept inmind by both practitioners and policy-makers.
Keywords: emerging science and technology issues; early warning signals; weak signals;
horizon scanning; policy support.

1. Introduction
The 2000s have witnessed increasing complexities insocieties. Although the world has improved for somepeople, the vast majority of people appear to be vulnerable
to social and economic instability and hostility due to theeconomic recession, lack of fresh water, shortages of foodand energy, climate change, regional conﬂicts, and population movements. At the same time, the new global context

suggests increased ﬁnancial, trade and investment ﬂows
leading to high levels of interconnectedness. It is also
observed that the current challenges, (such as the above),facing societies are highly complex with uncertain andfar-reaching implications. Within this context, it is crucial
to inform policy- and decision-makers about new and future
opportunities as well as threats and have them prepared (ormake them aware) for drastic changes and surprises/shocks.

ß The Author 2012. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com

Concepts and methods in horizon scanning . 209
A relevant tool for this purpose is horizon scanningdeﬁned as:. . . the systematic examination of potential (future) problems,
threats, opportunities and likely future developments,
including those at the margins of current thinking andplanning. Horizon scanning may explore novel and unexpectedissues, as well as persistent problems, trends and weak signal.
(Van Rij 2010a, 2010b)

Horizon scanning may also take place in speciﬁc policy
domains (Botterhuis et al. 2010).
Horizon scanning is generally seen as an instrument withtwo main functions: the alerting and the creative functions.
The alerting function helps policy-makers to anticipate
emerging issues better and earlier, while the creative
function enables the reassembly of issues or the creation of
new emerging issues on the basis of the analysis and integration of scan data. In this way horizon scanning contributes
to opening up new policy options or testing developed
policies in relation to their resilience to unknown, but plausible, new emerging issues. This is the case for example withthe horizon scanning services in the UK.1 Horizon scanningalso fulﬁls similar functions with scenario building. It may
even be used as a highly evidence-based source to provide
plausible discontinuities in the development of scenarios.
Recently, horizon scanning activities have been appliedat the national level to inform national foresight exercises,
thus providing a more evidence-based approach toresearch and innovation policy (Georghiou 2007). The
main focus of this activity has been to address new andemerging technological areas that may have an impact onsocial, economic, environmental and political developments. National horizon scanning activities have been
carried out as in the UK, the Netherlands and Denmark(Van Rij 2010a).
A special national horizon scanning activity is taking
place in Singapore under the Risk Assessment andHorizon Scanning programme which led to the development of a semi-automatic system that provides continuous
end-to-end capabilities to collect and classify data, analyseand understand relationships, and anticipate as well as
discover emerging issues that could have a strategic
impact on Singapore. It combines tools, methods andnetworks for effective scanning and analysis.2 Horizon
scanning places great emphasis on building a holistic framework whereby the output of scanning can be effectively
incorporated into vision building and strategic planning.
The present paper draws on the experiences from theseventh European Framework Programme horizon
scanning project Scanning for Emerging Science andTechnology Issues (SESTI).3 During the project thepresent authors developed an approach to organise andimplement horizon scanning processes effectively in order
to integrate the diverse information needs of policy-makers
about emerging issues. This paper is structured as follows:
Section 2 introduces the main deﬁnitions, as well as thedifferent scanning approaches and scanning tools thatwere applied in the SESTI project. Section 3 provides acomparative analysis of the two scanning approaches
and the associated tools that were used in SESTI.
Building on the advantages and disadvantages of eachtool and method, Section 4 evaluates the different
approaches and tools based on evaluation criteria reﬂecting the information needs of policy-makers about new andfuture opportunities and threats. Finally, Section 5 draws
conclusions on how horizon scanning can be used to
improve policy formulation and dialogue.

2. Defining and clarifying concepts
2.1 Weak signals alongside emerging issues andwild cards
Horizon scanning implies a search process, which isextended at the margins of the known environment andpossibly beyond it (Loveridge 2009). Horizon scanningaims to identify emerging issues and events which may
present themselves as threats or opportunities for society
and policy. Additionally, in this special issue Könnölä
et al. (2012) regard horizon scanning as:. . . a creative process of collective sense-making by way ofcollecting and synthesising observations that hold potential forthe formulation of pertinent future developments and thederivation of actionable implications on decision-making.

Horizon scanning typically builds on concepts such as
weak or early warning signals within the framework ofpolitical discourse and decision-making. Early warning
signals are the ﬁrst important indications of a change.
These signals are often difﬁcult to spot. Therefore they
are usually referred at as faint or weak signals.
These terms are closely connected, if not confused, withthe concepts of emerging issues and wild cards. Table 1
provides the deﬁnitions of these concepts, shedding somelight on their differences. While uncertainly and incompleteness characterise weak signals, it is their sudden
nature and high impact that mark wild cards. On theother hand, the concept of an emerging issue refers to ahigher level of abstraction in the sense that an issue
emerges when several weak signals combine together,based on interlinked possible impacts.
2.2 Exploratory and issue-centred scanning
There are different approaches which underpin thescanning process. One way of approaching scanning is todifferentiate between exploratory and issue-centred
scanning.
The exploratory scanning approach concentrates on
assembling potential emerging issues from a wide variety
of data from different signal sources, while theissue-centred approach concentrates on identifying core
documents (primary signals) that describe substantial

210 . E. Amanatidou et al.
Table 1.

Summary of concepts applied in horizon scanning

Concept

Description

Weak signals

Citing from Igor Ansoff, Hiltunen provides a deﬁnition of weak signals as: ‘warnings (external or internal), events anddevelopments that are still too incomplete to permit an accurate estimation of their impact and/or to determine theircomplete responses’ (Hiltunen 2008a). Weak signals can be further divided to physical or social signals (Uskali 2005)
Wild cards are events with a surprising character, a low probability and a high impact (Van Rij 2011). These events tend
to alter the fundamentals, and create new trajectories which can then create a new basis for additional challenges andopportunities that most stakeholders may not have previously considered or prepared for (Saritas and Smith 2011).
Wild cards represent the occurrence of a singular (historically original), sudden (abrupt, fast), surprising (unexpected) andshattering (serious, severe) event that come out of the blue and therefore present a signiﬁcant foresight challenge bothfor policy makers and researchers (Mendonça et al. 2009)
Beyond horizon scanning the concept of emerging issues is also used in the area of risk assessment (Marvin et al. 2009), in
the ﬁeld of environmental issues (Sutherland et al. 2010) and in organisations in their environmental scanning (Morrison
1992). It comes close to the concept of future narratives as used by Van der Steen (2008) in his political discourse
analysis. Van der Steen (2008) sees future narratives as ‘stories about what the future, or possible futures, may or will
(depending on the narrative) look like and that connect these possible futures to current issues for political debate (thediscourse)’. Recent horizon scans of the UK, Netherlands and Denmark show us many examples of smaller and larger
potential emerging issues. They usually describe a future narrative or mini scenario that, according to the cited author or
source, should be taken into account by policymakers, society and/or research. Recent examples have been analysed by
Boden et al. (2010)
Political discourse refers to the continuous communicative debate that takes place in the political arena. This debate is
unique for human beings with their ability of speech and writing (Chilton 2004; Van Dijk 2002) and nowadays audiovisualpresentations and computer simulations. The outcome of a political discourse is partly dependent on the contents andshape of the communicative expressions of the participants their cognition in wide sense as well as the power relationships
between the participants (Chilton 2004; Van Dijk 2002). Issue description and the early warning signals around them arecommunicative expressions that will eventually have to ﬁnd their place in this policy discourse in order to be relevant

Wild cards

Emerging issues

Discourse anddecision-making

parts of potential issues. In the ﬁrst approach emerging
issues are identiﬁed and described by processing information from different sources, while in the issue-centred
approach the (preliminary) descriptions of issues are used
as a core to identify potential additional signals that could
either conﬁrm or deny the real emergence of the issue.
2.2.1 Exploratory scanning. In exploratory or curiositydriven scanning the analyst, with no speciﬁc frameworkor any potential emerging issues in mind, searches for
information about new developments. This can be doneby using automated tools such as news alerts and feeds,
Google trends as well as automated text-mining. However,the novelty, quality and reliability of the information thatcan be retrieved in such a totally focus-free and open way
is highly questionable. A reﬁned method of exploratory
scanning is offered by applying a more bottom-up
approach where the analyst examines the observations
and topics that different (independent) scanning
communities focus upon, and how this information istransformed into potential emerging issues that can beinteresting and useful for decision- and policy-makers.
The aim of the exploratory scanning approach is to
identify a long list of signals that are precursors for
emerging issues, only demarcated by the policy domain
selected (e.g. healthcare or energy). The raw material
may be different observations about topics such as scientiﬁc and technological developments within various areas,
and may be obtained from research papers, laboratory

results, dissemination about new theories, experiments,
prototypes etc. Automated text-mining tools as well as
databases that allow for tagging and categorisation canhelp with clustering individual observations. Thus, one
may end up with different sets of individual observations
that could be related to each other under certain headingssuch as: new developments and funding aimed atminimising the negative health effects of human ageingor understanding, utilising and improving the working of
the human brain, or seeking and providing new
possibilities for new energy technologies and infrastructures. However, within a real bottom-up approach, theclusters are still open and are determined after a sufﬁciently large number of observations have been collected.
The exploratory scanning approach includes various
steps:. Setting a heuristic search proﬁle: Based on the domainof the audience a search proﬁle should be set in orderto obtain a more focused search. However, this search
proﬁle is only slightly focused, using keywords (or operators) such as: innovation, emerging, issues, impact,change, future, emerging, promising, threatening, solutions, discoveries, problems, crisis, tensions, growth,
breakthroughs, breakdowns, or new insights in combination with the domain demarcating keywords.
. Broad scanning for signals: The preliminary identiﬁcation of signals can use various sources, like wikis,
Twitter, and other internet sources such as blog
reports, literature consultation, but also conferenceConcepts and methods in horizon scanning . 211
proceedings. Additional information and leads are
obtained by participative approaches, for instancesetting up a (SESTI) wiki, evoking Twitter activity
around SESTI and more traditional methods such asa survey. The vast amount of data coming from thesesources can be analysed in terms of potential signals ofchange, where text-mining can be used as a tool for
analysis. It is important is that the search proﬁle isfocused on the selection of sources to be used (wiki,
experts invited, Twitter accounts, blogs and websites,
conferences, reports and papers).
. Preliminary assessment and selection: The selection of
key signals from the vast amount of data coming fromthe various sources is done by experts. This assessment
refers to the potential underlying emerging issue andnot to the individual signal. Criteria that can be used
include: plausibility (e.g. trustworthiness of source), potential impact (structural, interests at stake), novelty(lack of policy) and precursor’s weight (strength of
signal to the issue).
. Clustering of signals: The long list of signals is thenclustered, using potential emerging issues as a clustering mechanism. Text-mining can also be used to
identify clusters.

2.2.2 Issue-centred scanning. Whereas the above,
bottom-up approach can be considered as hypothesisgenerating, the issue-centred approach can be rather seen
as evaluating a hypothesis (the hypothesis of proposed
emerging issues). The issue-centred approach starts
from the wide range of existing and potential emerging
issues (hypotheses) and searches for weak signals inorder to strengthen or question the speciﬁc hypotheses,
i.e. reinforce the hypothesised emerging issue or revealchanges, modiﬁcations and disruptions of existing
emerging issues in a signiﬁcant way.
Automatic tools are seen as methods to support thescanning for potentially highly important weak signals
and to monitor and trace the weak signals and theirsources.
As a starting point a frame of reference is conceptualised
for the chosen policy domains. Signals are then sought thatgive a full or substantial future narrative with high impact
for a certain policy level. We refer to these signals as
primary signals. They are usually articles or presentations
by an author or group of authors, which describe a futurenarrative with a foreseen great impact on future science,society or economy in combination with a need forpresent-day (policy) decisions to prevent, mitigate or
stimulate the foreseen impact. The storyline is based on
their observations of breakthroughs, discoveries and/or
events which facilitate a single or multiple future storyline
leading to the impact. The story often contains some recommendations, or at least suggestions, on the way the

positive impact could be reached or the negative impact
could be prevented.
Only items (articles, websites and videos) with more orless full storylines that connect factual ﬁndings or plausible
assumptions in a logical way with a foreseen future highimpact are considered. These high impact items are supported by reliable sources such as: scientiﬁc articles, statistical analysis, trend reports and foresight exercises.
The storyline may address conﬂicts or common interests
as well as emotional, ethical and legal aspects. The storyline usually gives implicit or explicit elements that could be
used as indicators for the realisation of the storyline. Theseelements give clues for further scanning of signals that may
support or deny the possibility that the story will occur(the signals are known as secondary signals).
In a second round additional material is sought, that
may conﬁrm or oppose the selected storylines. If the secondary signals have manifested themselves in the period
before the scanning was done, then they can be added tothe description of the issues. In most cases it will also benecessary to track the secondary signals in the near future
to see whether the issue that was described in the primary
signals really evolves or dissolves. Even when the primary
signal is followed by many positive secondary signals andattracts a lot of policy attention the issue may suddenly
head in a completely undirected way because of the occurrence of wild cards or countervailing issues. It should also
be clear that issue-centred scanning does not predict issues.
Rather it provides tools to alert for potential impact-richissues that need policy attention.
In order to identify and assess the primary signals,
similar criteria were applied as for the bottom-up exploratory scanning presented above: described impact, desirability, factual basis, plausibility, novelty to policy-makers,
interests at stake, emotional aspects and critical aspects(if the issue appeals or concerns emotional or ethicallegal aspects), and changeability (if the story or its
impact can be altered by human action and how theissue relates to present-day decision-making and action).
Rather than being antagonistic, the two approaches
should be seen as complementary (see Fig. 1). Exploratory
scanning mainly refers to the ﬁrst scanning phase (identiﬁcation of weak signals) while issue-centred scanningspans throughout phases 1 and 2 of the SESTI scanning
process (cf. Section 3.1).
2.3 A variety of sources and scanning tools
Both of the above approaches used the internet as theirmain source of information. This made it possible not only
to search for signals in ofﬁcially published documents andreports but also to exploit relevant discussions in blogs andmicro-blogs (as Twitter), wikis and websites which provide
non-textual information as videos and audios. In addition,
experts and policy-makers were called upon to enrich,validate and discuss the scanning results.

212 . E. Amanatidou et al.

Figure 1.

In the SESTI project the following tools were used forthe scanning:4. web-based search engines as Google, Google News

Timeline,5 Google Insight6 and Bing7. expert review and survey. visits to conferences and seminars. a special SESTI wiki to evoke contributions to thescanning process. active use of blogging and micro-blogging (Twitter)8. text-mining. expert/stakeholder workshops
Based on the experience of SESTI, the scanning tools canbe grouped with regards to the level of participation, andthe means of processing information. Some of the methods
are suitable for obtaining speciﬁc information like expert
reviews, surveys and visits to conferences and seminars
while other tools like the initiation of a wiki and theactive use of blogs and micro-blogging can encourage
wider participation and dialogue. Search engines andtext-mining can be applied without the involvementof wider groups outside the project core team (nonparticipatory).
The second way of grouping involves the identiﬁcation,
processing and analysis of weak signals and emerging
issues with the help of various levels of automation.
These methods can be mapped within a methodologicalframework (see Fig. 2).
The focused expert review was mainly based on internet
scanning with the use of search engines. It was performedby professional scanners. Internet scanning makes it
possible to search for signals that can be traced in
written sources but also in audiovisual material. The
scanners were assisted by expert panels (expert review)that reviewed the raw observations of the searches. Theseraw observations contained issue-focused excerpts fromwritten, audiovisual and other sources. Written sources

Figure 2.

included scientiﬁc and non-scientiﬁc articles in e-journals
and newspapers, but also in wikis, specialised websites,
specialised web-blogs, policy papers and reports.
Audiovisual and other sources were audio and video
recordings of interviews, but also video documentaries,
reports and databases. A focused expert review supported
by internet scanning has the advantage that it can use any
available source and tool that is on the internet.
Wikis are a website-based technology for mass collaborative authoring. Looking at the considerable success of
Wikipedia, a wiki could be an interesting source fromwhich to collect signals on emerging issues. It can be aplatform for collecting information about signals andissues, as well as being a place to facilitate expert discussions. Wikis are ideally suited for collaborative learningand have the potential to foster participation, encourage
collaboration, and lead to a rapidly growing content.

Concepts and methods in horizon scanning . 213
Many of the sophisticated community annotation systems
that are currently used in biology are based on the wiki
concept (Mons et al. 2008; Stein 2008). In the SESTI
project an attempt was made to set up a wiki
that specialised in collecting voluntary descriptions frommany authors on new emerging issues for science andtechnology.
The attempt was not as successful as had been hoped
because most of the contributors did not want to spend
time writing new full-length articles on issues that had
already been described elsewhere. Thus, people tended tosend web links rather than writing contributions. This was
one of the reasons why the wiki was abandoned at an earlystage and an active Twitter account was started to collect
new links, which was successful in this respect. It may beconcluded that the functionality of an active wiki may
easily be caught by active twittering. It is important tonote that, for the wiki as well as for Twitter, contributions
can only be made by experts who are active in these typesof media.
Within the SESTI project, a survey was run asking
various experts about new emerging issues and signalsthat they perceived. This method has certain limitations.
First, the selection of the experts in the ﬁeld could bias theoutcomes. Respondents may be inclined to contribute theirpersonal interests rather than results from open thinking(even with ideas which compete with their own interests).
A second aspect is that a survey yields limited informationand is bound to the moment in which it is executed. Thus,
there is a limited possibility for retrieving new information.
However, a statement made by an expert that an issue isemerging can also be seen as a signal. Based on the SESTI
experience, an expert survey should be regarded as an additional tool with which to evaluate the issues which are
identiﬁed, rather than being used to obtain original signals.
A similar conclusion can also be drawn from attending
conferences. As conferences are often more focused on
new ideas, the assessment of discussions can be an interesting way in which to collect signals. Conferences may
also be particularly interesting for making contact with
future-oriented experts as well as policy-makers in therelevant domains.
A new way of collecting signals is through active
blogging and micro-blogging (Twitter). Twitter is aninternet social-network and micro-blogging platform withboth mass and interpersonal communication features forsharing 140-character messages (known as tweets), withother users (known as followers). A major advantage of
Twitter is that it is fast and information often appears on
Twitter before it is reported in conventional news formats(e.g. TV, radio and print media). Information about new
developments and also societal happenings can beobtained, even in real time. This can enable faster andtimelier assessments and thus earlier detection of events,changes and possible weak signals. Experts often tweet
about new trends and issues and also mention important

information from blogs and news websites. However, anapproach to processing the information is needed, as thesignals can very quickly amount to thousands of entries.
The SESTI experience with Twitter shows that, in principle, it is an interesting way of collecting signals, if theright information is ﬁltered. It is also crucial to selecttrusted and reliable twitterers (Pang 2010).
Text-mining enables large amounts of textual information to be assessed with specialised software. Within the
SESTI project, a number of foresight reports in the ﬁeld ofenergy and healthcare were processed with text-mining.
The experience showed that text-mining has a fundamental
ﬂaw when identifying signals. An essential criterion forthe signals to be identiﬁed is that they are unknown.
Text-mining uses a statistical routine to identify parts oftext that are frequently mentioned. This means that any
issues mentioned in these texts are relatively known. Moresophisticated software is needed to incorporate thefunctionalities that are needed for ﬁnding new signals
and emerging issues. Currently text-mining is especially
useful to identify networks and clusters of phrases withinhuge data sets, but less useful for identifying new signals
and issues. The disadvantage is that the publicly available
text-mining software is not yet useful for analysing
audiovisual material.

3. Comparative analysis of tools for
exploratory and issue-centred scanning
In order to place the analysis of the different approaches
and tools that were used in their correct context, the main
phases of the SESTI scanning process should ﬁrst be
clariﬁed.

3.1 SESTI scanning process
Three main phases can be distinguished, as practiced in the
SESTI project. Phase 1 relates to the identiﬁcation of weaksignals. Emerging issues are usually formulated on thebasis of searches in different sources and expert interviews.
These searches and interviews lead to a selection of moreor less full descriptions of potential emerging issues (futurenarratives) and also a body of additional signals that areconsidered to be indicative of the start or development of
their emergence.
Phase 2 relates to the processing of weak signals. This
phase consists of the following main steps:
Step 1 Selection of the broader area where emerging
issues will be examined.
Step 2 Clustering of weak signals.
Step 3 Assessing the signiﬁcance of clustered weaksignals.
Step 4 Framing the connected weak signals into clusteredtopics.

214 . E. Amanatidou et al.
Step 5 Tentative modelling of emerging issues into
possible emerging issues.
Step 6 Identiﬁcation of signiﬁcant emerging issues.
Phase 3 is devoted to the analysis and interpretationof emerging issues with relevance for policy-making.
Policy-makers interviewed at the start of the SESTI
project noted that it is important that the analysis of
emerging issues considers the possibilities to manage
emerging issues while also identifying the associatedpolicy implications. As has been highlighted, a proper assessment of weak signals should be translated into policy
recommendations. Thus, the last phase of the scanning
process should be devoted to this particular task. In the
SESTI project this task was complemented by workshops
which provided space for discussing the ﬁndings on
emerging issues with experts as well as with policy-makers
in order to draw conclusions about the implications forpolicy.

3.2 Comparative analysis of methods and tools
applied during the SESTI project
The focused expert review could be used for the entire
scanning process. Within the issue-centred approach it isa very useful tool with which to identify potential emerging
issues in a fast and cost-efﬁcient manner. It also enables
the identiﬁcation of potential secondary signals that can beused to contextualise these issues and to monitor theirfurther development. Database tools that are connected
to search engines such as Google News Timeline, Google
Insight, Web of Science,9 or Gapminder10 are very helpful
for this purpose. Another advantage of internet scanning isthat it uses all available sources of information on theinternet (including audiovisual information) as well as
web tools such as those mentioned above. In addition,internet scanners can use web-based translation facilities
to overcome language barriers. The restriction is the factthat professional scanners may have biases in theirsearches and interpretation of ﬁndings. Teams ofscanners with different backgrounds would help toovercome this pitfall.
The results of the scanning can also be a starting point
for taking into account the areas of unknown unknowns
by spotting gaps in the scan picture and by linking unconnected areas. That is about raising the awareness of
non-knowledge (areas beyond what we know we do notknow). Taking into account the plurality of perspectives
on non-knowledge can serve as a basis on which to establish transparent and pluralistic processes for exploring scientiﬁc non-knowledge (Böschen et al. 2010).
The focused expert review can also make use of all thetools for the identiﬁcation of signals (phase 1). In the
SESTI project online tools and especially web 2.0 tools
were used in different ways. The tool that turned out tobe less useful was the SESTI wiki. The reluctance of thecommunity to contribute made it difﬁcult to use this toolin any of the scanning phases.
Wikis and surveys are participative methods, and thus
the representativeness, level of expertise, and commitmentof participants are critical factors for success. The survey
conducted in the SESTI project was also confronted withthe limited time and visibility of the contributors.
Although a wide variety of potential participants was addressed, the results showed that the topics were often either
very general or very speciﬁc. The signals collected throughthe survey often led to websites and articles that werealready available from other sources. This makes thatspeciﬁc tool less appropriate for the identiﬁcation phase.
However, expert surveys can be quite useful in theprocessing and analysis phases where there is an explicit
focus on certain ﬁelds and issues. Comparing the SESTI
experience with experiences from other horizon scanning
processes, it seems that surveys are especially useful when
the ﬁelds are speciﬁed and the scanning starts fromwell-deﬁned ﬁelds such as energy, conservation or scienceand policy (cf. Czaplicka-Kolarz et al. 2009; Smith et al.
2010; Sutherland et al. 2010; 2011). Focusing on speciﬁc
ﬁelds, surveys can deliver additional information onvarious side aspects related to the core issue.
The use of Twitter could be useful for the identiﬁcationof signals and issues in multiple ways. Twitter can be used
as a source for signals and for tracing the way in which
signals (especially in regard to potential scientiﬁc breakthroughs) are spread and taken up by different
communities. Twitter can also be used as an interactive
tool. However, online software tools for clustering
Twitter tweets are necessary, especially for the processing
and analysis phases, but these have not yet been developed.
With the availability of such tools, Twitter could allow a
continuing clustering of signals related to emerging issues.
Policy-makers who were interviewed within the SESTI
project noted the importance of methods that allow for
identifying the connections, clustering of signals and thestakeholders behind them. To obtain this information(micro-) blogs are becoming interesting tools with which
to analyse which communities have taken up speciﬁc issues
and how they relate those issues to other issues. The
novelty of weak signals as well as rising ethical, legal,societal or cultural issues, and also possible implications,
could also be traced.
Twitter was also used to obtain data for text-mining.
However, with downloading and processing data fromTwitter one of the huge advantages of Twitter was lost:the ability to trace how signals evolve over time. In
general, however, the potential of Twitter to become amain tool for retrieving future-oriented information ishigh as colleagues, scientists and other futurists consider
future analysts to be their most valuable source for weaksignals (Hiltunen 2008b).
Overall, micro-blogging is one of the social web toolsthat could change how science is perceived and how

Concepts and methods in horizon scanning . 215
scientists communicate about controversial topics inscience. Today it is used only by a small minority. For
example, analysing and tracing Twitter tweets and therelated blog posts can be used to see how publications inhigh-ranking journals are commented on or criticised via
Twitter. For the scientists in the labs these developments
cause many problems because:. . . the speed of communication is ahead of the sheer timeneeded to think and get in the lab and work. (Mandavilli 2011)

The blogosphere in general is expanding at an unprecedented speed. With a better understanding of the blogosphere it would be possible to develop tools to use
collective wisdom to scan future-related signals andemerging issues. One important opportunity in thisprocess is to use web 2.0 tools in a participatory way to
involve interesting potential contributors (in particular abroad variety of experts) and the other opportunity is to
use them as fast expanding sources of structured information by clustering them. Clustering blog sites presents newchallenges for information science (Agarwal et al. 2010)
as no tools are yet available. In addition, there is also
the challenge of how to address and involve many individual contributors as well as to address a huge variety of
speciﬁc communities and contributors. Until these challenges are met, the tools of the social web will remain
mainly as additional tools which can be used by experts
when analysing issues.
Conferences may not primarily be interesting as places
where one can become aware of new signals as this information may be obtained sooner in other places. However,they are useful for discussing with future-oriented experts
as well as with policy-makers the validity of hypotheses on
speciﬁc emerging issues or their novelty, plausibility or
impact. This is particularly relevant for the analysis andinterpretation phase.
Text-mining, as noted above, needs more sophisticatedtools to capture the emerging and novel nature of weaksignals. However, text-mining is useful in the processing
phase as it identiﬁes networks and clusters of concepts
and phrases within huge data sets. However, certain
aspects of text-mining need to be considered. Today’s
text-mining software cannot be seen as a fully automatedsystem. The large lists of outcomes need to be processed
and an expert needs to ﬁne tune the underlying procedure(e.g. by adapting the search heuristics). A ﬁrst assessmentof the tools used in the scanning process in the SESTI
project is given in Table 2.
3.3 Policy workshops and policy discourse
Speciﬁc reference should also be made to the policy workshops that were conducted as part of the SESTI project.
These particularly helped the third phase of the scanning
process (analysing and interpreting the relevant implications of emerging issues for policy-making).

Table 2. Comparison of tools used in scanning process (rate of
appropriateness and usefulness: low, medium, high)Focused expert review
Wiki
Twitter
Surveys
Conferences
Text-mining

Phase 1:
Identiﬁcationof weak signals

Phase 2:
Processing of
weak signals

Phase 3:
Analysis andinterpretation

High
Low
High
Low
Low
Low

High
Low
Low
High
MediumMedium

High
Low
Low
High
High
MediumThree thematic workshops on cognitive enhancement,
energy and health were organised. The participants werepolicy-makers and experts with a broad view of thedomains being considered. Analytical presentations weremade on the selected emerging issues in the different
domains. These triggered the discussions that followed.
After discussing the issues participants were asked tovote on the terms of the perceived impact, plausibility,
novelty, policy relevance and strength of each of theemerging issues. Although policy-makers were underrepresented in these workshops, the results gave a clearer
idea of the relative importance of the issues and the way inwhich they could be assessed.
Although some issues were assessed as very important in
all aspects, policy-makers expressed doubts about whether
they would be taken into further consideration because
they were not sufﬁciently aligned with the policy
agendas. Linkages to issues that were already on theradar of policy-makers were considered helpful for improving the relevance and usefulness of the scanningoutcomes. The workshop experience also showed thateven though topics or issues may already be known, they
can still be considered to be emerging if they are considered to be of relevance for the future but have notbeen sufﬁciently taken into consideration by policy andsociety thus far.
Additionally, the timing of bringing up signals or
emerging issues could be a crucial factor for their further
consideration in the policy-making arena. Timing seems to
be a general challenge with early signal analysis. Due to thenovelty of issues the evidence basis at the beginning israther weak while the impact may be tremendous. Thus,
their dissemination may encounter scepticism or be totally
ignored. Once evidence is built up the claim for considering
them becomes more accepted and recognised. However,the signal will no longer be novel and the ability toprevent negative impacts may be lost. As the signal starts
to have visible negative impacts on its environment,
possible controversies and disruptions may emerge alongside a general frustration about the lost opportunity to be
informed earlier about the potential upcoming developments. This forms a difﬁcult challenge for scanners who

216 . E. Amanatidou et al.
want to be successful by giving the right alerts topolicy-makers at the right time in order not to over-occupy
their busy agendas.
Apart from timing, there are also challenges in bringing
forward a potential emerging issue or early signal topolicy-makers. Policy-makers may face barriers to taking
the results of horizon scans on board as they may becontradicted by vested interests. This may particularly be
the case with issues which include controversies or require
coordination and collaboration across different and segmented policies. In the cases where conﬂicts may ariseeven small investments in examining the issues may beblocked.
Overall, potential wild cards and emerging issues,
including their early warning signals, have to ﬁght for
attention in the political discourse except for situations
in which they ﬁt or are strongly connected into thepresent-day discourse and provided they are not against
the interests of powerful lobbies. Notwithstanding, astrong future narrative is able to draw the attention of
the participants and agenda setters. But, there is not only
competition among future narratives but also between
future narratives and present-day competing events,
which may include huge-impact wild cards like the surprisingly strong earthquake which occurred in Japan in March2011.

4. Evaluation of scanning approaches andmethods
The evaluation of the different approaches and methods
faces several challenges. First, each of the methods
described above has advantages and disadvantages
depending on the speciﬁc circumstances under which
they are applied, i.e. the interests of the client, the wider
policy context which they are associated with, time andbudget restrictions, capacities available or topic speciﬁcfeatures like potential risks and urgency, depth andwidth of implications, degree of public awareness andconcerns. Additionally, some methods are better for the
initial phases of the scanning process, while others ﬁtbetter into the analysis phase (cf. Table 2).
In this regard an evaluation across the different
approaches and methods is difﬁcult as their success ishighly contextual. However, some common criteria canbe identiﬁed if they reﬂect the information needs and interests of policy-makers and the degree to which they are metby the different tools and approaches (see Table 3).
A second challenge is that the two different approaches(exploratory scanning and issue-centred scanning) do notcover the whole scanning cycle but complement each other(cf. Fig. 1). In this regard, they are not directly comparable. They can be considered to be two departure points
for the scanning activity. For example, the exploratory
scanning approach represented by tools such as

Table 3.

Evaluation criteria for scanning approaches and methods

Connections, clustering of weak signals and degree of relevance to a
speciﬁc area
Duration of weakness of signal, also associated with time at which
signal is observed
Origin (stakeholder(s) behind them) and novelty of weak signals
Rising ethical, legal, societal or cultural issues
Existence of a strategy already concerned with speciﬁc weak signal(s)and emerging issues by a government or industry, political party or
lobby, or international organisation
Positive and negative impacts and associated policy implications
Policy recommendations

text-mining, Twitter or wiki scanning hardly goes beyond
the second phase, i.e. processing of weak signals. On theother hand, in the focused expert review, the main
scanning process of the issue-centred scanning, spansthrough all the different phases of the scanning cycle andcan be assisted by other tools.
Any useful evaluation of different methods based on theinformation needs of policy-makers should apply tomethods or combination of methods that cover all the different phases of the scanning activity. Following theframework of tools and methods presented in Fig. 2,
certain combinations of methods can be created toprovide a complete evaluation along the scanning
process. Three possible combinations are deﬁned below:. Twitter/wiki scanning which is complemented by pro-cessing of weak signals.
. Focused expert review which is complemented by

text-mining.
. Focused expert review which is assisted by experts’survey, literature review and attending conferences.
The ﬁrst combination can be considered to be more in linewith the exploratory scanning approach, while the last two
refer mainly to the issue-centred approach. These combinations present different features in terms of the degree ofautomation of the tools that are used and the degree towhich they depend on participatory tools bringing together
experts’ knowledge (cf. Table 4). Table 5 evaluates thethree combined approaches based on the criteria mentioned in Table 3.
The SESTI experience of applying these methods (notnecessarily as combined above) revealed that it is important to have several alternative methods and means to cover
the needs of the different scanning phases as the same
method may have advantages with respect to one criterionand scanning phase but disadvantages for another.
Additionally, the SESTI experience has shown the crucial
value of networks and human expertise. In general, acrossall combined tools the value of the ‘human factor’ outweighed the beneﬁts of any automation tool as these are
dependent on a human mind understanding, analysing andsynthesising various pieces of information.

Concepts and methods in horizon scanning . 217
Table 4.

Description of three combined approaches

Automation tools(a) Semi-automated, participatory
approach(b) Semi-automated, non-participatory
approach(c) Manual combined approach(Twitter/wiki+processing of weaksignals)
Search engines, automated news
aggregators, topic ﬁnders(Focused expert review+text-mining)(Focused expert
review+survey+conferences)
Search engines

Size of source of
information

Limited: speciﬁc

Duration:
observation timeThrough reports, blogs, groups, social
networks, Twitter, sites, etc. (observation time depends on participants)

Signal/issue selection Pre-deﬁned criteria based on sources
criteria
other futurists use (sense of credibility) and through interactions anddiscussions within experts andnetworks about implications

Veriﬁcation method

Cross-referencing with other articles

Text-mining based on proximity ofwords and frequency of terms.
Search engines for focused expert review
Large: wide
Limited: speciﬁc for survey and conferences, large for focused expert review,
literature review
Experts’ knowledge; literature;
Reports as pdf, docs, rtf and parts
conferences for additional signal/issue
from website, e.g. RSS (Really Simple
selection and reﬁnement of reference
Syndication) feeds for text-mining.
frame (observation time depends on
Articles, websites, videos for focusedparticipants and scanner)expert review. Any other info
piece for second-round scanning(observation time depends on scanner)
Pre-deﬁned criteria reﬂected in survey
Text-mining enables either ‘bottom-up’questions, also driven by speciﬁcanalysis (from ‘scratch’) or via
reference framework used in focusedmanually inserted keywords. Selection
expert review. For literature reviewof which documents to include isand selection of conferences same
crucial especially in ‘bottom-up’reference framework
analysis. Focused expert review isbased on pre-deﬁned reference frame
Discussions with experts
Discussions with experts; conferences;
further outlook in literature

Overall, the features of the participatory methods
enabling veriﬁcation and ﬁltering of results as well as
policy implications and recommendations are of crucial
importance for the last stages of the scanning process. At
the same time, semi-automated methods are appreciated,
especially those drawing on large expert communities, as
they enable systematic scanning and retrieval of ‘inside’
information in the ﬁrst scanning phases as well as in verifying the results. The value of the more ‘conventional’
methods like expert surveys, literature review and conference visits is more relevant for targeted tasks requiringexpert inputs like reviewing the state-of-the-art in a given
ﬁeld, searching for comprehensive reports on weak signals
and emerging issues, and verifying the results or reference
frames which may be required throughout all the scanningphases.

5. Conclusion
Horizon scanning is generally seen as an instrument withtwo main functions: the alerting function and the creative
function. Horizon scanning alerts policy-makers to anticipate better and earlier emerging issues that will probably
need their attention. At the same time, it is a creative
instrument that can be used to reassemble issues or evencreate new emerging issues on the basis of the analysis andintegration of scan data. This function allows a wider
range of policy options to be considered. It also helps in

the assessment of developed policies in relation to theirresilience to unknown, but plausible, new emerging issues.
For the alerting function a comprehensive method isneeded to scan and assess early warning signals that may
indicate potential emerging issues from of a variety of
media and sources. In the SESTI project this was doneby internet scanning with the use of the publicly available
search engines such as Google and Bing. Additionally,
expert networks were invited through web-based tools to
help in the search for early warning signals. For the morecreative function this method needs to be complemented
with tools and participative processes that, on one hand,
may be focused on clustering and synthesis of scan data
and, on the other hand, on human imagination and creativity. Within the SESTI project different text-mining
techniques were used for this purpose as well as exercises
which used participative web tools such as Wikipedia,
blogs and Twitter.
The SESTI project also included workshops with experts
and stakeholders that served both of the two main functions. On the one hand, alerts for emerging issues comingfrom the (expert reviewed) internet scanning were discussed and assessed, thus serving the alert function. On
the other hand, the signals and issues presented were also
used to evoke creative ideas by recombining (parts of the)identiﬁed issues, thus serving the creative function.
The SESTI experience provides insights on how toorganise and implement exploratory and issue-centredhorizon scanning processes to produce alerts but also to218 . E. Amanatidou et al.
Table 5.

Analysis of three combined approaches across evaluation criteria

Connection/clustering

Duration:
observation time(a) Semi-automated, participatory
approach(b) Semi-automated, non-participatory
approach(Twitter/wiki+processing of weaksignals)
MediumConnections pre-deﬁned throughtagging. Also inputs about links fromﬁeld experts and futurists. Clusters
are based on occurrence of similarwords in text ﬁles and throughattached criteria of collected weaksignals.
Cross-checking with results from recent
foresight exercisesHigh
As reported but also time series tracing
possible in blogs, google etc.

(Focused expert review+text-mining)(Focused expert review+survey+conferences)
MediumHigh
Connections and clustering is madeAutomatic clustering of keywords inas reported in survey responses,
text-mining. In focused expert review
literature and based on reviewers’
the primary signal contains a richexpertise. Visit of conferences tonarrative that already relates many
help conﬁrm ﬁndings and clustering
aspects of the emerging issues. Second
scan may deliver richer or alternative/
contradictory descriptions that may
lead to combined or meta descriptions
MediumAs attached to raw sources for
text-mining. In focused expert review
via tracing back earlier signals andby monitoring secondary signals
MediumAs revealed by text-mining; as reported
in focused expert review ﬁndings

OriginsHigh
As included/connected to info piece

NoveltyHigh
Cross-checking with scientiﬁc databases.
Cross-checking with results fromrecent foresight exercisesHigh
In focused expert review by tracing
back on internet (e.g. google timeline)

Ethical, legal,societal issues

MediumExpert discussions, futurists symposia,
how/where newspaper articles are
written. Further search for related
documents dealing with ELSI
dimensions
MediumThrough discussion with peers

High
In text-mining, the analyst can lookdeeper in case certain words canbe connected. As reported in raw
sources in focused expert review

Filtering based on
relevance, andcredibility

Existing strategies
and relevant
stakeholders

Policy implications
assessment

MediumAs reported/criticised in raw resources.
In processing existing policies are
identiﬁed and considered in framing/
reframing of potential issues bymanual search for related previous
foresight activities and related
ongoing policy initiatives anddiscourses
MediumAssociated policy implications ofemerging issues are analysed bycomparing emerging issues identiﬁed
with topics in previous publishedthematic foresight report as well as inrecently published policy documents(c) Manual combined approach

MediumIn text-mining dependent on sources
fed into database. In unstructured
Internet text-mining results can beﬁltered according to their issue, totheir organisation, to their reputationand creditability on the Internet. In
focused expert review ensured throughapplication of speciﬁc reference frame
and associated selection criteria
MediumText-mining can show policy-related
terms appearing in a document andlinks between keywords and stakeholders. These cases require moreexamination by analyst. In focusedexpert review secondary scanning
can reveal relevant information

MediumAs reported by survey respondents;
use of tools like google timeline infocused expert review; as reported
in literature/discussed in conferences
MediumAs reported by survey respondents;
as reported in focused expert review
ﬁndings; as reported in literatureHigh
As reported by survey respondents;
use of tools like google timeline infocused expert review; as reported inliterature/discussed in conferencesHigh
As reported in survey responses,
literature and raw sources in focusedexpert review; and as discussed atconferencesHigh
In survey ensured by survey questions.
In focused expert review ensured
through application of speciﬁc reference frame and associated selection
criteria. In literature review based on
documents selected for review basedon experts’ knowledge. In conferences
through discussions with peers
High
As reported in survey responses andin literature and synthesised byreviewers. In focused expert review
secondary scanning can reveal
relevant information. Conferences
are a place where existing strategies/
policies are usually discussed

High
High
Text-mining can show the policy related As reported in survey responses and inliterature. As facilitated by narratives
terms. In focused expert review narrain focused expert review. Conferences
tives in the primary scanning usuallyare useful to recruit potential policy
contain already policy implications orworkshop participants
even policy advice of author. Whilesecondary scanning usually gives ideas
on elaboration of proposed policies orof critics(continued)

Concepts and methods in horizon scanning . 219
Table 5. Continued

Policyrecommendations(a) Semi-automated, participatory
approach(b) Semi-automated, non-participatory
approach(c) Manual combined approach

MediumThrough discourses, networking, interaction with experts. Also via examination of relevant thematic foresightreports and recently published policy
documents

High
Meta descriptions of issues can be
discussed in Workshops with experts
and stakeholders and usually lead torecommendationsHigh
As reported in survey responses andin literature. As facilitated by policy
workshops

create new options taking into account the diverse information needs of policy-makers on emerging issues. Several
lessons can be drawn from this experience that should be
kept in mind by scanning practitioners and policy-makers
alike.
Different approaches to scanning, identifying andassessing potential emerging issues exist. It is clear thatissues are dynamic, social constructs that are partly
evidence-based, and partly the results of the imagination,
thinking and debating that takes place within different
segments of society including scientiﬁc communities.
These types of debates can also be evoked by thehorizon scanning process itself. Therefore horizon
scanning is not merely about searching for signals andtheir factual evidence. It is about analysing and understanding the societal contexts behind the entire processof initiation, communication, (r)evolution and dissemination of issues, as well as their early recognition and monitoring by detecting and assessing early warning signals.
Special attention has to be paid to the way in which
new issues are (r)evolving through communication (inall types of media) and are taken up by decisionand policy-makers. This means that not only theevidence-based plausible storyline in the identiﬁed futurenarrative counts. In addition, it is crucial to have information about who initiated the signals or issues, who
followed, who opposed them, when and why, taking into
account the interests, emotions and attitudes of different
stakeholders as well as experts. Tools such as Google
Insight, Statistics or Google News Timeline that allow us
to obtain metadata as well as backward views on
issue-related information, are important for retrieving
such information.
The SESTI project used the issue-centred and exploratory scanning methods for the alerting function as well as
the creative function. Each type of approach and method
has speciﬁc advantages and disadvantages. Overall, it canbe said that the added value of emerging issue scans lies inthe strategic combination of all available tools to broaden
the spectrum of possible signals and to interpret them in afunctional way for decision-makers by gathering and processing all relevant information. In this regard, it is important to improve the credibility of the assessment for

example by establishing interdisciplinary discussion
groups to validate it. It is also important to complement
the validation process with relevant surveys or workshops.
It is equally important to utilise automated tools for theclustering and network analysis of issues, as well as tomaintain close interaction between the clients of theanalysis (e.g. decision/policy-makers) and thoseundertaking the scanning task.
It also has to be kept in mind that human intelligence—either as a collective or of single experts—is a valuable
necessity, especially for the alerting function of horizon
scanning. For the more creative function it is necessary
to realise that the shaping of emerging issues is a social
activity where foresight intelligence is (re)shaped by theperceptions, interests and needs of those people who are
involved in the process. Given also the limitations of thescanning methods, as outlined above, the speciﬁc nature of
the ﬁeld examined, as well as the signiﬁcant importance ofthe human factor in creating, detecting, analysing andinterpreting weak signals, scanning results may only be
accepted with reluctance in evidence-based policy-making
environments. In this regard, obtaining relevant information alongside validating sources, information and theprocess itself is of crucial importance. At the same time,
horizon scanning has to be accepted as an additionalsource of evidence for policy-making.
Notwithstanding the above, experience has also shownthat the effective selection and communication of the mostimportant issues at the right time is both a challenge andan absolute necessity. In national scans, as in the
Netherlands and the UK, important issues were identiﬁedthat started causing severe damage in two to three years(van Rij 2010b). If these issues could have been put
forward in a more effective way in policy agendas, thenit would have been possible to contain this damage before
it reached its full scale. This also means that the policy
system should be made aware of the beneﬁts of taking
horizon scanning results into account early enough andmore seriously.
Another interesting aspect is to see how scanning results
are taken up within the present-day policy-making
processes in comparison with other results coming forexample from forward-looking activities drawing upon

220 . E. Amanatidou et al.
model-based forecasting. In general, model-based
forward-looking results are taken into account far more
seriously by policy-makers than horizon scanning data,
even though economic models completely failed toforecast the ﬁnancial crisis of 2007–8, even in the
shortest term. On the other hand, horizon scans in the
Netherlands and UK had already spotted the ﬁnancial
crisis two years before it started. In the Netherlands thisemerging issue was ranked among the top 20 most important issues. However, because of a shortage of resources tomonitor these 20 issues in more depth, policy-makers
downgraded the importance of the speciﬁc issue by maintaining their faith in the reassuring messages from theeconomic planning agencies (van Rij 2010b).
The special nature of horizon scanning activities points
to the importance of continuity to increase the relevance,
validity and timeliness of the information. This implies
sustaining resources. As Choo (2001) points out, horizon
scanning is a long-term investment that needs a criticalmass of talent and resources. Nevertheless, the results
may still not be taken up by policy-makers for a numberof reasons as mentioned above. Involving policy-makers
sufﬁciently early in the process so that they can provide
regular information on the process and obtain results isimportant. It is also important that policy-makers areactively engaged, especially in interpreting the results andderiving policy recommendations. Even more so, it is
crucial to make them aware of the beneﬁts of effectively
dealing with potentially important issues early enough aswell as the risks of overlooking them.

Acknowledgements
The views expressed are purely those of the author andmay not in any circumstances be regarded as stating anofﬁcial position of the European Commission.

Notes
1. See <http://www.bis.gov.uk/foresight/our-work/
horizon-scanning-centre> accessed 21 December 2011.
2. See <http://app.hsc.gov.sg/public/www/home.aspx>
accessed 8 September 2011.
3. See <http://sesti.info/> accessed 8 September 2011.
4. No use was made from more advanced tools as
web-crawler or spyware.
5. See <http://news.google.com/news/advanced_news_
search?pz=1&cf=all&ned=us&hl=en> accessed
21 December 2011.
6. See <http://www.google.com/insights/search/>
accessed 21 December 2011.
7. See <http://www.bing.com/> accessed 21 December
2011.
8. See <http://twitter.com/> accessed 21 December 2011.

9. Provided by Thomson Reuters <http://
thomsonreuters.com/products_services/science/science_products/a-z/web_of_science/> accessed
21 December 2011.
10. See <http://www.gapminder.org/> accessed
21 December 2011.

References
Agarwal, N., Galan, M., Liub, H. and Subramanyab, S. (2010)‘WisColl: Collective wisdom based blog clustering’,
Information Sciences, 180: 39–61.
Boden, M., Cagnin, C., Carabias, V., Haegeman, K. andKönnölä, T. (2010) ‘Facing the future: Time for the EU tomeet global challenges’, 24364 EN, 6/2010. Luxembourg:
Ofﬁce for Ofﬁcial Publications of the European Commission.
Böschen, S., Kastenhofer, K., Rust, I., Soentgen, J. and Wehling,
P. (2010) ‘Scientiﬁc nonknowledge and its political dynamics:
The cases of agri-biotechnology and mobile phoning’, Science
Technology and Human Values, 35: 783–811.
Botterhuis, L., van der Duin, P., de Ruijtera, P. and van Wijck,
P. (2010) ‘Monitoring the future. Building an early warning
system for the Dutch Ministry of Justice’, Futures, 42:
454–65.
Brohee, S., Barriot, R. and Moreau, Y. (2010) ‘Biological knowledge bases using Wikis: combining the ﬂexibility of Wikis withthe structure of databases’, Bioinformatics, 26: 2210–1.
Chilton, P. A. (2004) ‘Analyzing Political Discourse: Theory andPractice’. London: Routledge.
Choo, C. W. (2001) ‘Environmental scanning as information
seeking and organizational learning’, Information Research,
7/1<http://choo.ﬁs.utoronto.ca/IR/choo/choo.html> accessed
30 September 2011.
Czaplicka-Kolarz, K., Stanczyk, K. and Kapusta, K. (2009)‘Technology foresight for a vision of energy sector development in Poland till 2030. Delphi survey as an element oftechnology foresighting’, Technological Forecasting andSocial Change, 76: 327–38.
Freund, F. (2011) ‘Pre-earthquake signals: Underlying physicalprocesses’, Journal of Asian Earth Sciences, 41: 383–400.
Georghiou, L. (2007) ‘Future of foresighting for economic
development’, UNIDO, Vienna. <http://www.unido.org/
foresight/rwp/dokums_pres/tf_plenary_georghiou_201.pdf>
accessed 8 September 2011.
Hiltunen, E. (2008a) ‘The future sign and its three dimensions’,
Futures, 40: 247–60.
——. (2008b) ‘Good sources of weak signals: A global study ofwhere futurists look for weak signals’, Journal of Futures
Studies, 12: 21–42.
Könnölä, T., Salo, A., Cagnin, C., Carabias, V. and Vilkkumaa,
E. (2012) ‘Facing the future: Scanning, synthesizing andsense-making in horizon scanning’, Science and Public
Policy, 39: XXX–XXX, in this special issue.
Loveridge, D. (2009) ‘Foresight: The Art and Science of
Anticipating the Future’. New York and London: Routledge.
Mandavilli, A. (2011) ‘Trial by Twitter’, Nature, 469/7330:
286–7.
Marvin, H. J. P., Kleter, G. A., Frewer, L. J., Cope, S. et al.
(2009) ‘A working procedure for identifying emerging foodsafety issues at an early stage: Implications for European
and international risk management practices’, Food Control,
20: 345–56.
Mendonça, S., Pina e Cunha, M., Ruff, F. and Kaivo-oja, J.
(2009) ‘Venturing into the wilderness: Preparing for wild

Concepts and methods in horizon scanning . 221
cards in the civil aircraft and asset-management industries’,
Long Range Planning, 42: 23–41.
Mons, B., Ashburner, M., Chichester, C., van Mulligen, R.
et al. (2008) ‘Calling on a million minds for community annotation in WikiProteins’, Genome Biology, 9: 5.
Morrison, J. L. (1992) ‘Environmental scanning’. In: Whitely,
M. A., Porter, J. D. and Fenske, R. H. (eds) A Primer for
New Institutional Researchers, pp. 86–99. Tallahassee, FL:
Association for Institutional Research.
Pang, A. S.-K. (2010) ‘Social scanning: Improving futures throughWeb 2.0; or, ﬁnally a use for Twitter’, Futures, 42: 1222–30.
Saritas, O. and Smith, J. E. (2011) ‘The Big Picture: Trends,
drivers, wild cards, discontinuities and weak signals’, Futures,
43: 292–312.
Smith, J., Cook, A. and Packer, C. (2010) ‘Evaluation criteria
to assess the value of identiﬁcation sources for horizon
scanning’, International Journal of Technology Assessment in
Health Care, 26: 348–53.
Stein, L. D. (2008) ‘Towards a cyberinfrastructure for the
biological sciences: progress, visions and challenges’, Nature
Reviews Genetics, 9: 678–88.
Sutherland, W. J., Clout, M., Côté, I., Daszak, P. et al. (2010)‘A horizon scan of global conservation issues for 2010’,
Trends in Ecology and Evolution, 25: 1–7.

Sutherland, W. J., Fleishman, E., Mascia, B., Pretty, J. andRudd, M. (2011) ‘Methods for collaboratively identifying
research priorities and emerging issues in science andpolicy’, Methods in Ecology and Evolution, 2: 238–47.
Uskali, T. (2005) ‘Paying attention to weak signals: The key concept for innovation journalism’, Innovation Journalism, 2/11
<www.innovationjournalism.org> accessed 13 March 2012.
Van der Steen, M. (2008) ‘Ageing or silvering? Political debate
about ageing in the Netherlands’, Science and Public Policy,
35: 575–83.
Van Dijk, T. A. (2002) ‘Political discourse and political cognition’. In: Chilton, P. and Schäffner, C. (eds) Politics as Text
and Talk: Analytic Approaches to Political Discourse,
pp. 203–31. Philadelphia, PA: John Benjamins.
Van Rij, V. (2010a) ‘Joint horizon scanning: identifying
common strategic choices and questions for knowledge’,
Science and Public Policy, 37: 7–18.
——. (2010b) ‘Horizon scanning: Monitoring plausible anddesirable futures’. In: In‘t Veld, R. J. (ed.) Knowledge
Democracy: Consequences for Science, Politics, and Media,
pp. 227–240. Berlin/Heidelberg: Springer.
——. (2012) ‘Wild cards as future shakers and shapers’.
In: Giaoutzi, M. and Sapio, B. (eds) Recent Developments
in Foresight Methodologies. New York: Springer Verlag.