Proceedings World Geothermal Congress 2015
Melbourne, Australia, 19-25 April 2015

Methodology for Designing a Sustainability Assessment Framework for Geothermal Energy
Developments
Ruth Shortalla, Brynhildur Davidsdottira and Gudni AxelssonbaUniversity of Iceland, School of Engineering and Natural Sciences, Institute of Sustainability Studies, Sæmundargötu 2, 101
Reykjavík, Icelandb

Iceland Geosurvey (ISOR), Grensásvegur 9, 108 Reykjavík, Iceland
Ruth.shortall@gmail.com, bdavids@hi.is, gax@isor.is

Keywords: geothermal, sustainability, indicators, sustainability assessment
ABSTRACT
Geothermal projects have significant socio-economic and environmental impacts, both positive and negative. In order for energy
developments to contribute to sustainable developments they must result in positive impacts in all dimensions. Sustainability
assessments are valuable tools for policy- or decision-makers for making informed conclusions regarding policy effectiveness andprogress toward sustainable development. Currently available assessment frameworks are not suited to assessing geothermal
projects, thus a new, specialized framework is required. The methodology for developing a geothermal sustainability assessment
framework is described in this paper.
1. INTRODUCTION
1.1 Geothermal Energy and Sustainable Development
Energy usage worldwide is increasing. Global energy demand is predicted to grow by more than one-third by 2035, with China,
India and the Middle East accounting for 60% of the increase (International Energy Agency, 2012). The use of alternatives such as
geothermal energy is set to increase, since the world has only a finite supply of fossil fuels. As well as this, in order to combat
climate change and fulfill international agreements, low carbon energy sources such as geothermal energy are now being tapped ona larger scale. In 2008, geothermal energy represented around 0.1% of the global primary energy supply, but estimates predict thatit could fulfill around 3% of global electricity demand, as well as 5% of global heating demand by 2050 (Intergovernmental Panelon Climate Change, 2012).
Whilst energy is necessary for economic growth and is also vital for sustainable development, energy development also hasenvironmental and social impacts. Like any other energy source, geothermal energy developments can result in positive as well as
negative socio-economic and environmental impacts (UNDP, 2002). For example, geothermal projects can result in socio-economic
benefits particularly in developing countries and rural communities by improving infrastructure, or stimulating local economies.
Such projects may also act as a good source of baseload power for a region’s energy system. However, certain issues need to beaddressed as many geothermal energy developments have negative environmental impacts, including air and water pollution,
impacts on wildlife or landscapes.
The wide variety of available sustainability assessment frameworks highlights the ambiguity of the meaning of sustainable
development for different user groups, cultures and regions (Pinter, Hardi , & Bartelmus, 2005). Given the unique issues associatedwith geothermal energy projects, a specialized assessment tool is required to ensure that geothermal projects will be properly
guided into following best practices and result in positive impacts in all sustainability dimensions: environmental, social andeconomic.
1.2 Objective
The purpose of this paper is firstly to analyze the literature on means of developing sustainability indicators for energy
developments. Secondly, the steps for developing an assessment framework for geothermal energy projects are described. The
paper will illustrate the methods used in establishing an indicator framework in the Icelandic context and reflect on the learning
process therein.
2. BACKGROUND
Sustainability assessment is a means of showing the progress of development projects towards or away from sustainability.
Sustainability assessments are used for many different types of projects, including energy developments. Various assessment tools,
many of which involve the use of sustainability indicators, exist from the national level, to the local level (Pinter, Hardi , &Bartelmus, 2005).
Such indicators must provide a holistic view of sustainability. Furthermore, as well as indicators, sustainability criteria or goals arealso important for sustainability measurement. Such criteria and indicators should not be rigid but take account of the local context
as well as changes in opinions over time (Lim & Yang, 2009). In order to ensure this, broad stakeholder engagement is an essential
part of the indicator development process (Fraser et al., 2006).
Assessment frameworks range from overarching guidelines, such as the BellagioSTAMP principles to specific sustainability
indicator development approaches, such as the Pressure-State-Response (PSR)/ Driving Force-State-Response (DSR) framework or
1

Shortall, Davidsdottir and Axelsson
thematic approach (Shortall, Davidsdottir, & Axelsson, 2014). The International Institute of Sustainable Development’s
BellagioSTAMP principles identify desirable common patterns in sustainable development-related assessments.
The most widely used frameworks, especially for national indicator sets, are theme-based. In such frameworks, indicators aregrouped according to sustainability issue-areas or themes, which are chosen based on their policy-relevance. Theme-based indicator
sets allow decision-makers to link indicators to policies or targets (United Nations, 2007).
2.1 Sustainability Indicators and Energy
Several indicator frameworks exist to measure sustainable development in the context of energy developments. In 2005 theInternational Atomic Energy Agency (IAEA) in collaboration with several other bodies published guidelines and methodologies fora set of energy indicators for sustainable development (EISDs), emphasizing national self-examination (International Atomic
Energy Agency (IAEA), 2005). The EISDs were created to provide policy-makers with information about their country´s energy
sustainability. They are intended to provide an overall picture of the effects of energy use on human health, society and theenvironment and thus help in making decisions relating to choices of energy sources, fuels and energy policies and plans.
Athough not based on indicators as such, the International Hydropower Association published an assessment tool for hydropower
projects in 2006 (International Hydropower Association, 2006). The IHA-SAP assesses the strategic basis for a proposedhydropower project including demonstrated need, options assessment and conformity with regional and national policies and plans;
the preparation stage of a new hydropower project during which investigations, planning and design are undertaken; theimplementation stage of the new hydropower project during which preparations, construction, and other management plans andcommitments are undertaken and the operation of a hydropower facility with focus on continuous improvement (International
Hydropower Association, 2008).
The Gold Standard Foundation provides a sustainability assessment framework for new renewable energy or end-use efficiency
improvement projects. Projects must go through a number of steps, including a sustainability assessment, to become accredited with
the Gold Standard. These steps include a Stakeholder Consultation Process and development of a Sustainability Monitoring Plan,
which uses indicators of sustainable development relevant to the project. The aim of the Gold Standard is to promote investments inenergy technologies and energy management techniques that mitigate climate change, promote (local) sustainable development andare directed towards a transition to non-fossil energy systems (The Gold Standard Foundation, 2012).
The Commission for Sustainable Development (CSD) has produced guidelines for the creation of general sustainability indicators,
including energy indicators, at the national level (United Nations, 2007). In the EU, these indicators have been used in creating an
indicator framework, which includes the theme of Climate Change and Energy, to monitor implementation of the main EU
directives and other policy documents targeting sustainable energy development (European Commission, 2005).
The World Energy Council has also developed the Energy Sustainability Index, which looks at the impact of energy policies ofdifferent countries and ranks them in terms energy sustainability based on three core dimensions—energy security, social equity,and environmental impact mitigation. The index uses two types of indicator, energy performance indicators, covering supply anddemand, energy affordability and access; and contextual indicators, covering broader issues such as living standards and theeconomic and political conditions (World Energy Council, 2011).
Other renewable energy associations have attempted to improve sustainability assessment for energy projects. The World Wind
Energy Association (WWEA) have developed Sustainability and Due Diligence Guidelines (WWEA, 2005), for the assessment of
new wind projects, similar to those developed by the International Hydropower Association in their Sustainability Assessment
Protocol. These guidelines do not cover the operation stage of a wind energy project and do not provide a set of comprehensive
indicators. The WWF Sustainability Standards for Bioenergy (WWF, 2006) does not provide any indicators but does highlightsustainability issues in bioenergy and offer recommendations for its sustainable use. UN-Energy has also published a report with asimilar focus entitled Sustainable Bioenergy: A Framework for Decision-Makers (UN-Energy, 2007). However no indicators exist
for assessing the sustainability of geothermal power.
3. DEVELOPMENT METHOD
3.1 Overview
A sustainability assessment framework consists of a set of sustainability goals and indicators that allow monitoring of geothermal
projects during their entire life cycle. This section describes the methods used to carry out the first iteration of the indicator
development process in the Icelandic context.
The goals and indicators in this framework were developed using an iterative process for thematic indicator development(Davidsdottir, 2007), with the intention of carrying out further iterations in a number of different countries. One iteration consists ofchoosing sustainability goals and indicators guided by stakeholder input; calculating the indicators in a trial assessment of an
operational geothermal project and finally evaluating the indicators for suitability. The purpose of the iterative approach is to allow
the progressive refinement of the indicators following each iteration.
Guiding principles known as the Bellagio STAMP were incorporated into the entire development process. A literature review of the
impacts of geothermal energy projects on sustainable development (Shortall, Davidsdottir, & Axelsson, 2014) was carried out inorder to determine the boundaries of the system that the assessment framework was intended for. Following the literature review,
an initial, small group of stakeholders was gathered to critically review and a set of possible sustainability goals and indicators thatwere derived from the literature review. This set of goals and indicators provided a starting point for which further stakeholder
input would be sought later in the process.

2

Shortall, Davidsdottir and Axelsson
Stakeholder engagement methods were used at various points in the first iteration of the development process, from setting
sustainability goals to the choosing of sustainability indicators (Figure 1). The Icelandic stakeholder engagement process consistedof a pre-engagement “World Café” workshop and a Delphi survey. Each component of the development process is described indetail below.

Figure 1: Iterative Method of Indicator Development modified from Davidsdottir et al., 2007

3.2 Iterative Process
3.2.1 General Description of Method
An iterative approach (Davidsdottir et al, 2007) to indicator development was chosen because it lends itself well to the trialing of
the indicator set in several countries, allowing refinement of the indicators after each iteration. This was also intended to reducecountry or stakeholder biases, which could arise if stakeholders in only one country were consulted. The method (Figure 1) consistsof the following steps, which may be repeated as necessary, in an iterative fashion.
1.
2.
3.
4.
5.
6.
7.
8.

Definition of sustainability goals
Specification of dimensions
Selection of themes and sub-themes
Selection of indicators
Selection of aggregation function
Selection and calculation of weights (if needed)
Calculation of indicators
Reporting of indicators

3.2.2 Implementation of Method
The first four steps of the iterative process (Figure 1) required stakeholder input, which in this case was obtained through preengagement “World Café” workshops (Section 3.4.1) and the Delphi technique (Section 3.4.2). These methods are explained indetail in the sections below.
During the first four steps, the facilitators used personal expert judgment and stakeholder input to determine best and most suitableindicators, using as a guidance of the suitability criteria shown below. Once indicators were chosen, they were then calculated in atrial assessment on an existing geothermal energy project. By carrying out trial calculations, issues such as lack of data, thesuitability of reference values or responsiveness of the indicator were identified. The indicators were again evaluated for theirsuitability to their purpose against the following suitability criteria:-

Clear and unambiguous and able to show trends over time;
Responsive to changes in the environment and related human activities;
Relevant to assessing sustainable development progress;
Provide a basis for international comparisons;Have a threshold or reference value against which to compare it so that users are able to assess the significance of thevalues associated with it.
Theoretically well founded in technical and scientific terms
Based on international standards and international consensus about its validity to the extent possible
3

Shortall, Davidsdottir and Axelsson-

Lend itself to being linked to economic models, forecasting and information systems
Use data which is readily available or made available at a reasonable cost/benefit ratio;
Use data which is updated regularly or adequately documented and of known quality

3.3 Overarching Guidelines
3.3.1 General Description of Method
To guide the process of developing a sustainability assessment framework, the principles of the Bellagio group, known as the
Bellagio STAMP (Box 1) were used as overarching guidelines. The International Institute of Sustainable Development’s Bellagio
STAMP principles are a set of guiding principles designed to be applied when improving sustainability assessment systems andhave been widely adopted (IISD, 1997).
The principles are intended to serve as guidelines for the entire assessment process including the choice and design of indicators,
their interpretation and communication of the result. While the Bellagio STAMP principles identify desirable common patterns insustainable development-related assessments, they do not offer a detailed methodological approach required for the development of
an indicator set.
Box 1: List of Bellagio STAMP principles
1. Guiding Vision
Assessing progress towards sustainable development is guided by the goal to deliver well– being within the capacity of thebiosphere to sustain it for future generations.
2. Essential Considerations
Sustainability Assessments consider:- The underlying social, economic and environmental system as a whole and the interactions among its components.
- The adequacy of governance mechanisms.
- Dynamics of current trends and drivers of change and their interactions.
- Risks, uncertainties, and activities that can have an impact across boundaries.
- Implications for decision-making, including trade–offs and synergies.
3. Adequate Scope
Sustainability Assessments adopt:- Appropriate time horizon to capture both short and long–term effects of current policy decisions and human activities.
- Geographical scope ranging from local to global.
4. Framework and Indicators
Sustainability Assessments are based on:- A conceptual framework that identifies the domains that core indicators have to cover.
- The most recent and reliable data, projections and models to infer trends and build scenarios.
- Standardized measurement methods, wherever possible, in the interest of comparability.
- Comparison of indicator values with targets and benchmarks, where possible.
5. Transparency
The assessment of progress towards sustainable development:- Ensures the data, indicators and results of the assessment are accessible to the public.
- Explains the choices, assumptions and uncertainties determining the results of the assessment.
- Discloses data sources and methods.
- Discloses all sources of funding and potential conflicts of interest.
6. Effective Communication
In the interest of effective communication, to attract the broadest possible audience and to minimize the risk of misuse,
Sustainability Assessments:- Use clear and plain language.
- Present information in a fair and objective way, that helps to build trust.
- Use innovative visual tools and graphics to aid interpretation and tell a story.
- Make data available in as much detail as reliable and practical.
7. Broad Participation
To strengthen their legitimacy and relevance, sustainability assessments should:- Find appropriate ways to reflect the views of the public, while providing active leadership.
- Engage early on with users of the assessment so that it best fits their needs.
8. Continuity and Capacity
Assessments of progress towards sustainable development require:- Repeated measurement.
- Responsiveness to change.
- Investment to develop and maintain adequate capacity.
- Continuous learning and improvement.
4

Shortall, Davidsdottir and Axelsson
3.3.2 Implementation of Method
Below is an explanation of the development process’ adherence to the Bellagio principles (Box 1).
1. Guiding Vision
A set of sustainability goals were defined by the stakeholder group. The goals are intended to serve as an overall set of guiding
principles for the assessment of geothermal projects.
2. Essential Considerations
All sustainability dimensions (social, economic, environmental) were covered by the chosen goals and indicators, following an
extensive literature review and interviews with experts regarding the social, economic and environmental impacts of geothermal
projects. Detailed studies of individual power projects provided a good insight into the underlying systems and interactions withinthem. The governance of monitoring bodies is considered, in that the quality of environmental monitoring is assessed. Further
assessment of the monitoring bodies would be necessary to provide greater insight into governance mechanisms, but this is
somewhat outside of the scope of the assessment framework and would also be time consuming and costly.
An extensive literature review and the input of experts regarding the social, economic and environmental impacts of geothermal
projects helped to identify the dynamics of current trends and drivers of change and their interactions as well as risks anduncertainties. Furthermore, the detailed studies of individual power projects provide a good insight into the underlying systems andinteractions within them.
3. Adequate Scope
The assessment framework is intended to be used in such a way as to build up time series data for the set of indicators. Qualitative
information alongside the indicators can provide answers to questions regarding the short term implications of policy decisions. The
indicators that were developed cover issues at local and national level. For instance, indicators could measure greenhouse gas
emissions, which have a global impact, or resettlement of communities, which is a local issue.
4. Framework and Indicators
The sustainability goals in this framework provide a clear picture of the key issues that must be covered by the sustainability
indicators. Indicators that were chosen by the stakeholders were evaluated for suitability against a set of criteria outlined in Section
3.2.2. According to these criteria, only indicators for which recognized measurement standards exist are deemed suitable for use.
Each indicator has a corresponding reference value, where it is available.
5. Transparency
It is intended that the results of any sustainability assessment carried out using the framework would be made available to the
public online. Each sustainability assessment carried out in this study is reported along with supplementary information for eachindicator, which explains any assumptions or uncertainties. All data sources and methods are disclosed alongside any indicators thatare reported.
6. Effective Communication
During the indicator development process, stakeholders with varied backgrounds were consulted. Any indicators that were unclear
to the majority of people were removed, to ensure that the broadest audience would understand them.
Each indicator has a benchmark that is, as much as possible, international recognized, which means that subjective opinions ofassessors are not required when calculating indicators. Indicators are reported alongside supplementary information, giving a
background story for each issue. This allows the assessor to report on positive steps that may be underway to improve on indicatorsthat are currently scoring negatively. Indicators are presented in graphic format where possible and supplementary information in
the form of photos or graphs is also provided where appropriate.
7. Broad Participation
The World Café and Delphi technique were chosen as stakeholder engagement methods. Stakeholders have been consulted since
the very beginning of the process. Stakeholders include both experts and grassroots representatives with local knowledge.
8. Continuity and Capacity
The assessment framework is intended for repeated use in assessing projects, in order to build up time series data to show progress
towards or away from sustainability over time. Indicators have been chosen based on their responsiveness. The iterative indicator
development process (Section 3.2) implicitly requires regular reassessment of the suitability of the framework to its purpose, due tothe dynamic nature of sustainable development. To this end, further ongoing work will be required to continuously assess therelevant sustainability issues, consult with stakeholders and update the framework when necessary.
3.4 Stakeholder Engagement
Stakeholder engagement is the process used by an organization to engage relevant parties for a clear purpose to achieve accepted
outcomes. It is now also seen as a type of accountability mechanism. Good stakeholder engagement should clearly define the scope,
have an agreed decision making process, focus on issues material to the stakeholders, create opportunities for dialogue, betransparent, have a process appropriate to the stakeholders engaged, be timely, flexible and responsive (UK Institute of Social andEthical Accountability, 2011). Stakeholder engagement techniques have been used to address sustainability issues in diverse
sectors, including mining (Azapagic, 2004), forestry (Sharma & Henriques, 2005), transportation (Mihyeon Jeon & Amekudzi,2005), aviation (Amaeshi & Crane, 2006) and environmental management (Reed M. S., 2008).
5

Shortall, Davidsdottir and Axelsson
Stakeholders are generally defined as persons or groups who are directly or indirectly affected by a development project, as well as
those who may have interests in a project and/or the ability to influence its outcome, either positively or negatively (International
Finance Corporation, 2007). Another definition of stakeholders is as follows:“Stakeholders are not just members of communities or non-governmental organisations. They are those individuals, groups ofindividuals or organisations that affect and/or could be affected by an organisation’s activities, products or services and associatedperformance with regard to the issues to be addressed by the engagement.” (UK Institute of Social and Ethical Accountability,
2011)
Sustainability is an interdisciplinary field, which requires knowledge of the social, economic and environmental impacts of adevelopment. Yet often stakeholders from different fields have little interaction with each other, or little knowledge of other fields.
In geothermal energy development, this is also the case and no framework for the assessment of geothermal projects in the context
of sustainability has been developed to date. The wide-ranging topic of geothermal sustainability therefore requires the combined
expert input of a varied group of experts, obtained by using an appropriate stakeholder engagement technique.
For a geothermal project, stakeholders may include locally affected communities or individuals and their formal and informal
representatives, the geothermal industry, national or local government authorities, politicians, religious leaders, civil society
organizations and groups with special interests, the academic community, or other businesses, such as suppliers or those that may
use the geothermal power. For example, the risk of induced seismicity can become a real issue. In this regard, the concerned oraffected members of the public would need to be included when gathering a group of stakeholders.
The interest that each of these different individuals or groups has in a geothermal project tends to vary. For example, some people
would be directly affected by the potential environmental or social impacts of a project. Others may live far away or in anothercountry altogether, but may still wish to communicate their concerns or suggestions regarding the project. Some may have great
influence over the project, such as government regulators, political or religious leader or community activists. Some may havespecialized knowledge that can be helpful to the project. Others may represent “voiceless” things or groups, such as ecosystems,
future generations or indigenous peoples (International Finance Corporation, 2007).
A stakeholder mapping exercise was carried out before the engagement process to identify individuals or organizations that would
potentially be impacted by or have an interest or impact in the sustainable operation of geothermal projects. Stakeholders werechosen based on a number of characteristics, as recommended by the Australian government stakeholder engagement practitioner
handbook (Australian Government, 2008):
Responsibility: people to whom a hypothetical geothermal development would have responsibility to, such as the local community
or general public, community representatives or NGOs, environmental organisations, local businesses, future generations.
Proximity: those people who would have most interaction with a hypothetical geothermal project, such as: the geothermal industry
itself, researchers, governments, local communities.
Dependency: those who depend directly or indirectly on a geothermal project, such as: power companies, financiers, potential
users of the energy, the local community, local businesses.
Representation: those people that represent a constituency impacted by geothermal projects, such as: NGOs representing theenvironment or “voiceless” things such as landscape, geothermal features, delicate ecosystems, forests and so on; indigenous
peoples representatives, other community group representatives such as local authorities, trade unions, or local leaders.
Policy and Strategic intent: those people to whom geothermal projects (or companies) address their policy or value statements,
such as: NGOs, activists, community groups, financiers.
3.4.1 World Café Technique
The World Café is described as a powerful social technology for engaging people in conversations that matter. The World Café
approach is based on the understanding that conversation is the core process that drives personal, business, and organizational life
(Brown & Isaacs, 2005).
The World Café method has the advantage of being flexible and easily adapted to suit the needs of the group. Generally,
participants meet in a Café style setting, seated at tables where they hold conversations exploring a particular question, moving
between tables at prescribed time intervals (Brown & Isaacs, 2005). In this way, the method allows diverse information to begathered as well as the sharing of ideas and insight. Participants learn collectively, allowing the group to find solutions to the given
question, based on their new insights (Brown & Isaacs, 2005).
The disadvantages of using the World Café technique, as for any type of stakeholder group meeting (Thompson, 2007), include thepotential for conflict in a group setting, due to differences in opinion of stakeholders. The success of the World Café will depend on
the participants present. Furthermore, the cost of organizing and facilitating the workshop may be prohibitive and participants mayneed to travel long distances to reach the location.
Few studies using the World Café method exist in the literature, however the method has, for example, been applied in social
science research in order to help develop a culture of enquiry among practitioners in social service (Fouché & Light, 2011). Further
examples of its use appear in fields such as nursing (Brooma, Brady, Kecskes, & Kildea, 2013).

6

Shortall, Davidsdottir and Axelsson
3.4.2 Delphi Technique
The Delphi technique is used for policy, decision, and goal setting, when consensus is required from a group of stakeholders withwidely divergent opinions or backgrounds (Lim & Yang, 2009). The technique uses a structured format to elicit opinions andpotential consensus among a group of stakeholders or experts in their field. As a result, the method has become increasingly
popular and widely used in technology, education and other fields (Lim & Yang, 2009).
The Delphi technique also suits applications with the characteristics listed below (Thompson, 2007).
-

The problem does not lend itself to precise analytical techniques but can benefit from subjective judgments on acollective basis.
The individuals needed to contribute to the examination of a broad or complex problem have no history of adequate
communication and may come from diverse backgrounds and have diverse experience or expertise.
More participants are needed than can effectively interact in a face-to-face exchange.
Time and cost make frequent group meetings impossible.
Face-to-face meetings can be made more effective by using a supplementary group communication technique.
Serious disagreements exist among individuals or the issues are so politically sensitive that the communication process
must be refereed and/or anonymity assured.
The diversity of the participants must be preserved to assure validity of the results, i.e., avoid domination of the processby high numbers of people or by strong personalities (the "bandwagon effect").

The Delphi technique has been used successfully in developing indicators of sustainability in diverse fields such as roadinfrastructure projects (Lim & Yang, 2009), ecotourism (Barzekar, 2011) and communities (Hai, Hai, Khoa, & Hens, 2009). The
main steps taken by the facilitators in the Delphi technique (Linstone & Turoff, 2002; Barzekar, 2011; Lim & Yang, 2009) are as
follows:
1. Assemble/choose participants.
2. Present list of goals and indicators to be rated and added to by the group.
3. Rate and comment on each item.
4. Record each participant’s ratings and modify the list based on ratings or comments.
5. Return the statistics to all participants.
6. Rate and comment on items again.
7. Repeat the process (step 3-6)
8. Select the highest rated goals and indicators (those with the highest mean score) to use in final assessment framework
3.4.3 Implementation of Methods
As per the recommendations of the Bellagio STAMP (IISD, 2012), a diverse group of stakeholders was selected to contribute to theprocess of developing the sustainability assessment framework. The group consisted of participants from diverse backgrounds,
from government to industry to NGOs.
Stakeholder engagement is an integral part of the iterative indicator development process. Stakeholders have an influence throughtheir comments during the pre-engagement “World Café” workshop and the Delphi process, from the choice of sustainability goals
and indicators (Figure 1). Their input also defines the scope of the assessment itself by identifying the most important sustainability
issues that will be considered.
The World Café workshop technique was used as a starting point or pre-engagement method in Iceland, in order to gather
stakeholder input on potential sustainability goals and indicators for geothermal energy projects, before holding a full-fledged
Delphi process. Following the World Café in Iceland, the following steps were taken to refine the indicator set:-

Discard indicators that were voted to have low or no relevance (attempt to have less indicators overall)
Discard indicators that are hard to understand, even with supplementary information
Include new indicator suggestions, if they fulfill the criteria for good indicators.

The Delphi technique was chosen as the main stakeholder engagement method as it was considered the best technique to use given
the circumstances. Firstly, around 70 stakeholders were invited to take part in the Icelandic Delphi. This would have been too largea group to facilitate the effective extraction of opinions in a short time frame.
It would also have been difficult and costly to arrange repeated face-to-face meetings with the number of people involved. Even
though a pre-engagement World Café workshop was organized, it was not possible to arrange similar meetings for all three Delphi
rounds. However the Delphi technique was presented and explained in the pre-engagement workshop, leading to a great
understanding of the process by participants.
Furthermore, the Icelandic stakeholder group consisted of members of government and other institutions with differing views.
Since Iceland is a small community, the Delphi technique was chosen as a way to circumvent political differences that could arise
7

Shortall, Davidsdottir and Axelsson
in a group setting. Twenty-three stakeholders attended the pre-engagement “World Café” workshop in Reykjavik and the Delphi
had a good response (see Table 1 below).
Table 1: Response rates for Delphi rounds in Iceland

Round 1Round 2Round 3InvitedResponse Rate

70
70
70

47%
23%
16%

Responded(Partial / Complete)
33 (11/22)
16 (3/13)
11 (2/9)The list of goals and indicators from the pre-engagement workshop were presented to participants using online survey tools. Scores
were allocated by participants on a scale of 1-5, according to the perceived relevance of the sustainability goal or indicator. Levelsof agreement and consensus among stakeholders for individual goals and indicators differed, however participants agreed on a final
list of ten sustainability goals (Table 2) and twenty-four sustainability indicators. This is not intended to be a detailed description of
the results, but serves to illustrate how the method was used.
Table 2: Sustainability Goals
Goal

Score after
Round 2
4.72
4.68
4.65
4.18
4.12
4.04
4.12
4.5
4.4
4.4

GOAL 1 - Renewability
GOAL 2 - Water Resource Usage
GOAL 3- Environmental Management
GOAL 4 – Efficiency
GOAL 5 - Economic Management & Profitability
GOAL 6 - Energy Equity
GOAL 7 - Energy Security & Reliability
GOAL 8 - Community Responsibility
GOAL 9 - Research and Innovation
GOAL 10 - Dissemination of Knowledge

Score after
Round 3
4.55
4.09
4.45
3.64
4.09
3.64
4.00
4.00
4.18
4.27

The five highest scoring indicators after three Delphi rounds are shown in Table 3. These are the indicators that participantsconsidered most relevant to geothermal sustainability.
Table 3: Highest scoring indicators after three Delphi rounds
Indicator

Mean Score

Resource reserve capacity ratio of the geothermal resource

4.22

Utilization efficiency

4.22

Estimated productive lifetime of geothermal resource

4.56

Water quality

4.67

Air quality

4.78

The five lowest scoring indicators are shown in Table 4 below. These are the indicators that participants considered least relevant togeothermal sustainability. These five indicators all received the same low score of 3.33.
Table 4: Lowest scoring indicators after three Delphi rounds
Indicator

Mean Score

Average Income Levels in Project-Affected Communities

3.33

EBITA ratio per project

3.33

Expenditure on heat and electricity as a percentage of household income

3.33

Percentage of energy company expenditure given to R&D per year

3.33

Percentage of renewables in total energy supply nationally

3.33

4. DISCUSSION AND CONCLUSIONS
The meaning of sustainable development depends on a group or society’s opinions and values regarding issues that are important tothem. These opinions and values will determine which goals should be pursued and what should be measured (Meadows, 1998;
Shields, Šolar, & Martin, 2002). Sustainability is also a shifting, evolving and subjective concept (Mog, 2004). This means we must
constantly reassess the definition of sustainable development for a given context and adjust how we measure it when necessary.

8

Shortall, Davidsdottir and Axelsson
Indicators of sustainability are only likely to be effective if they provide users and the public with meaningful information they canrelate to. Users like policy- and decision-makers will be in a better position to set attainable policy goals if they understandenvironment-society interactions well, and this is all the more likely to happen if indicators are derived from a participatory
process, as they will reflect the objectives and values of the public (Shields, Šolar, & Martin, 2002). Whilst developing asustainability assessment framework for geothermal energy projects, the goals and most important sustainability issues measured by
the indicators chosen reflect the subjective opinions of the group of stakeholders, even though precise, analytical techniques may be
used to calculate the indicators once they are chosen. The very act of choosing sustainability goals and indicators can be an adaptive
learning process for all parties involved (Reed, Fraser, & Dougill , 2006).
In order to be influential, consensus must exist among policy actors that the indicators are legitimate, credible and salient (Cash etal., 2003). This means that the indicators must answer questions that are relevant to the policy actor, as well as providing ascientifically plausible and technically sound assessment. To be legitimate, the indicators must be perceived to be developed
through a politicaly, socially and ethically acceptable procedure. The list of indicators produced in this first iteration has been
critically evaluated through a stakeholder engagement process in Iceland and also against a set of theoretical criteria to determine
their suitability to their purpose. The results of the Delphi show the highest priority goals among Icelandic stakeholders to beresource renewability and environmental management, whereas the goals with lowest priority are those of energy efficiency andenergy equity.
Based on this first iteration of the indicator development process in Iceland, it is clear that inevitably, each geothermal energy
project will face unique sustainability challenges, due to the differing environmental and socio-economic setting in which it isfound. By carrying out further iterations of the indicator development process, we suggest that the final assessment framework
produced will be more likely to take into account the diverse and unique circumstances surrounding geothermal developments.
Further iterations of the indicator development process will also produce better, more refined indicators and further study may
reveal issues that may have been neglected previously.
4.1 Conclusion
This paper describes the development of a sustainability assessment framework for geothermal energy projects, using the input of
stakeholder groups and internationally recognized methods. The lessons learned from this iteration in the Icelandic context will beapplied to further iterations in New Zealand and Kenya. Following these iterations, more insights on the assessment framework will
become apparent and allow for a more comprehensive evaluation of its suitability. We suggest that these indicators have thepotential to inform policy decisions through either conceptual or instrumental use.
ACKNOWLEDMENTS
We gratefully acknowledge the GEORG geothermal cluster as our project sponsor, without whom this project would not have beenpossible. This project had its beginnings in 2009 as a Masters thesis at the University of Iceland, which was generously sponsoredby Orkustofnun (National Energy Authority of Iceland), Landsvirkjun Power and RANNÍS (Icelandic Research Fund).
We also acknowledge the support of University of Iceland, University of Auckland, Reykjavik Energy (Orkuveita Reykjavikur) andthe Kenya Electricity Generating Company Ltd. (KenGen). Furthermore, we sincerely thank the numerous stakeholders in Iceland,
New Zealand and Kenya and the UNU Fellows that took part in our stakeholder process.
REFERENCES
Australian Government. (2008). Stakeholder Engagement Practitioner Handbook. Department of Immigration and Citizenship.
National Communications Branch of the Department of Immigration and Citizenship.
Azapagic, A. (2004). Developing a framework for sustainable development indicators for the mining and minerals industry. Journalof Cleaner Production , 12 (6), 639–662.
Amaeshi, K. M., & Crane, A. (2006). Stakeholder engagement: a mechanism for sustainable aviation. Corporate Social
Responsibility and Environmental Management , 13 (5), 245-260.
Barzekar, e. a. (2011). Delphi technique for generating criteria and indicators in monitoring ecotourism sustainability in Northern
forests of Iran: Case study on Dohezar and Sehezar Watersheds. Folia Forestalia Polonica , 53 (2), 130-141.
Brown, J., & Isaacs, D. (2005). The World Cafe: Shaping Our Futures Through Conversations That Matter. Berrett-Koelher.
Brooma, M., Brady, B., Kecskes, Z., & Kildea, S. (2013). World Café Methodology engages stakeholders in designing a Neonatal
Intensive Care Unit. Journal of Neonatal Nursing , 19 (5), 253–258.
Cash, D. W. (2003). Knowledge systems for sustainable development. Proceedings of the National Academy of Sciences , 80868091.
European Commission. (2005). Measuring progress towards a more sustainable Europe. Sustainable Development Indicators for theEuropean Union. Luxemburg: Office for Official Publications of the European Communities.
Davidsdottir, B. (2007). Measuring Sustainable Energy Development: the development of a three dimensional index - the SEE
index. In Frontiers in Environmental Valuation and Policy. Cheltenham, UK: Edward Elgar.
Fouché, C., & Light, G. (2011). An Invitation to Dialogue: 'The World Café' In Social Work Research. Qualitative Social Work , 10(28).
Fraser, E. D., Dougill , A. J., Mabee, W. E., Reed, M., & McAlpine, P. (2006). Bottom up and top down: Analysis of participatory
processes for sustainability indicator identification as a pathway to community empowerment and sustainable environmental
management. Journal of Environmental Management (78) . Elsevier.
9

Shortall, Davidsdottir and Axelsson
IISD. (1997). Assessing Sustainable Development: Principles in Practice. IISD.
IISD. (2012). Measuring and Assessing Progress Identifying approaches, patterns and practices. Retrieved November 4, 2012,
from International Institute of Sustainable Development: http://www.iisd.org/measure/principles/progress/
Intergovernmental Panel on Climate Change. (2012). Renewable Energy Sources and Climate Change Mitigation: Special Reportof the Intergovernmental Panel on Climate Change. Intergovernmental Panel on Climate Change. New York: Cambridge
University Press.
International Atomic Energy Agency (IAEA). (2005). Energy Indicators for Sustainable Development: Methodologies andGuidelines. Vienna: International Atomic Energy Agency (IAEA), United Nations Department of Economic and Social Affairs(UNDESA), International Energy Agency (IEA), Eurostat, European Environment Agency (EEA), .
International Energy Agency. (2012). World Energy Outlook Executive Summary 2012. International Energy Agency.
International Finance Corporation. (2007). Stakeholder Engagement: A Good Practice Handbook for Companies Doing Business in
Emerging Markets . Washington D.C.: International Finance Corporation.
International Hydropower Association. (2008). IHA Connect Issue 3 (3) .
International Hydropower Association. (2006). Sustainability Assessment Protocol .
Hai, L. T., Hai, P., Khoa, N., & Hens, L. (2009). Indicators for Sustainable Development in the Quang Tri Province, Vietnam. J
Hum Ecol, , 27 (3), 217-227.
Linstone, H. A., & Turoff, M. (2002). The Delphi Method: Techniques and Applications. (H. A. Linstone, & M. Turoff, Eds.) New
Jersey Institute of Technology.
Lim , S., & Yang, J. (2009). A Delphi study on the critical sustainability criteria and indicators for Australian road infrastructure
projects. In A. van den Dobbelsteen (Ed.), Proceedings of the 3rd CIB International Conference on Smart and Sustainable
Built Environments (pp. 1-7). Delft: Delft University of Technology.
Meadows, D. (1998). Indicators and information systems for sustainable development. A report to the Balaton Group.
Mihyeon Jeon, C., & Amekudzi, A. (2005). Journal of Infrastructure Systems. Addressing Sustainability in Transportation Systems:
Definitions, Indicators, and Metrics. , 11 (SPECIAL ISSUE: SUSTAINABILITY OF TRANSPORTATION AND OTHERINFRASTRUCTURE SYSTEMS ), 31-50.
Mog, J. M. (2004, December). Struggling with Sustainability—A Comparative Framework for Evaluating Sustainable
Development Programs. World Development , 32 (12) , 2139–2160. Elsevier.
Pinter, L., Hardi , P., & Bartelmus, P. (2005). Indicators of Sustainable Development: Proposals for a Way Forward. IISD.
Sharma, S., & Henriques, I. (2005). Stakeholder influences on sustainability practices in the Canadian forest products industry.
Strategic Management Journal , 26 (2), 159-180.
Shields, D., Šolar, S., & Martin, W. (2002). The role of values and objectives in communicating indicators of sustainability.
Ecological Indicators , 2 , 149–160. Elsevier.
Shortall, R., Davidsdottir, B., & Axelsson, G. (2014). (Submitted Paper) Geothermal Energy for Sustainable Development: A
Review of Sustainability Impacts and Assessment Frameworks. Renewable & Sustainable Energy Reviews . Elsevier.
Reed, M. S. (2008). Stakeholder participation for environmental management: A literature review. Biological Conservation , 141(10), 2417–2431.
Reed, M., Fraser, E. D., & Dougill , A. J. (2006). An adaptive learning process for developing and applying sustainability indicatorswith local communities. Ecological Economics , 59 , 406–418.
The Gold Standard Foundation. (2014, March). Retrieved 2014, from http://www.cdmgoldstandard.org/about-us/who-we-are
The Gold Standard Foundation. (2012). The Gold Standard Toolkit 2.2. Geneva, Switzerland: The Gold Standard Foundation.
Thompson, K. (2007, April 27). Delphi collective group intelligence tool: powerful and free. Retrieved October 21, 2012, frombioteams.com: http://www.bioteams.com/2007/04/27/delphi_collective_group.html
UK Institute of Social and Ethical Accountability. (2011). Stakeholder Engagement Standard AA1000. AccountAbility.
UN-Energy. (2007). Sustainable Bioenergy: A Framework for Decision Makers. UN-Energy.
UNDP. (2002). Energy for Sustainable Development: A Policy Agenda. New York: UNDP.
United Nations. (2007). Indicators of Sustainable Development: Guidelines and Methodologies, Third Ed. United Nations.
WWEA. (2005). Sustainability and Due Diligence Guidelines. WWEA.
WWF. (2006). Sustainability Standards for Bioenergy. WWF.
World Energy Council. (2011). Policies for the future 2011: Assessment of country energy and climate policies . London: World
Energy Council.

10