GraSPP-DP-E-08-004 and SEPP-DP-E-08-003

Electricity System Diversity
in the UK and Japan:
A Multicriteria Diversity Analysis

Go Yoshizawa
Andy Stirling
Tatsujiro Suzuki
November 2008

GraSPP Discussion Paper E-08-003

SEPP Discussion Paper E-08-003

GraSPP-DP-E-08-004 and SEPP-DP-E-08-003

Electricity System Diversity in the UK and Japan:
A Multicriteria Diversity Analysis
Go Yoshizawa 1)
Andy Stirling 2)
Tatsujiro Suzuki 3)

November 2008

1)

2)

3)

注：

Project Researcher
University of Tokyo, Graduate School of Public Policy
7-3-1 Hongo, Bunkyo, Tokyo 113-0033 JAPANE-mail: g-yoshizawa@pp.u-tokyo.ac.jp
Professor Fellow
University of Sussex, SPRU
Freeman Centre, Falmer, Brighton, East Sussex BN1 9QE UK
E-mail: A.C.Stirling@sussex.ac.uk
Visiting Professor
University of Tokyo, Graduate School of Public Policy
7-3-1 Hongo, Bunkyo, Tokyo 113-0033 JAPANE-mail: tatsu@pp.u-tokyo.ac.jp

著者の所属、連絡先はいずれも執筆当時のものです。
本稿に関するお問い合わせは、東京大学公共政策大学院寄付講座「エネルギー・地球環
境の持続性確保と公共政策」（略称 SEPP）（〒113-0033 東京都文京区本郷 7-3-1 035841-1324 sepp@pp.u-tokyo.ac.jp）までお願いします。

Table of Contents

ACKNOWLEDGEMENT

4

1. INTRODUCTION

5

2. CONCEPTS

9

3. METHODOLOGY

133-1. Introduction

13

3-2. Diversity Analysis

17

4. ANALYSIS

18

4-1. Interview Results and Output Figures18

4-2. Two-Dimensional Scaling

39

4-3. Analysis of Disparity Structures45

4-4. Sensitivity Analysis

53

5. CONCLUSION61

APPENDIX 1. THE MULTICRITERIA MAPPING (MCM) PROCESS

64

Stage 1. Defining the Options

64

Stage 2. Defining the Criteria

65Stage 3. Scoring

67

Stage 4. Weighting

70

APPENDIX 2. EXCEL MACRO PROGRAM AND SPREADSHEETS

75

1. Setting Basic Parameters (Basic Sheet)77

2. Identifying Option Tranches (Options Sheet)80

2

3. Characterising and Weighting Disparity Attributes (Disparity Sheet)83

4. Illustrating Option Disparities (Dendrogram Sheet)86

5. Defining Option Interactions (Interactions Sheet)87

6. Exploring portfolio sensitivities (Sensitivities sheet)

88

APPENDIX 3. MATLAB PROGRAM OUTPUT FIGURES

93

REFERENCES

99

3

Acknowledgement
We gratefully acknowledge the support of the ESRC-funded Sussex Energy Groupat SPRU – Science and Technology Policy Research, University of Sussex andSustainable Energy/Environment & Public Policy (SEPP), University of Tokyo,
without which this international collaboration project would not have beenfeasible.
This pilot study would not have been possible without the exceptional patienceand attention offered by nine expert interviewees. It is unfortunate that
understandable requirements for anonymity mean that this acknowledgement
may not be extended on a personal basis. But we are very grateful to these leading
figures in energy policy and appraisal debates in both the UK and Japan, for thetime and effort that they expended in support of this exercise. We are deeply
appreciative of the generosity with which these nine individuals shared with us
the fruits of their expertise and experience.
We also would like to thank Gordon MacKerron, Jim Watson for their usefulcomments on the draft version of this working paper.

4

1. Introduction
There exists a growing number of studies, exploring ways to identify ‘optimal’
future portfolios of possible energy options. Such is the complexity, dynamism,
uncertainty and context-sensitivity in this field, that it has always been a difficult
task to justify any single apparently precise set of analytical results. Many of themost important strategic issues implicated in such analysis are highly
judgemental, often ambiguous and frequently controversial. Although techniques
proliferate, aims at ostensibly definitive quantitative analysis are especiallychallenging. As attention extends from historic fuel prices, it becomes ever moredifficult to justify apparently precise prescriptive conclusions. Now, as neverbefore, a diverse array of important strategic appraisal criteria have come into
view, under which it is difficult indeed to claim that probabilities of contending
possible future trends are fully addressed in past experience. Examples of such
issues include long-run economic competitiveness, supply stability, technology
change, resource depletion, climate change, social acceptance, and the many
disparate aspects of environmental sustainability. Although heroic attempts arestill made to assert precise prescriptive findings, these are often achieved at thecost of multiple (contestable but often invisible) ‘framing assumptions’ (Saltelli,2001). Quite apart from the substantive issues above, for instance, there typicallyremain many questions, even over the nature of energy diversity itself: what are
its characteristics, why is it desirable and what are the trade-offs with otheraspects of performance? Taken together, it does seem that these daunting
complications conspire to undermine any great confidence in the results that may
be obtained for ‘optimal’ future energy mixes.
These difficulties are, however, not grounds for despair. Although these kinds ofchallenges may preclude confidence in claims at transcendent ‘definitive objective’conclusions, they need not detract from the value of rigorous, systematic – evenquantitative – analysis. This involves explicit treatment of the divergent ‘framing
assumptions’ that emerge under alternative (equally valid) technical and expert
perspectives. Without assuming that any single (or aggregate) view presents a
final picture, careful analysis of contending judgements, uncertainties, issues and5assumptions can ‘map’ the alternative possible conclusions. In the process, many
initially apparently viable outcomes may be revealed as unsupported under any
reasonable assumptions. Attention can thereby reduce to a relatively narrow fieldof possible energy mixes, which – though they may themselves be quite diverse –do not rely on such volatile or idiosyncratic assumptions. Such ‘plural andconditional’ analysis is all the more robust, for being transparent about its own
dependencies and sensitivities (Stirling, 2008a). Using an innovative frameworkfor heuristic energy appraisal, this is the kind of analysis that is undertaken in
the present pilot study (Stirling, 2008b).
Nowhere are these challenges more acute than in the evolving energy policies of
the UK and Japan. Here, as in other countries, it has long been widely acceptedthat diversification of energy sources is a crucial means to address a broad range
of issues and hedge a variety of risks and uncertainties. In Japan, the Energy
Policy Basic Plan of 2002 presents a guideline both for resource diplomacy andenergy research and development. This stipulates that policies have to be madebased on a diversification of supply sources to secure stable availability of energy.
In the UK, a succession of government energy strategy documents and a recent
White Paper also highlight the central role of energy diversity (PIU, 2001; DTI,
2003). In both countries, this background requires that energy policy appraisal
make serious efforts systematically to address the complexities that are
implicated in contemplating current and future energy options.
For its part, the Japanese government admits that high level policy assessment ofthese kinds of issues has displayed relatively low performance. They ascribe thisdifficulty to methodological immaturity and observe that appropriate techniques
for analysis and assessment towards credible implementation are not developedand effectively deployed for the purposes of policy administration (CSTP, 2005). In
the UK, attempts have been made in the past to address some of the complexities
of energy diversity (DTI, 1995). Indeed, this analysis is based directly on
antecedents of the present analytical framework (Grubb et al., 2007). But, for themost part, recent UK policy appraisal has also stopped short of any explicit orsystematic effort at analysing the implications of contending views of energy
6diversity. An illustration of this situation lies in the UK Government’s treatment
of diversity in the official energy sector indicators 2008 (BERR, 2008: 20-22).
Although it is positive that diversity is addressed systematically at all, theparticular indicator that is used (the Shannon-Wiener index – Stirling, 1994)
addresses only a subset of the relevant aspects of diversity (see below) and ishighly sensitive to subjective categories (Stirling, 1998). These issues are
discussed in more detail elsewhere (Stirling, 2008b) and in Section 2 below.
It is against this background that the conclusion might be drawn that – both in
the UK and Japan – there exists considerable scope for experimental studies totest new frameworks for the appraisal of energy diversity. Recognising the serious
indeterminacies addressed above, there is a particular premium on interactive
studies, based on in-depth engagement with a diverse array of specialist
perspectives in each energy policy arena. Rather than relying on a single
analytical perspective, this will allow resolution of a ‘map’ of contending possible
expert judgements. It is only in this way that we may hope to develop novel
methodologies to become more mature and robust, and inform more rigorous,
legitimate and accountable policy discourses in this area. It is with this general
aim in mind, that the present pilot study therefore aims to apply and test a novel
heuristic approach to energy diversity analysis (Stirling, 2008b). In the process,the study will develop and refine this framework in a number of significant ways.
The result will be the first empirical analysis in either Japan or the UK of aconcept of energy diversity that addresses the full array of strategic issues
identified above. The parallel examination of perspectives from the two quite
radically divergent policy contexts in these two countries will provide an
additional comparative dimension to the exercise.
In setting out to achieve this, the present working paper is a final product of theinternational

collaboration

project

between

SEPP(Sustainable

Energy/Environment & Public Policy, University of Tokyo) and SPRU (Scienceand Technology Policy Research, University of Sussex) in 2007/08. The specificobjectives of the project were:7

(1) to develop a computer program for a practical interview methodology
integrating two quantitative strategic appraisal techniques (multicriteria
mapping and diversity analysis – both developed in at SPRU at the University of
Sussex (Stirling, 1997; 1998));(2) to apply and test this integrated package for multicriteria diversity analysis ina pilot study of the UK and Japanese energy policy contexts, involving a wide
range of relevant actors such as government policymakers, industrial engineers,
university scientists and NGO leaders; and(3) to analyse the resulting energy portfolios comparatively across perspectives
and national contexts and identify social and policy implications of thesimilarities and differences of their portfolios of a kind that may warrant further
research or policy analysis.
The basic outline of this report is as follows. Chapter 2 offers a brief overview ofmulticriteria diversity analysis by looking back the concept of diversity based ontheexisting

literature.

Chapter3

introducesthemethods

underlying

multicriteria diversity analysis. It first explains the basic parameters of this pilot
study and details of the interview process. Since the multicriteria mapping
method is already extensively published (Stirling, 1997; Stirling & Mayer, 2001;
Davies et al., 2003; Stirling et al., 2006), the subsequent discussion concentrates
on the diversity analysis method – explaining in some detail the procedures
employed in using a specialised Excel spreadsheet interface for a dedicatedMatlab program. Chapter 4 begins by outlining key features and outputs of eachinterview on a participant-by-participant basis. With appropriate caution given
the small sample sizes involved, it then analyses how individual perspectives are
distributed across affiliations and countries and discusses possible reasons forthis emerging distribution. Finally, the discussion explains how the many
uncertainties that are inherent in such appraisal yield sensitivities in the final
energy portfolios that emerge. Chapter 5 summarises the findings of this projectand concludes this working report.

8

2. Concepts
At root, diversity is a property of any system whose elements may be apportioned
into categories (Leonard & Jones, 1989). Energy diversity is no exception, wherethe ‘systems’ in question may variously be construed as electricity supply
infrastructures, or energy systems taken as a whole; where the ‘elements’ areindividual generating facilities or primary fuel sources and the ‘categories’ arevariously-aggregated types of technology or resource. Any method that is adapted
to examine diversity as a property of a system whose elements are apportioned
into categories is thus well-placed as a framework for examining a variety ofperspectives on energy diversity.
In the area of focus of the present study – the electricity supply sector –discussions of diversity span an array of disparate supply and demand sidetechnologies and primary resources. The scope of diversity analysis is further
extended by a variety of other relevant factors, including: the regional sourcing offuel and associated supply routes; concentration among trading, supplier orservice companies; reliance on generic equipment or component vendors;
dependencies on monopoly utilities, shareowners or labour unions; and theconfigurations and spatial distribution of infrastructures (PIU, 2001; Farrell et al.,
2004; CEC, 2007). These are all prominent features of debates over diversity inenergy security. Likewise, each is potentially relevant to diversity as a means tohedge ignorance, foster innovation, mitigate lock-in or accommodate plural values
and interests, in the broader senses discussed above in relation to transitions toenergy sustainability (Stirling, 2008b). It is therefore desirable that any
framework for the analysis of energy diversity be equally applicable in principle
across all these aspects.
A general examination of interdisciplinary approaches to the analysis of diversity
reveals that this system quality is repeatedly defined in terms of three basic
properties (Stirling, 1994).

9•

Variety: the number of diverse categories of ‘option’ into which systemelements may be apportioned.

•

Balance: the apportionment of the energy system across the identified
options.

•

Disparity: the manner and degree in which energy options may be
distinguished.

Each is a necessary but individually insufficient property of diversity (Stirling,1998). A series of methodological questions follow from this. How can these quitedistinct aspects of diversity be aggregated into a single coherent framework? How
might such an analytical framework be applied such as to accommodate the rangeof relevant perspectives typically engaged in real debates over energy strategy?
And how can the results of any diversity analysis on these lines be articulatedwith wider policy considerations – such as the performance of individual
generating options under criteria of economic efficiency, environmental quality,
social impact and security of supply raised earlier in relation to broadsustainability goals? General investigation of the property of diversity yields aseries of evaluative criteria, with which any candidate analytical framework mustcomply (Stirling, 2007).
Although there presently exists no uniquely specified axiomatic way to resolvethese contending analytical criteria, it is remarkable that there does exist a
relatively simple quantitative framework which satisfies the demands of all
criteria taken together. Although this is quantitative in nature, it takes the formof flexible general heuristic, rather than an ostensibly definitive index. Instead ofaiming to measure diversity in some unconditional objective fashion, a heuristicoffers an explicit, systematic basis for exploring sensitivities (Stirling, 2007). In
this way, a heuristic framework for the analysis of energy diversity aims to
combine the rigour, transparency and specificity of quantification with the scope,
applicability, flexibility and symmetry of qualitative approaches. The way inwhich this can work in practice, is the subject of this pilot demonstration exercise.
10

The details of this approach and its associated rationale are discussed elsewhere(Stirling, 2007; 2008b). For now, the basic underlying mathematical formulationis readily expressed. This specifies the aggregate policy value of an energy
portfolio (V{S}) as a function of the value due to the performance of the individualenergy options (V{E}) plus the incremental net value due to the degree of diversity
displayed by the system as a whole (V{P}) (see also Section 3-2).

V {S } = V {E} + V {P} = ∑i ∑c ( wc ⋅ S ic ) ⋅ pi + δ ⋅ ∑ij ( i ≠ j ) (d ij ) α ( pi ⋅ p j ) ⋅ ι ij
βHere, the performance of the individual energy options (V{E}) is characterised as
the simple linear product of technical scores (sic) assigned to each of i options
under

a

seriesofcappropriately-weighted(wc)

evaluative

criteria:∑ ∑ (w ⋅ S ) ). This is a model that is well-established in decision analysis –
i

c

c

ic

including that favoured for application in UK Government policy appraisal (DTLR,
2001). It is important to note here that there exist different formal ways in which
various specialist disciplines have become habituated in characterising option
performance (Vincke, 1992; Bana e Costa, 1990). Preferences or allegiances in thisregard may sometimes be raised as an issue (for instance, in the present study, in
discussion with JP4 addressed in the results chapter below). In short, the ‘linear
additive weighting’ method embodied in the above algorithm, is that which is
widely held to comply best with basic tenets in rational choice theory (DTLR,
2001; Salo, 1995). It is closest in character to the body of this theory underlying
conventional economic assessment (Bonner, 1986). Although arguably deficient inthese ways, other choice models (eg: ‘pairwise comparisons’ or ‘outranking
methods’) might equally be adopted for the present purpose, without altering thebasic framework of this present analysis of the trade-off between diversity andperformance.
The second part of the term expresses the value due to the degree of diversity
displayed by the system as a whole (V{P}) in terms of the ‘Stirling diversity
11

heuristic’ under which diversity is simply the sum of the degree of disparity
displayed between each pair of energy options (dij), weighted by the proportional
contribution of each option (pi, pj): δ ⋅ ∑ij(i≠ j )(dij )α ( pi ⋅ p j ) β ⋅ ι ij . The additionalparameters α and β are exponents which govern the relative priority assigned tothe diversity properties variety, balance and disparity. (iij) is an expression of
positive and negative portfolio interactions between options and δ is a coefficient
expressing a subjective judgement of the relative priority to place on system
diversity compared with option performance.
The practical logic underlying this mathematical framework will be explained in
discussing the elicitation process below (Section 3-2).

12

3. Methodology
3-1. Introduction
Multicriteria diversity analysis is conducted through an in-depth face-to-faceinterview with an individual participant. There are two basic steps to the process.
First, participants appraise the performance of individual energy options. Second,
they analyse a series of diverse portfolios. Each step is facilitated by use of aunique dedicated computer software package.

(1) Performance appraisal: Participants are asked to identify a set of relevantenergy options and appraise their relative performance under a range ofevaluative criteria. These criteria can address any issue felt to be relevant by
the participant (e.g. cost, environmental sustainability, security). Based onwhatever technical data is felt to be salient, options are then scored by theparticipant under each criterion, with uncertainties expressed by defining a
range of scores in each case. The criteria are then weighted to reflect theparticipant’s judgement of their relative importance, in order to yield a set ofoverall performance ranks for each option. This is a multicriteria mapping(MCM) process, using the ‘MC-Mapper’ software. Further details of thismethodology are widely published and explained in Appendix 1 to this report.

(2) Diversity analysis: By entering this MCM data into an Excel spreadsheet
interface for a dedicated diversity analysis program (written in Matlab), theinterviewer can derive in real time a picture of various diverse portfolios
reflecting different possible valuations of diversity in relation to performance.
This range of diverse portfolios is based on the degree to which options aredisparate from each other in terms of the structures of their performance as
obtained in the participant’s MCM. This is displayed as a continuum ranging
from the option mix that yields maximum portfolio performance, to that whichyields maximum portfolio diversity. The diversity analysis package also
produces a dendrogram to indicate the basic structure of the underlying option
disparities implicit in the MCM data.
13

On this basis, participants can then address a series of more detailed factors
relevant to the analysis of diverse portfolios. First (using the Excel spreadsheet
interface), they can identify any constraints held to act on the contributions ofindividual options, groups of options, or sub-tranches of particular options.
Second, if they are unsatisfied with the structure of the disparities embodied intheir performance appraisal, they can define further ‘disparity attributes’ suchas to modify this structure to arrive at a picture of disparity with which they
are happy. Third, they may also enter their estimations of the effects ofportfolio-level interactions between energy options.
The final stage of the interview (time permitting) is to conduct sensitivity
analysis on all the key input parameters. If at any stage a participant wishes todefine variant options or subdivide options into distinct tranches (for instanceto reflect constrained subordinate contributions available at different cost),
then this may also be conducted using the MC-Mapper software. If necessary
and possible, participants may iterate to return to the performance appraisaland define entirely new options, criteria or re-assess performance using MCM.
During this iterative two-stage process, participants typically comment on why
they evaluate performance scores and diversity characteristics in such a way andhow the final energy portfolio relates to their prior intuition. In a more elaborate
multicriteria diversity analysis exercise, this information – suitably prompted byquestioning from the interviewer – can be transcribed and provide a rich source ofcontextual qualitative information. As conducted for the present study, the wholeprocess takes 1 to 3 hours, depending on participants’ availability and enthusiasm– and the detail of their appraisals. As it transpired in the present project, allJapanese participants completed the exercise far more quickly than the British.
They instantly valued each performance data and disregarded many criteria forthe assessment. Although this was evidently partly because they could not afford
to spend much time for this, it is interesting to observe here a cross-culturalcontrast in subjects’ engagements with this analysis. It appears that Japanese
respondents were more uncomfortable than their British counterparts with the14unfamiliar and subjective character of this exercise. Our experience in interviewssuggests that the policy culture on these issues in Japan is somewhat more
sympathetic to a ‘technocratic’ style of analysis, under which – rather than being
elicited on the basis of expert judgement – aspects like option disparities arederived automatically from some technical parameter and treated as ‘objective’.
Either way, this issue is somewhat tangential to the focus of the present study.
The affiliations of participants are shown in Table 1. Participants are anonymisedin order to protect privacy as this is a pilot case study and there are sensitivities
concerning how the results might be represented or interpreted by other energy
specialists or policy actors.
Table 1: List of ParticipantsID

Affiliation

State

Interview date

UK1 Senior academic energy economist

UK

16 Nov. 2007

UK2 Research director, public energy research instituteUK

16 Nov. 2007

UK3 Former industry energy executive, government advisor UK

19 Nov. 2007

UK4 Senior executive, major national utility

UK

20 Dec. 2007

JP1

Senior academic energy policy researcher

Japan

30 Nov. 2007

JP2

Senior executive, public energy research instituteJapan

10 Dec. 2007

JP3

Executive director, private energy research instituteJapan

11 Dec. 2007

JP4

Academic energy system engineer

Japan

20 Dec. 2007

JP5

Councillor in government ministryJapan

26 Dec. 2007

The participants were selected in order to ensure an appropriate degree ofpluralism in each country, both of affiliation and expertise. The selection also
reflected the necessity for the purpose of the present pilot exercise that each could15

be assumed to hold the requisite knowledge meaningfully to appraise the various
energy technologies and policies.
For the purposes of facilitating the interviews, the research team compiled fromgovernment policy documentation and peer-reviewed sources an illustrative set of
performance data for a representative range of consistently-defined electricity
supply options and a series of energy policy appraisal criteria. The data compiled
included issues of economic cost, various atmospheric emissions, land use,
employment intensity and so forth. This database was made available as areference point to all participants when conducting their own individual MCM
appraisal, in order to enable them more readily to understand the kind of issue
that they might wish to consider. However, the participants (as energy experts intheir own right) were at liberty where they felt they could justify this, to modify
this data to accord better with their own specialist judgement.
Given that the purpose of the multicriteria mapping methodology is quite
specifically to ‘map’ divergent views, this is conducted not to enforce an artificial
coherence across participants, but to provide back-up resources only where theseare felt to be necessary. In practice, as expected, all participants were sufficiently
specialist in the fields in question, that these criteria and data were employedsimply as a prompt, with attention focusing overwhelmingly on each individual’s
own technical knowledge and expert judgement. That this is so, can be seen in the
manifest divergence between the pictures of energy option performance, as
derived by different individual interviewees (Chapter 4). However, the basic
template of ‘core options’ which formed the basis for this initial dataset, did serve
the successful function of somewhat standardising the definition and partitioningof energy options, such as to enable effective comparison across individuals andnational contexts. This will be discussed further in the chapter on comparative
analysis below.

16

3-2. Diversity Analysis
As summarised above, the present diversity analysis method is based initially on
data generated in performance appraisal. The basic idea is that the degree of
disparity displayed between any pair of energy options is, to a first approximation,
expressed by the differences observable in their performance profiles. Other
aspects – such as the intrinsic physical nature or origins of the resources ortechnologies involved – may be taken into account as a second order consideration.
But it is found in practice that these kinds of additional attributes beyond thoseroutinely considered in performance appraisals, actually typically exert relatively
little impact on the structure of disparities as embedded in the performance data.
In this way, for instance, an option displaying relatively high air quality scores,
low catastrophic risk scores, high security scores and low economic scores will behighly disparate from one displaying the opposite characteristics in each respect.
Other options with more mid-range values in each case will be correspondinglyintermediate in their mutual disparities. A dedicated Excel macro program has
been devised for this pilot study with the aim of quickly converting output datafrom the MC-Mapper software to input data for the Matlab diversity analysis
program. See Appendix 2 for more detail on the macro program and Excel
spreadsheets.

174. Analysis
4-1. Interview Results and Output Figures
The following pages illustrate, for each participant, (1) charts displaying theoverall performance rankings obtained in the MCM appraisal for each energy
option defined by the participant; and (2) a range of diversity-optimal portfolios
obtained in the diversity analysis process (Figure 4 from the Matlab program; see
Appendix 3). Each picture represents the outcome of detailed deliberation on thepart of the participant in question. Key features that emerge from these pictures
are discussed in each case. This includes some discussion of the underlying
performance criteria and associated scoring (which also forms the basis for the
disparity attributes), as well as further qualitative information arising from theinterview and more reflexive observations concerning the overall process ofinteraction and analysis.
Taken together, these empirical findings confirm other comparable ‘mapping’studies, in demonstrating the relatively high degree of variability between theperspectives of different specialists – even within a particular national context.
Suitably qualified by reference to the small numbers of respondents engaged within this pilot study, the following sections will undertake various forms ofcomparative analysis to identify cross-cutting issues, questions and potential
generalisations, that span the two national contexts.
The individual disparity structures that underlie these pictures will be discussed
separately in later analysis – and also be taken as a basis for more generalcomparison.

18

1. UK1

Figure 1: Overall Performance Rankings (UK1)
UK1 is a senior academic energy economist. The performance appraisal criteria inthis case comprise engineering cost, climate change impact, land use, employment,
regional economic development, public acceptability, community empowerment,
proliferation, catastrophe, and long-run security. UK1 noted that land use
includes infrastructure and amenity issues, catastrophe includes all catastrophic
and mortality risks and commented that public acceptability and community
empowerment are somewhat related each other.
The criterion weighting for engineering cost is the highest (1200), followed byclimate change impact (600), long-run security (600) and public acceptability(300).
Wind and hydro energy options are the best performing options overall, and quitedistinct in this respect from other options. Driven by (highly weighted) poor
economic scores, solar PV displays the worst performance. The uncertainty
intervals displayed by these rankings are relatively wide, which means that UK1
19

takes into account a considerable degree of uncertainty.

Figure 2: Diversity-Optimal Portfolios (UK1)
UK1’s range of diversity-optimal portfolios illustrates that gas CCGT (combined
cycle gas turbine) and coal with CC (carbon capture) are dominant where theperformance is prioritised over diversity (left-hand side of the above figure). In
other words, the parameter represented on the horizontal axis (delta) represents
the relative weighting placed on aggregate option performance as compared withaggregate portfolio diversity (see equation in Section 2). This is due to theconstraints held to limit contributions from better-performing options. As thevalue of diversity increases against that of performance, biomass, coal, nuclear
and offshore wave options each enter the portfolio in succession and acquiresignificant shares. As the performance-diversity trade-off continues to rise, wind
and tidal stream options diminish their shares and – at high weightings on
diversity – actually leave the portfolio.
This latter feature is due to the fact that these better-performing renewable
options are positioned in an intermediate region of the multidimensional disparity
20

space defined for all options taken together. As the weighting on performancediminishes, so the diversity-optimal portfolios select for those options occupying
the envelope of this disparity space. In UK1’s case for example, wind and tidal
stream show better performance than other renewables such as waste, wave,
geothermal and solar power and lie in the ‘interior’ of UK1’s disparity space – withthese other renewables defining the envelope of the disparity space. In otherwords, waste, wave, geothermal and solar power display similar kinds of disparity
to wind and tidal stream (with respect to other prominent options in the portfolio– like gas, oil, coal, biomass and nuclear), but to a more pronounced extent. With
relatively low weightings placed on diversity, then, wind and tidal stream enter
the portfolio before these other renewables. As the diversity weighting increases,
however, and wave, geothermal and solar power all begin to enter the portfolio,
then the diversity benefits accrued through the wind and tidal stream
contributions are increasingly dominated by the diversity benefits accrued
through these lower-performing options. This is because these latter options
display similar disparity characteristics with respect to other prominent options,but – being at the ‘edge of the disparity envelope’ – these are more pronounced. As
the diversity weighting continues to rise and the performance advantage
displayed with respect to these options by wind and tidal stream continues todiminish, so wind and tidal stream are eventually displaced from the portfolio by
the options that dominate them in terms of disparity.
This phenomenon reflects a general feature of diversity analysis that is also
relevant in other cases in this pilot study. Where the exit of options with rising
diversity weightings is held to be counterintuitive, this is simply because theunderlying disparity attributes have been incompletely specified, such that they
are readily dominated. In other words, it is an indication that the exiting options
need to be characterised more distinctively. This can be addressed by defining ahigher resolution disparity space in which more fine-grain differences areresolved between the options on the envelope of the space and those in the interior.
This can readily be achieved by defining further disparity attributes during thediversity analysis, but was not conducted in the present pilot exercise.

21

2. UK2

Figure 3; Overall Performance Rankings (UK2)UK2 is the Research Director of a public energy research institute. The evaluation
criteria comprise engineering cost, climate change impact, SO2 emissions, landuse, noise, indirect energy input, regional economic development, public
acceptability, community empowerment, proliferation, catastrophe, and long-run
security. In this view, land use also means land sterilised for other uses. Public
acceptability is regarded as an outcome of political process. Options are assessed
under a time frame of 2025-30, which in allows for full consideration of new
options. The Severn Barrage, a potential site for tidal power generation in the UK,
is added to the initial set of energy options and distinguished from the tidal
stream option.
Engineering cost is the highest-weighted performance criterion (230), followed byclimate change impact (60) and long-run security (40).

22

Figure 4: Diversity-Optimal Portfolios (UK2)
In the diversity analysis, UK2 subdivided the gas CCGT option into six, according
to the origin of the resource. Among others, gas CCGT-UK continental shelf isdominant at the performance-oriented side of the range of portfolios, since UK2
did not constrain the maximum proportional contribution from this energy option.
As with UK1, it is evident that several options move in and out of thediversity-optimal portfolios as the weighting on diversity increases, giving theportfolio range a particularly dynamic and complex appearance. Again thisreflects the configuration of the disparity space noted for UK1, amplified by thegreater variability in the performance ranks.

23

3. UK3

Figure 5: Overall Performance Rankings (UK3)UK3 is a former senior energy industry executive and member of a UK
government energy policy advisory committee. The evaluation criteria compriseengineering cost, climate change impact, speed of implementation, infrastructurecompatibility, intermittency, investment risk, and geopolitical exposure. Speed of
implementation includes wider issues like research, the state of technology,
barriers and their removal, such as regulatory and other non-market obstacles.
The meaning of ‘infrastructure’ in the infrastructure compatibility criterion isbroader than just electricity infrastructure including, for instance, carbon captureand storage – which are regarded as highly uncertain technologies. UK3 also
notes that investment risk is a particular issue for nuclear power.
Unlike others, UK3 put a relatively low weighting on engineering cost (20) as
compared to the two main criteria – climate change impact (100) and speed of
implementation (100). Other criteria are weighted at the same level as24

engineering cost.
The performance ranking chart clearly distinguishes hydro option as holding
highest performance. Gas and coal (each with carbon capture) and oil havesignificantly lower performance scores than other options.

Figure 6: Diversity-Optimal Portfolios (UK3)
UK3 obtains a high contribution for nuclear at the performance-weighted side of
the range of portfolios. This reflects the tight constraints that are held to bear on
the similarly-performing renewables. Gas CCGT with carbon capture is a growing
option as diversity increases. In this case, as that of other participants, thespecific rationales for these kinds of framing assumption were not fully exploredin the present pilot exercise – the focus being on testing the elicitation of diverse
portfolios. However qualitative elicitation of framings is a strong feature of otherfull multi-criteria mapping exercises aimed at exploring performance alone (asdistinct from performance and diversity). More comprehensive and in-depth
follow-on research may thus extend the analysis to include full consideration forthe divergent rationales underlying the contrasting evaluative positions ofdifferent participants.
25

4. UK4

Figure 7: Overall Performance Rankings (UK4)UK4 is a senior executive with a major national utility. The evaluation criteria
comprise engineering cost, development cost, climate change impact, SO2
emissions, land use, noise, indirect energy input, regional economic development,
publicacceptability,community

empowerment,

terrorism/proliferation,catastrophe, long-run security, and load factor. UK4 expressed a need for further
informationinperformanceappraisal,

requiring

government

data

on

disaggregating capital and operational costs within the category of engineeringcost. Load factor refers to the proportion of time and rated capacity that a given
type of plant is actually generating power. UK4 added fuel cells as an additionalenergy option. There are in this view two types: micro-CHP (combined heat andpower) and larger cells for electricity storage/generation.
The highest-weighted criteria are engineering cost (19), climate change impact(19), load factor (17) and public acceptability (13). The rest are weighted at values
26that are less than half of this. With regard to overall performance, nuclear clearlyoutranks all others, with little to distinguish the rest, aside from the relatively
high ranks for large tidal and fuel cells and low-ranking positions for oil and solar.

Figure 8: Diversity-Optimal Portfolios (UK4)
The range of diversity-optimal portfolios shows a clear pattern of increasing
diversity. Nuclear dominates at the performance-oriented side of the portfolio,
constrained only by system-operational factors to admit a 5% contribution fromfuel cells. This reflects UK4’s overall performance rankings as illustrated in
Figure 7, in which nuclear shows the best performance followed by fuel cells. Of
course, it is not realistic that the UK could accept nuclear at 95% of total
electricity supply. Again, a more detailed exercise than the present pilot study
would prompt participants to return to introduce further second order constraints,
according to their own judgement of the technical position. As the weighting on
diversity increases, so many energy options enter into the portfolio and thecontribution of the nuclear option decreases significantly. The large tidal option
enters in a gradual fashion, because a large step size was not specified for the
purpose of this pilot study.
27

5. JP1

Figure 9: Overall Performance Rankings (JP1)JP1 is a senior academic energy policy researcher. The evaluation criteria
comprise engineering cost, climate change impact, regional economic development,
public acceptance, proliferation, catastrophe, intermittency, air pollution, waste,
and ecosystem risk. JP1 notes that regional economic development includes
employment.
The highest-weighted criterion is engineering cost (300), followed by climatechange impact (60), public acceptance (50), catastrophe (50) and intermittency(50). The rest have the same weightings (30).
Regarding overall performance, the waste option ranks highest, slightly above
wind, hydro and biomass. LNG (liquefied natural gas) and nuclear are also highperforming, but have wider uncertainties. Oil, wave, tidal and ocean thermal are
relatively low performing, with other options intermediate.

28

Figure 10: Diversity-Optimal Portfolios (JP1)
As with UK1, JP1’s range of diversity-optimal portfolios displays the feature ofcertain options leaving the portfolio as the weighting on diversity increases. JP1
queried this, for instance in relation to LNG. Again (as discussed above for UK1),
the reason lies in the positioning of the excluded options in dominated regions of
the disparity space. In this way, it is contrary to JP1’s intuition – but not to JP1’s
definition of disparities as embodied in the performance appraisal – that LNG is
not so distinctive from oil. As discussed in relation to UK1, further refinement of
the disparity space by addition of further disparity attributes such as to more
finely resolve the options in question would reverse this effect. In particular, thiswould allow JP1 to define further disparity attributes under which LNG is
credited more fully with its own distinguishing attributes.
JP1 also asked what about adding energy conservation to the initial set of energy
options for appraisal in MCM. This raises a question about the focus in this study
on electric power supply technologies. Energy conservation can indeed be taken
into account as an energy option by adopting an ‘integrated resource planning’framework. It was not possible for the purpose of this pilot exercise to adapt the29framework accordingly, or acquire the necessary data, so this remains a
potentially significant issue for a future larger-scale study.
6. JP2

Figure 11: Overall Performance Rankings (JP2)JP2 is a senior executive for a public energy research institute. The evaluation
criteria comprise cost, political risk, technological risk, public acceptance,
environmental impact, and available supplies. JP2 notes that environmental
impact is mainly based on global warming.
The available supplies criterion has the highest weighting (23) and political risk
has the lowest (7). Other criteria are equally weighted in between (19).
In performance terms, LNG emerges as the best option overall, with coal, oil andsolar PV following in succession. Nuclear performs highly at one end of the range,
but displays high uncertainty. Coal with carbon capture and storage is the lowest
performing option, although also displaying high uncertainty.

30

Figure 12: Diversity-Optimal Portfolios (JP2)As with JP1, JP2 queried why LNG is eliminated from the diversity-optimal
portfolios at higher weightings on diversity. The reason is due to the same cause
as discussed for UK1 and JP1, reflecting a logical but intuitively difficult featurethat is resolvable through more refined definitions of disparity of a kind that were
not carried out for this pilot study. Both JP1 and JP2, as energy security experts,
appraised LNG to perform better than oil under almost all criteria, recognised a
difference of kind between oil and LNG in geopolitical and economic terms. The
counter-intuitive aspects of the portfolios they have each obtained, thus offer aconcrete basis for applying more fine-grain disparity attributes which reflect thefull depth and extent of their understanding of the issues bearing on distinctions
between these energy options.

31

7. JP3

Figure 13: Overall Performance Rankings (JP3)JP3 is the Executive Director of a private energy research institute. The
evaluation criteria comprise cost, public engagement, environmental impact,
resources constraint, regional economic development, industrial economic impact,
geopolitical risk, local/national politics, technological uncertainty, and innovation
effect (facilitating spill-over and industrial infrastructure development). The costcriterion includes social cost and the biomass option excludes traditional grain
and ethanol as first generation biofuels.
JP3 assigned three levels of weightings. The heaviest weighting (50) is assigned tocost, environmental impact, and resource constraint. The lightest (10) is assigned
to technological uncertainty and innovation effect. Middle-level weightings (30)
are assigned to the other criteria.
The performance ranking chart clearly illustrates two groupings of energy options.
One consists of renewable energies including solar thermal, solar PV, wind,geothermal and biomass, which display higher performance. The other group
includes the conventional energy options. This group displays greater variability
in rankings than does the higher-ranking group, but the uncertainty ranges for32the individual options is similar across all groups.
JP3 notes this distinction between two groups of energy options reflects a generalperspective that is characteristic of the particular energy-environmental interestsof the associated independent research institute, which broadly favours
renewable over conventional energy options. In particular, the waste option isregarded as broadly similar to the conventional energy options, with aperformance that also reflects this.

Figure 14: Diversity-Optimal Portfolios (JP3)
The range of diversity-optimal portfolios further reflects this picture of
performance, with renewable options dominant at the performance-oriented side.
Solar PV remains dominant throughout the range of portfolios. As with UK1, JP1
and JP2, it is notable that one of the highest-performing options (in this case,wind) is removed from the portfolio at high weightings on diversity. As discussed
in these other cases, this reflects the relatively course-grained characterisation of
disparity in the present pilot study.

33

8. JP4

Figure 15: Overall Performance Rankings (JP4)JP4 is an academic energy system engineer. The evaluation criteria comprise cost,
political risk, public acceptability, environmental impact and supply capability.
Political risk, in these terms, includes resistance of electric power companies.
Ocean energy as an option that includes wave, tidal and ocean thermal.
Cost is the highest-weighted criterion (50) taking up one third of the total
allocated weighting. Environmental impact (30) and supply capability (30) weight
second, with political risk (20) is third.
Coal and nuclear perform significantly better than other options in the overall
rankings. Ocean energy displays a particularly low ranking.

34

Figure 16: Diversity-Optimal Portfolios (JP4)JP4 expressed some surprise that natural gas does not become included in theportfolio, even as the weighting on diversity increases to high values. Intuitively,
JP4 considers natural gas to be quite distinctive in its characteristics, and should
thus be expected to appear where higher values are placed on diversity. Again,
this is likely to arise from this high degree of distinctiveness being under-reflectedin the actual disparity attributes that underlie the analysis, as based on the MCM
performance criteria. By taking further time to define additional disparity
attributes, this would be resolved.
JP4 pointed out that it is difficult to address ‘rapid dispatch’ as a collective
constraint because it is dependent on temporal considerations. The present static
analysis does not allow full account to be taken of these kinds of dynamic
characteristics.
JP4 also mentioned that he had trouble in weighting some performance criteria in
the MCM process. This was based on the view that two criteria can be compared,
but more than two are difficult to compare in a direct way. JP4 explained this in35

terms of the ‘intransitivity’ of criteria. This refers to a detailed technical debate inmulticriteria analysis, mentioned in the ‘Concepts’ chapter earlier of a kind that
does not affect the nature of the outcome of this exercise.

36

9. JP5

Figure 17: Overall Performance Rankings (JP5)JP5 is a councillor in a government ministry. The evaluation criteria compriseenvironmental impact (90), cost (70), technological risk (40), public acceptability(50) and political risk (60). Note that environmental impact mainly reflects
impacts on global climate change. The waste option includes agricultural, forestry
and municipal waste.
Solar thermal, solar PV and wind are the best-performing options, followed by the
nuclear option, although this displays high uncertainty.

37

Figure 18: Diversity-Optimal Portfolios (JP5)The highly static nature of the range of diversity-optimal portfolios in this case,reflects the highly restrictive set of constraints that JP5 set for the contributions
from individual energy options. Indeed, JP5 carefully calculated the sum of
individual constraints to be nearly 1. Taking pride in being an environmentally
cautious decision-maker (which is also suggested as the heavy weighting for the‘Environmental Impact’ criterion in MCM), JP5 claims that the policy challenge is
not so much a matter of pursuing energy diversity as an end in itself, but ofharnessing the intrinsic diversity in the energy options (including as many
renewable

optionsaspossible)requiredtocomplywithinternational

environmental treaties and agreements. Accordingly, JP5 expressed little surpriseat the form taken by the performance picture yielded in the MCM appraisal or the
structure of the diversity-optimal portfolios.

38

4-2. Two-Dimensional Scaling
The present exercise is a pilot study primarily for the purpose of demonstratingand exploring the diversity analysis methodology. Accordingly, the numbers ofparticipants involved in this study are relatively low – much too low to aggregate
analysis seeking high levels of statistical significance across groupings like that
contrasting the UK with Japan. On the other hand, difficulties of this kind arelikely to be experienced even in a more detailed analysis. This is due to the highly
specialised nature of the information required in diversity analysis coupled withthe relatively small pool of appropriately-engaged people and the high degree ofvariability typically displayed between individual specialists in this kind ofcomplex technical area. Subject to this general qualification, then, there is somevalue in demonstrating the kind of comparative analysis that might be conducted(with correspondingly greater confidence) in a more elaborate application of themethod piloted in this study.
The most obvious comparative question that might be asked, concerns thepossible existence of patterns in the highly varied ranges of diversity-optimal
portfolios obtained by these nine expert participants from disparate institutional
backgrounds in two different countries. This is a question that can be approached– even for relatively low numbers of participants – using two-dimensional scaling.
The first step required in undertaking such a comparative analysis, is to renderconsistent the different sets of energy options. This is necessary, because many
participants defined additional options to those suggested in the core set
developed by the analysts. The present comparative analysis is thus based on a‘core set’ of options, that were defined consistently and appraised by all nine
participants from both countries involved. The distributions of portfolio
contributions for these core options can then be analysed and two principal
components extracted for a series of different points in the spectrum of portfolios(Table 2). This operation is expected reveal the internal structure of the data in a
way which best explains the variance in the data. It transforms the data to a new
orthogonal coordinate system such that the greatest variance by any projection of
the data comes to lie on the first coordinate (called the first principal component),
39

the second greatest variance on the second coordinate. In simpler terms, the firstprincipal component identifies the main contrast in viewpoints across all
participants taken together over the performance of different energy options,whilst the second component explains the second most significance contrast in
views.
Table 2: Factor Loadings for Two Principal Components
1-0.637-0.436
0.322
0.247
0.911
0.262
7.20E-02
0.787
0.627
-0.837
1.11E-02
0.289
-0.34
-0.11
-0.276-0.492-0.4430.7

Gas (CCGT)
Gas (CCGT) + CC(S)LNG
LNG + CCS
Coal
Coal + CC(S)
Oil
Nuclear
HydroWind (On. large/micro, Off.)
Solar PV
Solar Thermal
Geothermal
Biomass
Waste (incl. Landfill Gas)
Tidal (incl. Severn Barrage)
Wave
Others2-0.156-0.2110.341
0.441-0.3350.796
1.74E-02-0.2040.297-0.2770.757
0.161-0.1260.726
0.427-0.718-0.757-0.449The horizontal axis and the vertical axis in these figures represent the normalized
factor scores for the first component and the second component in Table 2,
respectively. The factor scores are computed by the regression method, whichworks by multiplying the factor loadings by the inverse of the original correlation
matrix. Conceptually, the factor score represents the degree to which eachindividual performance score rates high on energy options that have high loadings
on a factor. Factor extraction, rotation and factor score computation is processedby SPSS computer software and two-dimensional scaling is displayed with thehelp of Excel software.
In the present study scaling analysis was performed for four different points in
the ranges of diversity-optimal portfolios. These were bounded at one end by the‘maximum performance’ portfolio (with no weighting on diversity) as shown in
Figure 19 and at the other end by ‘mid point’ of the range (where the total
portfolio value is constituted equally by the performance of the individual options
40

and the value placed on portfolio diversity) as shown in Figure 23. Intermediate
between these two cases, scaling analysis was also conducted for portfolios
displaying 75%, 60% and 55% of total value being due to option performance(Figure 20, Figure 21 and Figure 22).
Despite the variability displayed within the two national groups of experts, theoverall picture arising from this two-dimensional scaling procedure across all four
sampling points is a pretty clear separation between UK and Japaneseparticipants. This implies that UK participants tend to favour gas CCGT andwind as high-performing energy supply sources whereas Japanese participants
tend rather to favour coal, nuclear and hydro. This may to a large extent reflect
the different contexts and resource endowments in each country, but may also tosome extent be a reflection of distinct cultural and institutional aspects of a kindthat might bear further exploration.
Reviewing the picture represented in Table 2 in more detail, the first component(shown on the horizontal axis in the charts that follow) is defined at the extremes
by coal, nuclear and hydro in positive terms and gas CCGT and wind in negative
terms. This is the axis that discriminates best between the UK and Japanesecases. Japanese portfolios favour coal, nuclear and hydro, whilst wind and gas
CCGT are relatively favoured in the UK. The second component consists of coalwith carbon capture (and storage), solar PV and biomass at one extreme and wave
and tidal at the other. This component is displayed on the vertical axis. For its
part, the second component seems to reflect the common distinction made acrossparticipants as a whole between renewable energy options (as a group) andconventional options (as a group – including carbon capture and storage).
Here, significant differences emerge in the two-dimensional representations
obtained in the case of 100% weighting on performance and 50% weight on
performance. Among other things, this reflects the fact that, the share of
renewable options becomes more similar between participants as diversity
increases.

41

2.5

Second principal component(favouring coal+CC(S), solar PV and biomass)

2

1.5

1

JP3

UK3

0.5

First principal component

UK1 UK2

JP1(favouring gas CCGT and wind)

0
-2

-1.5

-1

-0.5

0

0.5

1

JP5

1.5

2

2.5

2

2.5

First principal component(favouring coal, nuclear and hydro)

-0.5
JP4

UK4

-1

JP2

Second principal component(favouring wave and tidal)-1.5

Figure 19: Distribution of Participants at Performance 100%

2.5

JP3
2

1.5

1

JP2

0.5
UK3

JP4
-2

-1.5

0
-1

-0.5
JP1
JP5

0

-0.5

0.5

UK4

1

1.5

UK1

-1
UK2
-1.5

Figure 20: Distribution of Participants at Performance 75%
42

2.5

JP3

2

1.5

1

0.5
JP2
UK3
JP4
-2

0

-1.5

-1

-0.5

0

0.5
UK1
UK4

1

1.5

2

2.5

-0.5
JP5
UK2

JP1
-1

-1.5

Figure 21: Distribution of Participants at Performance 60%

2.5

2
JP3

1.5

1

UK1

0.5
UK3
JP4

JP2
0

-2

-1.5

-1

-0.5

UK4
0.5

0

1

1.5

-0.5
JP5
UK2
JP1

-1

-1.5

Figure 22: Distribution of Participants at Performance 55%
43

2

2.5

2.5

2
JP3

1.5

1
UK1

0.5

UK3

JP4
JP2
0
-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

UK4
-0.5
JP5

UK2
-1

JP1

-1.5

Figure 23: Distribution of Participants at Mid Point
As noted above, it is difficult to be confident about results obtained in a pilot study
involving such small samples sizes. Despite this there are some conspicuous
features, which warrant further attention.
First, there is the strikingly consistent distinction between the perspectives of UK
and Japanese participants (as displayed on the primary scaling axis for all points
in the range of diversity-optimal portfolios).
Second, these is a marked convergence in the distribution of participants on thesecondary scaling axis, as the weighting on diversity increases. This illustrates
the tendency for increasing valuations of diversity in the energy mix, not only toaddress matters of uncertainty and energy security (as is much discussed), butalso to enable a higher degree of reconciliation between the highly contrastingevaluative perspectives of different stakeholder groups. In a hotly contested arena
such as energy policy, this presents a degree of empirical support (both in the UK
and Japanese cases) for the hypothesised importance of diversity as a means to
44

2.5

accommodate greater pluralism in the politics of energy technology choice.

4-3. Analysis of Disparity Structures
This section provides a comparative analysis of the disparity structures obtainedin the MCM performance appraisals conducted by the nine participants. For thispurpose, it makes use of qualitative deliberation over the form of the dendrograms(obtained using the Ward method) obtained in each individual participant’s
diversity analysis (as described above). The purpose is to examine the degree andtype of difference that may be observed between the basic structures in the
disparities displayed under these specialist perspectives. In particular, it will beinteresting to compare the nature of these differences, with the quite radicalvariabilities that have been documented in Chapter 4 concerning the relative
performance (and consequently portfolio compositions) of the different options.

45

UK1

UK2

46

UK3

UK4

47

JP1

JP2

48

JP3

JP4

49

JP5

Although there are many differences between these alternative independent
assessments of disparity, there are some notable – and potentially significant –common features.
(1) Conventional energy vs. renewable energy
First, the great majority of participants discriminate at a fundamental level
between conventional energy options (as a group including coal, oil and nuclear)and the renewables as group. This is shown by the repeated appearance of a
two-cluster solution at the right-hand (high disparity) side of the figure. This is
not remarkable in itself, but it does present an unusually detailed empiricalgrounding for what is often little more than a casual assertion. In particular, thismakes a novel direct link to performance appraisal results, of a kind that are
widely available – thus suggesting a ready means to test the reproducibility ofthis finding. The distinction is also interesting in that it highlights a number ofpotentially significant exceptions.
50•Partitioning of ‘conventional’ options

A relatively minor variant on the general conventional/renewable distinction
described above is UK3, who groups coal, oil and gas CCGT as one cluster, with
nuclear, gas+CC and coal+CC. grouped separately at a high level of disparity.
Likewise, JP1 clearly differentiates coal and nuclear from LNG and oil.

•Hydro as conventional or not

There is a tendency for the UK participants to regard hydro energy as arenewable option whereas several of the Japanese participants (JP1, JP3 andJP5) regard it as conventional.

•

Biomass and waste as conventional or not

There is particular ambiguity in the UK over the ‘renewable’ status of biomass,
with some UK participants grouping it alongside conventional options and others
including it with the renewables. All the Japanese participants group biomass
with the renewables. This distinction of patterns is even more pronounced in thecase of waste. This probably reflects the particular (and sometimes ambiguous)environmental status of combustion technology, but provides an interesting
indication of divergent cultural views on this.
Taken together, the detailed features of this conventional/renewable divide
provide interestingly explicit confirmation of some often-implicit features ofenergy policy debates. In this respect, the analysis of disparity structures that are
inherent in performance appraisal, may offer a useful basis for more rigorous andtransparent attention to this important aspect. Of course, this overall picture of aquite fundamental difference between broad categories of ‘conventional’ and‘renewables’ may be significant in its independent substantiation, but it is hardly
surprising. There is however a more important and less intuitive consequence of
this, discussed below.

51

(2) The internal diversity of the renewable energy options
It is a feature of every disparity structure, without exception, that the broad
category of ‘renewable energy’ is disaggregated by the clustering algorithms on
the basis of its internal disparity at a level that is at least equal to theconventional partitioning of coal, oil and gas. This constitutes a very important
finding, to the effect that conventional analysis of diversity, based on accepted
linguistic and official statistical categories like “coal, oil, gas, nuclear andrenewables”, seriously understates the diversity benefits of the renewables. In
other words, for no better reason than casual use of historically-contingent
terminology, conventional analysis tends to group renewables together as if they
were less disparate than renewables are as a group from coal, oil, gas and nuclear.
What this analysis shows is that, under every one of the diverse expert
perspectives involved, renewables are actually subdivided on the basis ofdisparity at levels at least equivalent to the distinctions between fossil fuels.
Although the details vary, what this means is, that rather than being analysed
according to categories ‘coal’, ‘oil’, ‘gas’, ‘nuclear’, ‘renewables’, energy systems
should rather be analysed in terms of ‘coal’, ‘oil’, ‘gas’, ‘nuclear’, ‘renewables 1’,‘renewables 2’, ‘renewables 3’, ‘renewables 4’ and so on.
More specifically, although the details vary between different specialist
perspectives, it is a common feature that this large group of energy options(including, for instance, “wind, wave, tidal, hydro, biomass, waste, geothermaland solar”) may be seen generally to be equally disparate from each other, as are
the fossil fuels, and indeed even as the fossil fuels are from nuclear. Whatever thedifferences in other respects, this robust general result has potentially enormous
implications for the considerations over how to realise diversity benefits inelectricity supply portfolios. In short, the common practice of aggregating a
number (even all) of the ‘renewables’ together as a category for the purposes ofdiversity analysis will have the effect of seriously understating the diversity
benefits of renewable energy options.

52

(3) Scale of Disparities
The horizontal axis may be thought of as providing an indication of the levels of
disparity at which different clusters of energy options differ from one another.
Although the scale is arbitrary, the normalisation of the input data means that it
is comparable across participants. As shown by the contrasting horizontal
distributions of the dendrograms against this axis, then, the overall degree of
disparity obtained by each participant as a consequence of the multicriteria
mapping exercise, varies quite significantly.
Let us compare UK3 and UK4. UK3 displays a disparity distance of 0.5 to obtain a
one-cluster solution, whereas UK4 needs a corresponding disparity distance of
only about 0.25. UK3's dendrogram has a more complex and deeply-nested
structure as compared to UK4's. Comparison of dendrogram structures also tells
us that, for instance, UK2 and JP4 have a relatively clear mind-set over theextent to which conventional and renewable energy options are distinct.
It is thus clear from these dendrograms, that – despite the significant underlying
patterns noted above – different specialists do hold quite distinct views not only
over the performance of individual energy options, but also over the scale andnature of their mutual disparities. This is also an important finding for analysis
that tends to impose finely-specified concepts of energy diversity.

4-4. Sensitivity Analysis
In any complex policy appraisal exercise of this kind, it is always important to betransparent about the scale and nature of the key uncertainties. This is especiallytrue where the object of assessment is as ambiguously-defined and the analyticalframework as novel as is the case with conventional energy debates over diversity.
The present section will therefore outline the form of the sensitivity analysis thathas been conducted and convey the main findings. For the purposes of the present
53pilot study, this should also assist in conveying some of the features, limits andqualifications associated with this methodology. In order to illustrate a much
more general set of issues, we will here take as an illustrative example, the rangeof diversity-optimal portfolios generated by UK1. As such, the picture in the
diagrams below should be compared with the default picture shown in Figure 2.
Since both the disparity structures and the performance profiles are represented
as ranges, the sensitivity analysis focuses on the effect of taking different specificvalues with these ranges. It begins by displaying the median values for theseranges, as used as defaults in the diversity analysis. As discussed in the following,
there are three kinds of sensitivity analysis. The first analysis observes thechange from the lowest value (pessimistic) to the highest value (optimistic)
accommodated in the performance appraisal for each energy option. The second
analysis performs the same sensitivity analysis with respect to performance
criteria (rather than options). The third analysis explores a difference between
lowest and highest weighting amounting to a total factor of one order ofmagnitude. Note that the choice of a total sensitivity interval of one order ofmagnitude is purely illustrative (although it should accommodate most scope forreasonable differences of expert judgement).
(1) Scoring sensitivity by performance option
This can be illustrated by considering the case of coal with carbon capture. If thedefault performance values are substituted with the lowest ranking in theinterval obtained for the coal with carbon capture option (and everything else isleft ceteris paribus), then it has the significant effect of entirely excluding thisoption from the entire range of diversity-optimal portfolios (Figure 24). The
contribution is substituted instead by the biomass option, which also further
substitutes gas CCGT at the ‘high performance’ end of the spectrum.

54

Figure 24: Lowest Ranking for Coal with CC (UK1)
On the other hand, if we take the highest ranking value in the performanceinterval for coal with carbon capture (again ceteris paribus), then this option(unsurprisingly) significantly increases its contribution at the high-performanceside of the spectrum (largely at the expense of gas CCGT), and continues to makecontributions to high-diversity portfolios at a stage when this option is eliminatedunder default assumptions (Figure 25).

55

Figure 25: Highest Ranking for Coal with CC (UK1)(2) Scoring sensitivity by performance criteria
Turing to indicative sensitivity analysis on criteria (again using UK1 as an
example), if we take the lowest values for performance scores (for all options)
under the engineering cost criterion, then we see only some relatively minorchanges of contribution in the high-performance end of the range ofdiversity-optimal portfolios (Figure 26). The contribution of gas CCGT issomewhat diminished and that of biomass increased.

56

Figure 26: Lowest Scores under the Engineering Cost Option (UK1)(3) Weighting sensitivity by performance criteria
If we take the corresponding highest scores in the range obtained in appraisal
under this cost criterion, the effect on the form of the range of portfolios is again
relatively gentle (Figure 27). Indeed the relatively low sensitivity of the portfolio
structures to the taking of optimistic and pessimistic scores under this criterion isa fairly general finding where (as is often the case) the uncertainties in scores arerather similar across all options, thus leaving the relative orderings also similar
at either end of the range.

57

Figure 27: Highest Scores under the Engineering Cost Criterion (UK1)
By contrast with this, there is typically much higher sensitivity to shifts inweighting across criteria. For instance, if this same engineering cost criterion isdivided by root ten (leaving other criteria weightings in the same proportion to
one another but correspondingly higher than cost), then coal with carbon capture
comes to dominate the high-performance end of the range at the expense of other‘conventional’ options, leaving the renewable contributions relatively unaffected(Figure 28). Again, the choice of root ten as a baseline is determined by thejudgement that an overall sensitivity interval of factor ten accommodates areasonable range of expert debate.

58

Figure 28: Lowest Weighting on the Engineering Cost Criterion (UK1)
If this same cost criterion weighting is instead multiplied by root ten (again,

ceteris paribus), there is an even more significant change. With its economic
benefits (under this view) thus amplified, gas CCGT becomes the highest
performing option and so dominates the performance-oriented end of the range ofdiversity-optimal portfolios (Figure 29). At higher weightings on diversity
however, the effect of substituting coal with carbon capture with the lower-cost
option of conventional coal (which displays similar disparity characteristics andtherefore dominates coal with carbon capture as increasing weight is placed ondiversity).

59

Figure 29: Highest Weighting on the Engineering Cost Criterion (UK1)
These indicative findings are simply illustrative of the kinds of sensitivities that
may be expected in this kind of analysis – or indeed any attempt at assessing sucha complex and uncertain area of technology policy. The details will of course bedifferent in the case of the multiple options, criteria and parameters included inthis analysis. For present purposes, this discussion at least serves to underscore
the need for caution over specific details of any given appraisal, with attention
best concentrated instead on the overall structural form of the picture. However,when repeated over a number of different sets of portfolios, it does appear that thegeneral structures are quite robust to fairly radical shifts in the underlying
performance data.

60

5. Conclusion
With particular emphasis on the methodological implications, the present
working paper has discussed a pilot case study of a novel general framework forthe analysis of energy diversity. It has shown that the multicriteria energy
diversity analysis technique is applicable to an unconstrained array of different
specialist, institutional or stakeholder perspectives. Participants in the tworather contrasting policy contexts of the UK and Japan have shown themselves
able to undertake the procedure and derive intuitively interpretable results.
Taken together, they show that it is possible to obtain meaningful representationsof energy option performance and diversity of a kind that is substantively novel inthis field. What is particularly of interest, is that these detailed and intuitively
interpretable representations of energy diversity are directly constructed on thebasis of performance appraisals, of a kind that are widely available in the energy
policy debate. This suggests that the method is potentially quite widely
applicable.
With respect to the particular empirical findings obtained in this pilot study,
caution has been repeatedly emphasised over the small sample size. However, thetwo-dimensional scaling illustrates a strikingly consistent distinction between theperspectives of UK and Japanese participants. This is despite an equally striking
general convergence in the position taken by participants over the distinctions
between renewable options and conventional options. The comparative analysis of
the disparity structures revealed in the dendrograms shows that the great
majorityofparticipants

discriminateatafundamental

level

betweenconventional energy options and the renewables as group. It further argues thatthe broad category of ‘renewable energy’ is disaggregated at a level that is at least
equal to the conventional partitioning of coal, oil and gas. This is also a finding ofsome potential general importance for the wider analysis of energy diversity.
In examining the range of different portfolios, it is clear that – despite theconsistencies and convergences noted above – different expert perspectives in bothnational contexts yield radically different pictures of the best means to procure
61

diversity in electricity supply systems. Even when taking an overall interval as
high as an order of magnitude, sensitivity analysis shows that the differences
displayed between individuals are typically greater than the volatility of
individual portfolio ranges under radical changes of input parameters under asingle perspective. This is again, a finding of some significance with regard tostudies which assume a single ‘optimal’ configuration for diverse future energy
systems. More detailed conclusions concerning the most favourable structures fordiverse future energy portfolios in each national context, would require moreelaborate and wide-ranging interviews than it has been possible to undertake inthis short pilot study. However, the results do suggest that a meaningful andpotentially quite significant input to policy making might be obtained by thismeans. In short, it does appear that this analytical framework shows somepromise, as a way to be more systematic and transparent in articulating a rangeof different salient perspectives. In the end, the value of such a framework lies notin prescribing decisions, but simply in informing more robust, rigorous andaccountable policy deliberation.
In methodological terms, the dedicated software package for multicriteria
diversity analysis developed through this pilot project has performed successfully.
Although very demanding on interviewees, it is capable of yielding policy-relevant
findings on the basis of a broadly feasible interaction with busy high-level experts.
The method allows articulation of a wide range of highly complex factors in a waythat is broadly intuitive for specialists. Of course, a range of more detailed issues
have arisen concerning the appropriate implementation of the interview
procedure, the use of the software and the conduct of the analysis – as well as thedesign of a diversity analysis study as a whole. These may usefully inform future
larger-scale initiatives. In short, the method appears to offer an operational andpotentially robust approach to unpack the otherwise often highly obfuscatedconcept of energy diversity.
It is also worth observing that – aside from the methodological and empiricaloutcomes themselves – the process of undertaking this kind of analysis can yield
benefits for participants and analysts alike in terms of reflexive learning about
62the issues in hand. In particular, an appreciation for the nature and sources ofdifferences between perspectives offers a potentially important way to focus, in anotherwise sometimes prohibitively complex field, on those issues that are ofgreatest policy (and political) salience.
Finally, this pilot study has revealed a number of important areas for further
research. As discussed in Section 4-4, since diversity is in large part a strategy for
responding to uncertainty, there is the potential to make greater use of theinterval data obtained for uncertainties in option performance. Techniquesdeveloped in applied mathematics, operational research and risk studies bringtheoretical insights and computational techniques that may prove highly relevant
to methodological development in this area.
It only remains to conclude this pilot study. As is discussed in introducing thisreport – and further shown by the results of the multiple appraisals andsensitivity analyses reported here – decision making over the future compositionof diverse energy mixes is fraught with uncertainty. A prerequisite for robust
responses to such uncertainty is the promotion of more systematic andsophisticated deliberation over the nature and implications of energy diversity –and the contending means to achieve it. Efforts towards this end, should have thecapacity symmetrically to accommodate the disparate array of expert perspectives
and associated uncertainties, without circumscribing the options that may beconsidered or privileging some particular scenario or viewpoint. They should
combine the rigour and precision of quantitative analysis with the open-ended
flexibility and unconstrained fidelity of qualitative approaches. They should
convey the full plurality of implications of different equally-valid technicalperspectives, yet also highlight the underlying convergences and common ground.
It is in this way, that methods for the analysis of energy diversity might best yield
clear practical messages for policy-making, expressed in terms of the available
resources and technologies and respecting their many complexities and limits. It
is in all these various regards that we would argue that the diversity analysis
methodology piloted in this study, represents a potentially significant first step
forward.
63

Appendix 1. The Multicriteria Mapping (MCM) Process
The MCM process has four stages, although it is possible to go back and review,
add or remove things at any stage. The information will be recorded on a laptop
computer and we can leave with a participant a copy of the MCM software andtheir own assessment after the interview if they are interested.
This note will explain the main stages in the MCM process and then provide a
concrete example from energy generation options, where the UK Government
wishes to take advice over its policy priorities concerning which electricity
generating options to encourage, and which to discourage.

Stage 1. Defining the Options
The options represent a range of possible things that could be done in a certain
situation. For the present exercise on energy generation options we have several‘core options’ that we would like participants to evaluate. We are specifying thesein order to enable us to make comparisons across the perspectives of different
participants.
In this example, the 3 core options were:
1. Nuclear power
2. Coal burning
3. Wind energy
However, there may be other options they believe should be considered and they
will be asked if there are others they would like to add.
Finally, participants are entirely free to add further options of their own choosing,defined as they wish, at their own discretion. These may involve slight
permutations on some of the options specified above, or may be entirely different.

64

Example
One specialist undertook an MCM exercise as part of this exercise. The specialist
added ‘gas generation’ to the list of the above core options because of theimportance of gas to current energy production in the UK (Figure 30).

Figure 30: Choosing the Options

Stage 2. Defining the Criteria
The criteria are the different elements that participants consider when theychoose between, or compare, different courses of action. These may address any
issue that they feel has relevance to their assessment of the performance of thedifferent options. For instance, in the present case, they may involve economic,
environmental, social or geopolitical aspects.
Participants are entirely free to identify and define their own criteria as they
think fit. However, it is important to be as specific as possible in their definitions,
to be clear about the differences between criteria and to minimise any overlaps ordependencies. Any residual minor overlaps or dependencies between criteria canbe dealt with as uncertainties in assigning scores (see below).
65

It is recommendable that participants restrict themselves to identifying twelve
criteria or less. The interviewer will ask them to explain what they mean by eachcriterion and why it has been chosen. Criteria definitions will typically become
more clear and detailed as scoring proceeds, so they are free at any time to amendor refine their criteria.
ExampleThe specialist decided upon the following criteria:zworker safety: incidence of fatal or serious injuries or disease across whole‘fuel cycle’ (from mining to waste disposal)zpublic health: incidence of adverse public health effects due to emissions,wastes or accidents (excluding global warming effects)zcontribution to climate change: equivalent carbon production taking into
account whole fuel cycle and material and energy use during constructionzelectricity cost: taking account of capital costs, fuel cycle costs and waste
management costs under prevailing market conditions

An additional criterion introduces an issue of principle as follows:zmaximum accident: a limit on the maximum extent of acceptable damage
arising from a single possible accident. Set at costs in excess of £ 15 billion or
total committed public mortality in excess of 10,00066Figure 31: Defining the Criteria

Stage 3. ScoringCriteria are issues under which participants can evaluate the relative
performance of the difference options. Typically, they can express this by using
numbers to rate the performance ‘scores’ of different options on a scale. On theother hand, a criterion may sometimes reflect an issue of principle, which does not
admit relative ratings, but simply allows them to say whether an option is
acceptable or not.
Where participants feel able to come up with a relative ordering of different
options, they can use any scale they like in assigning scores. For instance, thismay range from one to ten, or one to a thousand. We recommend a scale of one toone hundred. Either way, the higher the number, the better the performance.
Fractions may also be entered.
Usually, participants express their technical performance evaluations using a
subjective personal scale under each criterion. Sometimes, they may wish directly
to consult suitable quantified information, such as cost data. But it is important to67

remember that their judgements of performance (and so their scores) need nottranslate directly from this data. However they approach this, the scoring follows
a simple rule. An option that is assigned a score of eight is judged to perform twice
as well under that criterion as an option that is assigned a score of four.
We will ask participants to score each option under each criterion by assigning a
number in this way. Of course, it is often difficult to be sure about this. They may
feel uncertain about the performance. Or the performance may depend in some
way on the circumstances or some other particular assumptions. In order toenable them to express this kind of uncertainty or variability, the MCM processlets them assign two scores for each option under each criterion. One to reflect themost optimistic end of their judgement over likely performance, the other toreflect the most pessimistic end.
As they proceed with their scoring, the interviewer will ask them to clarify thereasons for their decisions on the relative performance of the different options.
Here, we are interested in their ‘technical’ justifications for their judgements.
With two scores to be assigned for each option under each criterion, the scoringprocess is the most time consuming part of the MCM process.
If they find that a criterion does not lend itself to quantification or ‘trading off ’against other criteria, but instead reflects an issue of principle under which
different options are either acceptable or not, this can also be accommodated by
the MCM process. Here, they simply identify that criterion as a ‘principle’. You arethen invited to register those options that are in some way inadmissible underthat criterion.
This can also be used as a way to reflect absolute performance thresholds. Above
the threshold, performance can be expressed by a scoring process as describedabove. But options whose performance falls below this level are ruled out in thesame way as under an issue of principle.
As participants proceed through the scoring process, the interviewer will note the68reasons for their judgements and may prompt them to clarify or elaborate certain
points.
ExampleThe specialist scored each option under each criterion. A scale of 1-10 was chosen,with 10 being good and 1 being bad. Where she was uncertain, or where her
judgement of likely performance depended on particular assumptions about thecontext, she assigned a pessimistic (low) and an optimistic (high) score. Based onher understanding of the available data, the specialist explained some of herscoring processes this way:
“Worker safety in nuclear power is generally very good but there can be accidents
although these are very rare. I’d score worker safety for nuclear as 6-8.
Worker safety for coal is not so good. Mines can collapse and many miners can be
killed. I’d score worker safety as 4-7 for coal…….
“Nuclear and wind power don’t contribute to climate change unless the energy
used in their construction is produced by burning carbon, so they both score
between 9 and 10. Coal is the worst and so scores 1-3. Gas is somewhere in
between, scoring 3-5…….
“Nuclear power is the only option that presents risks of a type of disaster that,
although very unlikely, is beyond the threshold of what is acceptable to society, so
it is ruled out on principle in relation to maximum accidents”
The final set of performance scores arrived at by this specialist were as follows:
ScoresWorkerPublicClimateElectricity

Maximumsafetyhealthchangecost

accident

Nuclear

5-6

3-8

9-10

5-6

Ruled out

Coal

4-5

4-5

1-3

6-7

Gas

5-6

7-8

3-5

7-8

Wind

5-6

8-9

9-10

5-7

69

Figure 32: Scoring

Stage 4. WeightingThis is when we ask participants to tell us the relative importance of theperformance judgements they have made under different criteria. For example
cost may be more important to you than safety or vice versa. The way we express
this is by assigning a numerical ‘weighting’ to each criterion. If something is half
as important as something else, it will be assigned half the number of points.
This business of weighting is very different to scoring. Scores reflect relatively
technical judgements about the relative performance of options under individual
criteria. Weights reflect essentially subjective judgements about the relative
importance of the different criteria themselves. For instance, how muchsomething costs (a score) is a very different matter to the relative importance of
cost and, say, safety (a weight).
When making these kinds of judgements, it is important to bear in mind exactly
what is being compared with what. For instance, a judgement over whether cost is70

more important than safety will depend on “how much cost?” and “how muchsafety?”. Even if safety is felt to be a key issue, a tiny improvement in safety may
nonetheless be considered less important than a massive improvement in cost – soparticipants might end up assigning a bigger weight to the large cost difference
than to the small safety difference.
For this reason, judgements about weightings are based on the difference in thescores for the best and worst option under each criterion. It is the relative
importance of these performance differences that are being compared andweighted. To help participants, the MCM software reminds them of the worst andbest performing options under each criterion and the particular assumptions thatthey made in assigning these scores.
There are a number of ways to go about assigning these weightings. One way issimply to share out some round number – say 100 – ‘importance points’. Anotherway is to start with a particular criterion and then weight all the others in
relation to this. We recommend that they start by assigning a weight of ten to theleast important criterion and then move on to the next most important and so on.
At any time, the MCM software allows them to scale the weightings that they
have assigned as proportions of a total of 100 ‘importance points’. But theirweightings can total to any number they like.
Where they have identified a criterion as being an issue of principle, then theycannot ‘trade off ’ performance with other criteria. Here, an option is either
acceptable or not. For this reason, such issues of principle are not part of theweighting exercise.
ExampleBearing in mind the magnitude of the differences between best and worst option
under each criterion, the specialist first identified the criterion that was least
important to her. Worker safety and climate change first seemed fairly equal inthis respect. But when she noticed that the difference between best and worst
options under health (between 3 and 9) was significantly larger than under71

worker safety (between 4 and 6), she decided the latter difference was lessimportant. She therefore assigned worker safety ten ‘importance points’ andclimate change twelve ‘importance points’.
She then thought about which were the next most important criteria. She settled
on the difference in public health performance being twice as important as thedifference in workers safety performance and so assigned it twenty points. To her,the difference in financial costs was the most important single criterion, and she
assigned this forty points.

Figure 33: Assessing the Weighting
The outcome
At the end of the process the computer will generate a simple picture of how theoptions perform overall under all their criteria scores and weightings. Effectively,
this involves multiplying the scores by the weightings, taking into account thedifferent scales that have been used.
A simple chart shows the relative performance of all the different options, as wellas the range of uncertainties associated with each. Where an option has been72

judged inadmissible under an issue of principle, then this is also clearly displayedon the chart.
Participants will be able to use this chart to help them decide how their picture of
relative performance concurs with their general expectations. They will have theopportunity to express any surprises they may feel and then explore the model tosee what the reasons are. For instance, they will be able to test the effect ofchanging the weightings. This may help them settle on a pattern of weightings
which better reflects their viewpoint. During this process, the interviewer will ask
them to explain the reasons for any judgements they make.
In the end, the outcome is a detailed picture of the relative performance of all theoptions that they have defined, reflecting their choice of criteria, their technicaljudgements over the performance of their options under their criteria, theiruncertainties and their subjective priorities concerning the relative importance ofthe different criteria.
Later, their MCM results will be compared with those of other participants. We
will look for similarities and differences and consider what these together say
about the various options and perspectives. The results will form a basis forhelping to inform, clarify and structure continuing discussions over the best way
forward.
ExampleAs a result of all these inputs, the final chart shows a pretty clear distinction
between relatively high ranks for wind and gas and a relatively low rank for coal
(Figure 33). The ranking ranges for wind and gas show considerable overlap.
Focusing on the detail, the chart shows that gas – at the most favourable end of
its range – offers the best option overall. The top end of the range for wind power
comes next. However, the pessimistic end of the range for wind power ranks
significantly worse than that for gas. Despite the overlap, this suggests that – forthis participant – wind is a second best option to gas.
73

The range for coal burning overlaps a little with the low end of the range for wind,
but is clearly lower than the range for gas. This places coal third in the rankings.
Nuclear power is ruled out under the ‘maximum accident’ criterion and so ranks
last among these four options.
In order to explore some of the hidden implications of her judgements, theparticipant decides to see what would happen if she slightly changed the
maximum accident principle so that nuclear would no longer be excluded. In thiscase, it becomes clear that the ranking range for nuclear would overlap
considerably with coal. Indeed, tho’ the low end of the range for coal is slightly
higher than the low end of the range for nuclear, the high end of the range fornuclear is significantly higher than that for coal. Under these new conditions,then nuclear is placed third overall, with coal becoming the lowest ranking option.
As a final exercise to test the robustness of this final ranking picture, theparticipant decides to explore the effect of varying her criteria weightings. Firstsaving a file recording the weights settled on already, she uses the sliding scale tomove the weightings to and fro. She notices that, if climate change were to become
a much more important issue, then nuclear could readily move into second place
position overall.
The participant remains content that her original set of weights and rankings
does meaningfully reflect her own perspective at the moment on the key issues
bearing on the performance of energy options. However, she is struck that the full
attention to her uncertainties at each stage in the exercise reveals that there ismore scope for overlap than she had expected. Indeed, depending on the detailed
assumptions and conditions, it would be entirely consistent with her appraisal tonote that gas, wind or even nuclear power might possibly prove the highest
ranking option.

74

Appendix 2. Excel Macro Program and Spreadsheets
A dedicated Excel macro program has been devised for this pilot study with theaim of quickly converting output data from the MC-Mapper software to input data
for the Matlab diversity analysis program. The original version of this interface
program was written by Toby Champion to a basic design by Andy Stirling andthen substantively upgraded by Go Yoshizawa. In its present form, it can convert
the data in two ways: as ‘basic mode’ and an ‘advanced mode’.
The basic mode, offers a relatively convenient way for a participant to quickly geta grasp of the process as a whole. It uses a single ‘Basic’ worksheet in the Excel
interface program. Without attending to various potentially significant details, it
proceeds quite quickly to yield a provisional picture of the disparity structure andcorresponding range of diverse portfolios that are implicit in the raw structure of
the participant’s own appraisal. In particular, it does not allow participants to add,
partition or re-characterise energy options, to define additional disparity
attributes, to express the natures of various kinds of portfolio interaction between
options, or to conduct sensitivity analysis on key aspects of performance ordiversity. The basic mode takes only 5-10 minutes, thus both offering to save time
for busy participants, as well as form a good basis for the participant to return tomore detailed scrutiny using the advanced mode.
The ‘advanced’ mode, is more time-consuming and complicated. Usually beginningwith a quick run in ‘basic’ mode, participants go through several worksheets inthe Excel interface program. First, there is the ‘Options’ worksheet, in which
participants register detailed characterisations of the options themselves,
including subordinate tranches, their constraints. Then there is the ‘Disparity’worksheet, in which the participant can review the normalised performance
criteria used to create a disparity structure, experiment with criteria weightings,
and add any further attributes of disparity to address aspects neglected inperformance appraisal. This feeds directly in to the ‘Dendrogram’ worksheet,
which simply illustrates a picture of the disparity structure yielded by theperformance appraisal and any additional disparity attributes that have been75

added.
The next stage is the ‘Interactions’ worksheet, which allows the participant toexpress the aggregate effect of various forms of positive or negative interactions
between each pair of options. Finally, there is the ‘Sensitivities’ worksheet, whichallows the participant or analyst to experiment with the way in which variationsin the many parameters of diversity analysis lead to variabilities in the structureof the resulting portfolios. Throughout the sequence of worksheet tasks, thecomplexity is somewhat eased by the use of drop-down menus in the Excel
interface and by assistance from (or direct mediation by) the interviewer. Aided by
the overview diagram (Figure 34), the ensuing discussion here will now go
through each of these methodological steps in more detail.

MC-Mapper

MC-Mapper Output Data
Basic

Start
Advanced

Options
Identify option tranches

Basic
Set basic parameters

Disparity
Characterise and weightdisparity attributes
Dendrogram
Illustrate option disparities

Yes

Chart?
No

Interactions
Define option interactions
Sensitivities
Explore portfolio sensitivities

MATLAB input Data

MATLAB

Figure 34: Flowchart of the Excel Macro Program76

1. Setting Basic Parameters (Basic Sheet)
As explained above, this worksheet enables a participant to skip many steps byallowing them to concentrate only on the most essential data and the setting ofindividaualintermittency

carbon reduction

rapid dispatch

Nuclear

step

Note

basic parameters. The data required in this mode simply concern the following.

0.000

0.30

0

0

0

LNG

0.000

0.20

0

0

0

Coal+CCS

0.000

0.15

0

0

0

Oil

0.000

0.05

0

0

0

Coal
Hydro

0.000
0.000

0.15
0.05

0
0

0
0

0
0

Waste

0.000

0.08

0

0

0

Wind

0.000

0.08

0

0

0

Solar PV

0.000

0.05

0

0

0

Solar Thermal
Geothermal

0.000
0.000

0.02
0.01

0
0

0
0

0
0

Biomass

0.000

0.02

0

0

0

Ocean Energy

0.000

0.01

0

0

0

LNG+CCS

0.000

0.01

0

0

0

Maximum 1.00 1.00 1.00

Figure 35: Constraints Section
First, there is the ‘step scale’ that is characteristic of each option (i.e. theminimum unit size in which that option is available). This is entered using theorange-shaded cells in the Basic sheet (as illustrated in Figure 35). It is expressed
as a proportion of the portfolio mix as a whole, under whatever metric is held to be
most relevant (whether capacity, output, investment – in this case it isapproached in terms of output). The value will typically be high for nuclear power
but small for wind. For example, if an evaluator estimates that the installation ofa nuclear power plant contributes to 1% of the total electricity generation from all77

energy sources in the issued region, the evaluator should fill “0.01” in the “step”
cell for the nuclear option.
Second, there are the constraints that are held to apply to the contributions ofindividual energy options. This is entered using the lavender-shaded cells in the
Basic sheet (Figure 35). This represents the maximum extent to which the option
can contribute to the total electricity generation. In order not to overly constrain
or confound the dynamics of the optimisation process in the later diversity
analysis, it is advisable to make only the most measured use of this ‘constraints’
parameter – for instance reflecting concrete physical resource limits rather thanless tangible social, political and cultural aspects.
Third, there are constraints acting on the contributions of groups of options (forinstance, concerning the collective contribution that is acceptable fromintermittent renewables). This is entered using the green-shaded cells in Basic
sheet (as illustrated in Figure 35). For the purposes of the present pilot study,
intermittency, carbon emissions reduction and rapid dispatch were suggested inadvance as possible collective constraints. If, for example, a participant wishes to
set an upper limit for energy options with high carbon dioxide emissions in orderto forestall global warming, an upper system contribution is set in the separate
row of green cells alongside the word “Maximum” at the bottom of the block ofgreen cells (Figure 35). The options to which this collective constraint applies arethen each indicated by entering “1” in the corresponding green columns cells
above. This field thus allows only the binary entries, 0 or 1.
Fourth, the only other parameters that need to be specified in this Basic mode
involve a series of ‘housekeeping’ issues to do with the plotting of diagrams andthe configuring of the diversity heuristic. These inputs from the Basic worksheet
are shown in Figure 36 below.

78

minDelta 0.1
maxDelta 100
nDeltas 500
deltaScale log
alpha 1.0
beta 1.0

Figure 36: Basic Parameters Section
There is also the option of setting the values taken by the terms ‘alpha’ and ‘beta’in the Stirling diversity heuristic, which reflect different prioritisations ofdiversity attributes (though the default values of alpha = beta = 1 serves as arepresentative diversity index).
The rose-shaded cells are used to frame the plotting of the diversity-optimal
portfolios. The first two set the minimum (“minDelta”) and maximum(“maxDelta”) values to be used for the parameter ‘delta’ ( δ ), a scaling coefficientwhich expresses the trade-off between performance and diversity. The thirdrose-shaded cell (“nDeltas”) determines the number of values that are plotted forδ , which determines the resolution of the range of portfolios. In other words, allpossible conditionally optimal portfolios from those that maximise value due toaggregate performance of individual elements at the point of minDelta, to thosethat maximise value due to portfolio interactions and system diversity at the
point of maxDelta. In the fourth rose-shaded cell (“deltaScale”), the participant or
analyst may enter the form of the plot required for displaying the range ofdiversity-optimal portfolios – either on a linear or a logarithmic scale.
Finally, the lavender-shaded cells in this section (Figure 36) allow the participant
or analyst to enter the values for the exponents used in the Stirling diversity
heuristic, α (“alpha”) and β (“beta”). These give the relative emphasis placedin the characterisation of diversity, on the subordinate properties of disparity (andassociated variety) and balance. Values of α = β = 0 yield a pure index ofvariety. Values of α = 1; β = 0 yield a pure measure of disparity. Values of α =79

0; β = 1 yield a generic index of balance. Values of α = β = 1 yield a genericindex of diversity. Shifting values of alpha and beta between 0 and 1 collectively
addresses all possible relative weightings on variety, balance and disparity. The
default values used throughout this pilot exercise were those of the generic index( α = β = 1).
Having reviewed these entries in the Basic worksheet and satisfied themselvesthat the inputs are sufficient for a provisional run, the participants then click the
button and the program automatically creates an Excel file comprising
the input data for the Matlab diversity analysis program. The calculation processembedded in this worksheet automatically converts performance scores (which
are loaded from the MC-Mapper output data in the form of Excel worksheet) into
disparity attributes. The process does not allow re-sorting, adding, duplicating
and deleting energy options, adding disparity attributes, changing interaction
coefficients and conducting sensitivity analysis. Therefore all interaction
coefficients are fixed as 1 and the median value is taken from the performance
score.

2. Identifying Option Tranches (Options Sheet)This worksheet allows modifications to the characterisation of energy options as
originally defined for the purposes of appraisal in the MC-Mapper software. In
short, it allows the re-sorting, adding, duplicating and deleting of energy options.
The sheet therefore revisits the constraint values, which are automaticallymirror-copied from the Basic sheet, where these have already been entered.
However, the participant can also set new values under each of these parametersin the Options sheet, in which case, these will automatically be mirror-copied to
the Basic sheet, as soon as this is opened.
The form of the raw MC-Mapper appraisal data that is imported in the Options
sheet is illustrated in the following table (Table 3). The table contains lowest andhighest values of performance scores and performance weightings.
80

Table 3: MC-Mapper Output Data Field
Criterion 1

Criterion 2…

Criterion c
performance

Weighting… wc

Option 1

2

Option 2

4…

Criterion n

…

4
6
min
2

7

…

1 S

Option i

3

8

…

6

9 S2

Option m

56… S ic

S ic

…

max

S ic : lowest value of the performance score for option i and criterion c
S ic : highest value of the performance score for option i and criterion c

wcperformance : performance weighting for criterion c
The lowest and highest performance scores for each option under each criterion
are transferred in their raw form from MC-Mapper into the light yellow andyellow -shaded cells respectively in the “Options” sheet (Table 4). Each
performance score interval is represented as follows.
S ic = [ S ic , S ic ]For the calculation of disparity, the median of the performance score interval iscalculated as follows and displayed in beige-shaded cells of the “Options” sheet(Table 4).
Sicmed = ( S ic + S ic ) 2

Criteria weightings are also imported into beige-shaded cells, this time moved forease of representation to the bottom of the table (Table 4).

81

Table 4: Performance Data Field in the Options Sheet
Criterion 1…

Option 1

2

1

3

Option 2

3

1

5

3.5

3

4

3.5

2

5

Criterion c…

Criterion n…
Option i…

S icmed…

wcperformance

S ic

S ic

…

…
Option m

Weighting…

Options can be re-sorted in this sheet by clicking the up-arrowanddown-arrowicons positioned at the upper-left corner of the worksheet when
the relevant option name is selected by the cursor. When clicking the up-arrowicon, the selected option goes up and is exchanged for the option locatedimmediately above. A new option can be added by clicking the icon
at the top
in the set of icons. A dialog box appears and requires the operator to enter the
option name. Data for steps, collective constraints, and performance scores are set
as a default at 0. Data for individual constraints are set as a default at 1.
All data for an option shown in this worksheet (i.e. individual steps, option
constraints and performance scores) can be duplicated by clicking the second iconfrom the top, but only when the name of the option to be duplicated is selected.
This duplication function can be used to break options down into “tranches”. New
tranches will have the same performance as their parent option, but may havedifferent steps and/or constraints, or even lower or higher performance values
under certain criteria. All data for an option can be deleted by clicking the crossicon
at the third from the top, where the name of the relevant option is
selected.
The purpose of this worksheet is to allow detailed re-examination of the way inwhich options are characterised and partitioned, as well as to permit
82consideration of step-sizes, individual constraints and collective constraints inoption contributions.

3. Characterising and Weighting Disparity Attributes (Disparity
Sheet)
This worksheet takes the form of a matrix containing two differently-shaded cells.
Those shaded beige colour display normalised performance score values from the
Options sheet. The normalisation renders the overall ranks equal for all options,but preserves the proportional contributions to these ranks from different
performance criteria. In this way, it contains the disparity structures that areembedded in these scores, without introducing bias due to the fact that certain
options rank better overall than do others. Default weights for each of these
normalised criteria for the purpose of characterising disparity, are set at 50, but
can be changed in subsequent analysis. Also displayed in this characterisation
matrix (using light green cells) are columns for entering a series of further
disparity attributes and corresponding weightings. The weights for all thesedisparity attributes can be altered using vertical slider controls displayed underthe matrix.
It appears from this pilot exercise that many participants experience difficulty indiscriminating between the idea of characterising disparity and that of appraising
performance. The option of adding further disparity attributes (in the light green
cells) is therefore not often taken. In such cases, the disparity structures
employed in diversity analysis are yielded entirely by the implicit disparities in
the MCM appraisal data.
In more technical terms, the analysis of disparity is undertaken in the Disparity
sheet in the Excel file in the following fashion. As discussed earlier, the raw
performance scores imported from MC-Mapper may be represented as follows (seeTable 3):83

Scmin = min{S 1c ,L, S mc }Scmax = max{S 1c ,L, S mc }where 1 ≤ i ≤ m

For the purpose of appraising performance, scores and weights are normalised (asthey are in MC-Mapper) as shown below:

sic = Snorm
ic

Sic − Scmin= max
Sc − Scmin

wc = wcnorm =wcperformance∑ wnperformancecc =1

Whilst the overall performance rank for each option is given (as explained above)as the sum of these weights and scores, it is the performance ‘sub-rank’ for each
option under each criterion that is employed in further normalising these scoresto yield disparity attribute values. These sub-ranks can be expressed as follows.

ric = wc × s ic
This further normalisation of performance scores to derive disparity attributes isconducted in the following fashion:

wcdisparity : disparity weighting for criterion c
Dic : disparity attribute for option i and criterion c

Dic =w cdisparity sic

∑ w
n

c =1

disparity
c
ic

s

84

Table 5: Disparity Data Field in the Disparity Sheet
Attribute 1… Attribute c

Option 1

0.4

Option 2

0.5 Dc… Attribute nAttribute n+1… Attribute Nmax…
… 0.3 Dic

Option i

…

…

…

…

min
c

0.1 D

…
Option m

0.2

Weighting…

wcdisparity

As mentioned above, cells for additional disparity attributes and theircorresponding weightings are shaded in light green in the Disparity sheet (Table5). In this worksheet the participant can add new attributes (attribute n+1 toattribute N) and their weightings for the further characterisation of energy
options.
In the Matlab program, disparity is addressed as the squared Euclidean distance
between different options, represented as co-ordinates in the space defined bytheir corresponding disparity attributes as displayed in the Disparity worksheet.

d ij =

∑ (w
n

c =1

disparityc) 2 ( Dic − D jc ) 285

4. Illustrating Option Disparities (Dendrogram Sheet)This worksheet shows a dendrogram, which reflects the degree of disparity
between different options, according to the disparity attributes that have beenentered into the Disparity worksheet. This ‘tree diagram’ helps the participant
grasp how individual energy options are clustered together in the ‘disparity space’defined by the various attributes. As a default, the dendrogram is calculated using
Ward method, or squared Euclidean distances.
It is important to recognise that this constitutes just one (arguably the mostappropriate) of a number of clustering methods that might be used to represent
thecomplexmultidimensional

distributionofdisparities

inasimpletwo-dimensional tree structure. The representation is thus not perfect, and loses
information when compared to the original multi-dimensional distribution itself.
It is the full set of disparity data that is employed in the diversity analysis itself,
however, with the dendrogram simply providing a convenient visual guide to theunderlying structure. In particular, this simple graphical representation helps theindividual participants reflexively to understand the structure of option
disparities that is implicit in their performance appraisal.
Following the construction of the dendrogram a dialog box appears which allows,
options to be re-ordered in all worksheets in the same sequence. Aside from thisfinal step, the Dendrogram worksheet is simply for information – not requiring
any deliberate data entry or modification.

86

5. Defining Option Interactions (Interactions Sheet)This worksheet allows the participant to determine the interaction term in thefull portfolio equation. Represented as ι ij , this is an additional parameter toreflect the effect on system value of synergies or tensions between elements i and j.
The net positive or negative effect of a range of different possible interactions isexpressed as a marginal departures from a default of unity ( ι ij = 1 ± ∂ι ). For most
systems ∂ι << 1 .
The inputs for this purpose are made in a half matrix of pale blue –shaded cells(Table 6). These are pre-set with a default value of 1 (indicating no net
interaction). The diversity analysis method provides no definitive way objectively
to determine the magnitude of this value. Instead, it simply reflects the product ofexpert deliberation over factors such as system interactions, operational factors,
industrial relationships, economies of scale and scope and so on.
For an array of m options, the table in this worksheet (Table 6) shows ahalf-matrix of m by m-1. The number of values that may be determined in thisway is thus m(m-1)/2. Typically, only a small sub-set (if any) of these are judged tobe relevant by the participant.
Table 6: Option Interactions Data Field in the Interactions Sheet
Option 1

Option 2…

Option i…

Option m-1

Option 1
Option 2

1.0

…
1.0

ιi 2

Option m-1

1.0

ι( m −1) 2

ι( m −1)i

Option m

1.0

ιm 2

ιmi

Option i…

87ι m ( m −1)

6. Exploring portfolio sensitivities (Sensitivities sheet)This final worksheet helps the participant or analyst to perform sensitivity
analysis on the main parameters determining the structure of the diverse
portfolios. Since both the disparity structures and the performance profiles aregoverned by the MCM performance appraisal – and these are represented as
ranges – the sensitivity analysis focuses on the effect of taking different specificvalues with these ranges. It begins by displaying the median values for theseranges, as used as defaults in the diversity analysis. By moving sliders presented
in this sheet we can change any performance score from the lowest value(pessimistic) to the highest value (optimistic) accommodated in the performanceappraisal for each energy option. We can also perform the same sensitivity
analysis with respect to performance criteria (rather than options) – with sliders
provided to test the effect of changed criteria scores and weightings. In this way,
the three kinds of sliders allow consideration of the systematic effects of optimism
or pessimism across different options or criteria.
(1) Vertical sliders for eliciting scoring sensitivity by performance criteria(2) Horizontal sliders for eliciting scoring sensitivity by option(3) Vertical sliders for eliciting weighting sensitivity by performance criteria
Cells displaying the magnitude of the sensitivity factor in each case are shaded
yellow-brown and labelled “Sensitivity”. For ease of reference by the participant
or analyst, the correspondingly modified values for the scores and weightings are
displayed in the associated beige-shaded cells.

88

Table 7: Portfolio Sensitivities Data Field in the Sensitivities Sheet
Criterion 1…

Criterion c…

…

S ic ( χ i , χ c )…

Sensitivity

…

χc

…

Weighting…

wc ( χ cw )…

Sensitivity

…

χ cw

…

Criterion n

OverallSensitivity

S ioverall

χiOption 1…
Option i…
Option m

For ease of calculation, it is the median of the performance score range that is
adopted as a default in the diversity analysis. This sensitivity analysis thus
enables the participant or analyst to examine to what extent uncertainty inperformance score affects the composition of the final series of diversity-optimal
portfolios. In other words, it shows how portfolio composition responds to theuncertainty expressed in the appraisal of the energy options. This is represented
as follows in Table 7:χ c : performance sensitivity for criterion cχ cw : weighting sensitivity for criterion c
The sensitivity analysis for options extends the performance score from lowest
value to the highest value, based on the weighted minimum and maximum scores
under each criterion respectively. In other words, the lowest end of the sensitivity
scale should correspond with assigning the lowest value in the performance scoreinterval for each option under that criterion. Likewise, the highest end of thesensitivity scale should correspond with assigning the highest value in theperformance score interval for each option under that criterion. The weightings ineither event remain unchanged.
89This sensitivity analysis is structured on a scale from 0 to 100. If the performancesensitivity for a given option or criterion is 0, then the relevant performance score
takes the lowest value in the associated range yielded in appraisal. If thesensitivity is 100, the performance score takes the highest value in this range. If
the sensitivity is 50, the performance score takes the median value, which is thedefault used in the diversity analysis. The performance sensitivity by option andby criterion function individually.

Sic ( χ i , χ c ) = S ic +S icnorm ( χ i , χ c ) =

χi χc
100

S ic +( S ic − S ic )χi χc

( S ic − S ic ) − S cmin100
S cmax − S cmin

By contrast with this, the sensitivity analysis of weightings is independent of thescoring ranges. Here, the sensitivity analysis explores a difference between lowest
and highest weighting amounting to a total factor of one order of magnitude. In
other words, the lowest end of the weighting sensitivity scale corresponds with the
product of the normalised weight and reciprocal root ten. Likewise, the highest
end of the weighting sensitivity scale corresponds with the product of thenormalised weight and root ten. In performing this sensitivity analysis, all othercriteria weights are preserved ceteris paribus – i.e. with the original normalised
values. In other words, only the values for the selected criterion are changed, andthe new set of weightings subsequent to the selected sensitivity value for the
chosen criterion, are then re-normalised to sum to one.

wcperformance
(
χ
)
=
wc
∑c wcperformancewcwcnorm ( χ cw ) =( 10 )

χ cw
50

−1

wc ( χ cw )∑c wc ( χ cw )For ease of reference by the participant or analyst, the overall performance score90

yielded by the sensitivity multipliers for each option under each criterion are also
displayed in this sheet. This displays the option that is thereby attributed thehighest overall performance score. It is this option that will be dominant indiversity analysis at the performance-maximising extreme of the range ofdiversity-optimal portfolios.

Sioverall = ∑ w c (χ cw ) Sic ( χ i , χ c )c

The normalised performance rank r with sensitivity is calculated as follows.

ric = wcnorm ( χ cw ) × Sicnorm ( χ i , χ c )v portfolio = ∑∑r p + δ ∑ij ( i≠ j ) d ijιij pi p jic ic i

When the sensitivity analysis is completed, the participant and / or analyst should
check through the worksheet as whole. As a final stage in the diversity analysis
process, it is wise to make a unique file containing all the relevant data settings.
For this purpose, it is possible to select “Make datafile” from the “Appraisal” menu
in the top menu bar.
A pop-up dialog box will also at this stage provide the option of changing thedefault set of colours that are used in the Matlab program to display the ranges ofdiversity-optimal portfolios. This will be useful, where the neighbouring options
are coloured inconveniently, or where it wished to highlight or associate different
kinds of option by means of colour. The colours are also displayed at the left-handside of option names in the Sensitivities worksheet.
The Excel macro program produces as an output the file “data.xls” and overwrites
the corresponding file located as default in the folder “C:/MCMdiversity/mainline”.
This output file contains performance data, disparity data, interaction data andbasic parameters, for all of which values are set in the Excel program.

91

Matlab is a numerical and computing environment and programming language.
The fundamental part of the Matlab optimisation code was written by Toby
Champion based on an original basic format by David Waxman and modified withminor additions by Go Yoshizawa. All output figures generated by the portfolio
analysis are illustrated in Appendix 3.

92

Appendix 3. Matlab Program Output Figures
The Matlab program produces the following seven output figures.
z

Figure 1 is a graph entitled “Ratios”, which displays on the x-axis the values
taken by δ , the coefficient which scales diversity against performance in thediversity optimisation equation. This is plotted against a y-axis displaying the
product V {E} V {P} . This reflects the ratio between the two main componentsof portfolio value as also characterised in the diversity optimisation equation.

V {S } = V {E} + V {P} = ∑i ∑c ( wc ⋅ S ic ) ⋅ pi + δ ⋅ ∑ij ( i ≠ j ) (d ij ) α ( pi ⋅ p j ) ⋅ ι ij
βHere, V{E} is the aggregate portfolio value due to the sum of the performanceof the individual options and V{P} is the corresponding value due to portfolio
diversity. This provides one way to show the relative scale (and identify any
interesting features) in the relationship between these two key components ofportfolio value.

93

z

Figure 2 is a graph entitled “Efficient Frontier”. This shows on the x-axis theaggregate portfolio value due to the sum of the performance of the individual
options (V{E}) against a y-axis displaying the value due to portfolio diversity(V{P}). This shows as a ‘Pareto frontier’ the co-ordinates of those portfoliosthat display either a maximum value due to portfolio diversity or a maximum
value due to option performance. Portfolios plotted below this curve are
inefficient in one or other respect. It is this curve which provides the source forthe range of diversity-optimal portfolios, which form the centre for analysis(See under Figure 19 below).
By displaying the shape of the efficient frontier in this way, we gain anotherview of the relative scale of the two main components of portfolio value as
characterised in the portfolio optimisation term This also provides aconvenient way to identify interesting features in the trade-off between thesetwo parameters. Where the frontier displays loops or saw-tooth features, forexample, there is an indication of an underlying difficulty in the optimisation
process, which can be addressed by checking the input parameters.

94

zFigure 3 is titled “Indices”, generating four graphs displaying on the y-axis
various diversity-indices; namely the Shannon index, Simpson index, Stirlingindex for variety and disparity, and Stirling index for balance and diversity.
All the graphs are plotted with the performance/diversity trade-off term δ on
the x-axis, to index the position along the range of diversity-optimal portfolios
as drawn from the efficient frontier.

95

zFigure 4, titled “Portfolios”, is the principal useful output of this program. It
graphs on the y-axis the percentage contribution to the overall mix drawn
from each defined energy option. This is the composition of thediversity-optimal portfolios drawn from the Pareto efficient frontier shown in
Figure 2. It is indexed on the x-axis it with the performance-diversity trade-off
for each portfolio ( δ ).
This is the graphic that is referred to in the text as the ‘range ofdiversity-optimal portfolios’. The value of δ at which the portfolio value due
to option performance (V{E}) is equal to that due to portfolio diversity (V{P}) is
indicated with a red vertical dashed line. Low values of δ in this range
express high confidence in performance appraisals of individual technologies,
with little concern over deep uncertainties to which diversity is a reasonable
response. Likewise, low values of δ imply that a priority is attached tomaximising this performance, rather than the other benefits of diversity (infostering innovation, mitigating lock-in or accommodating pluralism). Highvalues of δ , on the other hand, reflect a dominant interest in these benefits ofdiversity, with little concern over the resulting compromises on performance.

96

z

Figure 5, titled “Dendrogram”, uses the same algorithms and metrics as theclustering procedure selected in the Excel interface program, and therefore
displays the same picture of the underlying disparity structure. The only
difference is that the vertical ordering of the energy options is reversed.

97

z

Figure 6 is titled “PieCharts” and is newly introduced for this pilot study. This
figure has four pie charts of percentage contribution of individual energy
options at the performance-diversity half point, 55% point (performance :diversity = 55 : 45), 60% point and 75% point. These charts provide
participants with a clearer representation of portfolio compositions along a
typically-favoured interval of the diversity-performance trade-off, of a kindthat is normally held to yield realistic levels of diversity in the energy mix.

98

References
Bana e Costa, 1990

C.A. Bana e Costa, Ed. Readings in Multiple Criteria Decision

Aid. Berlin-Heidelberg, Springer-Verlag.
BERR, 2008

Department for Business Enterprise & Regulatory Reform, UK

EnergySectorIndicators

2008.

http://www.berr.gov.uk/files/file47147.pdf (accessed 12 August,
2008)
Bonner, 1986

J. Bonner, Politics, Economics and Welfare: an Elementary

Introduction to Social Choice. Harvester Press, Brighton.
CEC, 2007

Commission of the European Communities, An Energy Policy

for Europe, Communication from the Commission to theEuropean Council and the European Parliament, SEC(2007)12.
CSTP, 2005

Japan Council for Science and Technology Policy, [OnFollow-up Results and Revision of ‘National Guidelines on the
Evaluation for Government-Funded R&D’], 29 March 2005.
Davies et al., 2003

G. Davies, J. Burgess, M. Eames, S. Mayer, J. Staley, A.
Stirling, S. Williamson, Deliberative Mapping: Appraising

Options for Addressing ‘the Kidney Gap’, Final report to
Wellcome Trust. Wellcome Trust, London, 2003.
DTI, 2003

UK Department of Trade and Industry. Our Energy Future –Creating a Low Carbon Economy, Energy White Paper, 2003
DTI, 2005

UK Department of Trade and Industry, The Role of Fossil FuelCarbon Abatement Technologies (CATs) in a Low Carbon
Energy System – A Report on the Analysis Undertaken to99

Advise the DTI’s CAT Strategy, London, December 2005.
DTLR, 2001

J. Dodgson, M. Spackman, A. Pearman, ‘Multicriteria Analysis:
a Manual’, Department of Transport, Local Government andtheRegions,

HMSO,

2001.

At:

http://www.communities.gov.uk/documents/corporate/pdf/14686
8.pdf.
Farrel et al., 2004

A. Farrell, H. Zerriffi, H. Dowlatabadi, Energy Infrastructureand Security, Annual Review of Environment and Resources,
29, 2004: 421-69.

Grubb et al., 2007

M. Grubb, L. Butler, P. Twomey, ‘Diversity and security in UK
electricity generation: The influence of low-carbon objectives’,

Energy Policy, 34(18), December 2006: 4050-62.
Leonard & Jones, 1989

R. Leonard, G. Jones, Quantifying Diversity in Archaeology,
Cambridge University Press, Cambridge, 1989.

PIU, 2001

PerformanceandInnovation

Unit,

Working

Paper

onGenerating Technologies: Potentials and Cost Reductions to
2020, UK Cabinet Office, London, 2001.
Salo, 1995

A. Salo, Interactive Decision-Aiding for Group Decision
Support, European Journal of Operational Research, V84,
134-149.

Saltelli, 2001

A. Saltelli, Sensitivity Analysis for Importance Assessment.
Ispra,

Italy:EC

Joint

Research

Centre.

http://www.ce.ncsu.edu/risk/pdf/saltelli.pdf (accessed 17 July,
2008)
Stirling, 1994

A. Stirling, ‘Diversity and ignorance in electricity supply

100

investment: addressing the solution rather than problem’,Energy Policy, 22(3), 1994: 195-216.
Stirling, 1997

A. Stirling, ‘Multi-criteria mapping: mitigating the problems ofenvironmental valuation’, in J. Foster (ed.) Valuing Nature.
Routledge, London, 1997.

Stirling, 1998

A. Stirling, On the Economics and Analysis of Diversity. SPRUElectronic Working Paper Number 28. University of Sussex,
October

1998.

At

(June

2008):

http://www.sussex.ac.uk/Units/spru/publications/imprint/sewps/sewp28/sewp28.pdf.
Stirling, 2007

A. Stirling, ‘A general framework for analysing diversity inscience, technology and society’, Journal of the Royal SocietyInterface, 4(15), August 2007: 707-19.
Stirling, 2008aA. Stirling, ‘“Opening up” and “Closing down”: power,
participation, and pluralism in the social appraisal oftechnology’, Science, Technology, & Human Values, 33(2),March 2008: 262-94.

Stirling, 2008b

A. Stirling, ‘Diversity and sustainable energy transitions:multi-criteria analysis of electricity portfolios’, M. Bazilian et al.
(eds.) Portfolio-Based Electricity Planning: Enhancing Energy

Diversity and Security – A Tribute to Shimon Awerbuch,
Elsevier, forthcoming.
Stirling et al., 2006

A. Stirling, T. Lobstein, E. Millstone, ‘Methodology for
obtaining stakeholder assessments of obesity policy options in
the PorGrow project’, Obesity Review, 8(2), December 2006:
17-27.

101

Stirling & Mayer, 2001

A. Stirling, S. Mayer, ‘A novel approach to the appraisal oftechnological risk: a multicriteria mapping study of ageneticallymodifiedcrop’,

Environmental

Planning

Government and Policy, 19(4): 529-555.
Vincke, 1992

P. Vincke, Multicriteria Decision-Aid. John Wiley & Sons.

102

C: