# Sustainability Assessment Tools – State of Research and Gap Analysis

Marco Taisch, Vahid Sadr, Gökan May, Bojan Stahl

► To cite this version:

Marco Taisch, Vahid Sadr, Gökan May, Bojan Stahl. Sustainability Assessment Tools – State of Research and Gap Analysis. 20th Advances in Production Management Systems (APMS), Sep 2013, State College, PA, United States. pp.426-434, 10.1007/978-3-642-41263-9_53. hal-01449750

HAL Id: hal-01449750

https://inria.hal.science/hal-01449750

Submitted on 30 Jan 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Sustainability Assessment Tools – State of Research and
Gap analysis

Marco Taisch¹, Vahid Sadr¹, Gökan May¹, Bojan Stahl¹

Politecnico di Milano, Department of Management, Economics and Industrial Engineering,
Piazza Leonardo da Vinci 32, Milano, 20133, Italy

{marco.taisch, vahid.sadr, gokan.may, bojan.stahl}@polimi.it

**Abstract.** This paper investigates the current tools and frameworks for sustainability development assessments in industrial sectors and aims to find out the gaps between theory and practice toward sustainability assessments. An analysis of existing literature in the area of sustainability assessment tools has been complemented by interviews with experts in the area of sustainability and energy efficiency. Based on the body of knowledge and the practitioner feedback, time required and specificity are key challenges in sustainability development analysis. The paper opens the room for ideas concerning future research initiatives to overcome those drawbacks and challenges.

Keywords: Sustainable manufacturing; energy efficiency; sustainability assessment tools; manufacturing

## 1 Introduction

In recent years, sustainable development became more important for policy makers in industrial sectors. Associations such as The World Business Council [1] or the Global Reporting Initiatives [2] have paved the way towards the implementation of sustainability in business and industry. The development of standards and measures [3] are initiations for reporting on the triple bottom line.

As of today there have been remarkable efforts to investigate and establish methods, frameworks and techniques concerning sustainability assessment. This paper discusses the evolution of manufacturing paradigms and systems towards sustainability. Figure 1 shows evolutionary and remarkable steps in the development of the manufacturing function over time.

The necessity of information analysis in order to make efficient decisions highlights the importance of assessment tools for decision makers. A large amount of sustainability assessment tools exist and each one according to its characteristics provides different information and analysis. Therefore it is able to fulfill a specific purpose, thus choosing the proper assessment tool is a critical decision to be made.
<figure><img src="image_2.png" /><figcaption>**Figure 1:** Evolution of manufacturing systems toward sustainable manufacturing</figcaption></figure>

For instance assessment tools for making short-term decisions may be different from those being used for making long-term decisions.

OECD sustainability studies divided assessment tools in three types: analytical tools and methods, participative tools and methods, and managerial assessment frameworks [4]. Analytical tools try to bring sustainability assessments into communications for instance national income or genuine savings. Participative tools are one of the key tools in integrated assessments and they are based on sharing data, knowledge, views and ideas of different participants such as researchers, policy makers, social organizations, etc. Assessment frameworks are used to investigate different aspects of sustainability and try to find the linkage between them.

Quick assessments and consequently simple policy processes as well as long term and complicated processes follow a four-generic-step procedure according to OECD sustainability development which has published a table that summarizes the theoretical framework used for selection of specific tool. Table 1 shows how each type of tools follow these generic set of steps consisting of four phases, in integrated assessment approaches and consequently helps in decisions regarding selection of proper tool [4].

This paper analyzes the current state of the art concerning existing assessment methods and investigates how comprehensive these tools are taking the triple bottom line [5] into consideration. The paper is organized as follows: the theoretical background on sustainability frameworks for assessments precedes the methodological chapter and the subsequent review. Finally, the identified gaps are presented and discussed.
<table><caption>Table 1: The Role of Tools in Sustainability Assessment Adopted from OECD (2004)</caption><thead><tr><th></th><th>Phase I<br>Problem Analysis</th><th>Phase II<br>Finding Options</th><th>Phase III<br>Analysis</th><th>Phase IV<br>Follow-up</th></tr></thead><tbody><tr><th>Participatory tools</th><td>Problem framing (mobilizing and integrating knowledge and values)</td><td>Supporting scenario building</td><td>Providing the context for and improve robustness of MCA, CBA and CEA</td><td>Evaluating the assessment process</td></tr><tr><th>Scenario tools</th><td>Providing the future perspective to future framing</td><td>Visioning features, finding options and setting options</td><td>Providing references for the application of analytical tools</td><td>-</td></tr><tr><th>Multi-criteria analysis tools (MCA)</th><td>-</td><td>Definition of criteria</td><td>Comparing different alternatives</td><td>-</td></tr><tr><th>Cost-benefit analysis (CBA) and Cost-effective analysis (CEA)</th><td>Providing the analytical basis for problem framing</td><td>Supporting objective settings</td><td>Full analytical characterization of options to enable comparison</td><td>Ex-post assessments</td></tr><tr><th>Accounting tools</th><td></td><td></td><td></td><td></td></tr><tr><th>Model tools</th><td></td><td></td><td></td><td></td></tr></tbody></table>

## 2 Theoretical Background

Warhurst states that sustainability development can be measured through a two-step approach [6]. In the first step the progress of sustainability development will be measured by the help of Sustainability Development Indicators (SDIs) in a number of selective individual fields. In the second step overall progress toward sustainable development will be found by assessing these individual fields considering their relation together. Lancker and Nijkamp postulate that an indicator has no correlation and causality with sustainability improvement measurement unless there is a reference threshold for it [7]. This proposition directly talks about benchmarking the indicators and their assessment. So it highlights the need for developing a set of comprehensive indicators, which enables benchmarking in both domestic and international levels.

The Pressure-State-Response framework has been developed by OECD and is widely used as a reference framework [8]. Pressure indicators explain how the pressure of human activities and the way they use natural resources is influencing environment. While the framework is well suited for assessing environmental aspects, it has limitations in performance assessments of the remaining two pillars.

Lundin describes two different approaches that can be used in order to create a framework and indicators. These two typical approaches are distinguished as:

* Top-Down Approach: experts and researchers develop the framework and then they set the appropriate Sustainability Indicators (SIs) for the framework.

* Bottom-up Approach: This approach is based on corporation and synergy. Participants and stakeholders try to share ideas and develop a framework and its required indicators. Examples of this approach can be Sustainable Seattle, Bellagio Principles and Picabue [9].
The LCSP framework was introduced in order to organize the existing indicators and to develop new indicators. The framework suggests starting from easy and simple performance indicators to measure sustainability developments and move toward complex ones. The LCSP framework mainly focuses on environmental, safety and health aspects of sustainable development [10].

One of the popular and famous frameworks toward sustainability development is founded in 1997 by non-profit organizations named Environmentally Responsible Economies (CERES) and the Tellus Institute in Boston. In early 1990s a framework was pioneered for environmental reporting and after implementing more developments on this framework it came up with the framework of Global Reporting Initiative (GRI). GRI came along with the motive to “do more than environment”, hence the scope of the framework was broadened to cover social, economic and governance issues. GRIs guidance became a sustainability-reporting framework, with reporting guidelines at its heart. GRI uses a hierarchy principle covering all the aspects of sustainability including social, economic and environmental aspects. [11]

## 3 Research Methodology

As this paper is a review on the current literature of sustainability assessment tools, so the methodology applied to carry out this review consists of five consecutive steps. First a web-based search has been carried out to collect all the related documents. After collecting all related materials a short review of each one resulted in a classification scheme according to the level in which they are related to the subject. Following, an analysis of the related material has been carried out to be able to do a gap analysis by integrating and comparing the results coming from the literature review and collected facts from the fourth step in which interviews with experts in sustainability have been carried out. The goal of the gap analysis was to find out the differences between what exists in the literature and what is currently in practice, so it can be used as a body of knowledge for future researches.

<figure><img src="image_3.png" /><figcaption>Figure 2: Methodology</figcaption></figure>

## 4 State of Research

This section will focus on the main goal of this paper that is evaluating sustainability assessment tools that are currently in use. The context of this section comes from the both literature review and interviews.
## 4.1 Indicators

Indicators and indices assemble the first part of sustainability assessment tools. Main characteristics of indicators are being efficient and easy to use. Mainly they are quantitative indicators although also qualitative indicators are applicable. Indicators are able to represent economic, social and environmental aspects of sustainability in defined criteria. Harger and Meyer stated that indicators should be as simple as possible while at the same time they should be able to be specific or comprehensive depending to the objective of usage. Indicators and indices should allow to identification of trends through time horizons [12].

## 4.2 Product-based assessment tools

The difference of these tools with indicators is the fact that in this category, tools intend to evaluate various flows between products and consumptions. These tools are trying to find out the impacts of using resourcing to satisfy the demands on environment. These assessments can be performed through evaluation of production line and their impact on sustainability aspects or through products life cycle assessment from the moment they are intended to produce till their burial. Although life cycle costing assessment includes also economic aspects but these tools mainly are focused on environmental dimensions.

**Life Cycle Assessment:** The most used tool in the category of product-based assessment tools is Life Cycle Assessment (LCA). This tool is among the oldest and well-developed tools for sustainability assessments. It is considered as a comprehensive tool for assessing environmental impacts because it analyzes actual and potential impacts that a product may has on the environment during raw material acquisition, production process, use, and disposal of the product [13].

**Life Cycle Costing:** Life Cycle Cost analysis is an economic method that considers all the costs over the entire life cycle. This method calculates all the possible costs related to a product, activity or a process over its lifetime [14]. The main goal of life cycle cost analysis is to highlight the impact of operation costs during the life time compared to the investment costs.

**Material Flow Analysis:** MFA is defined as systematic accounting of the flows and stocks of materials within a system defined in space and time. It connects the sources, the pathways, and the intermediate and final sinks of a material. Because of the law of conservation of matter, the results of an MFA can be control by a simple material balance comparing all inputs, stocks, and outputs of a process [15].

**Life Cycle Energy Analysis:** Life cycle energy analysis measures the required energy to produce a product or providing a service [16].

## 4.3 Integrated Assessment

The third category includes integrated assessment tools. These tools help decision makers in decisions regarding a policy or project. Project related tools are used for
local scale assessments, whereas the policy related focus on local to global scale assessments [17]. Integrated assessment is usually based on forecasts than actual results. Many of these tools are based on integration of society and environment dimensions. Integrated assessment tools are useful tools for understanding complex problems [18].

**Conceptual modeling:** Conceptual Modeling is a useful qualitative tool that can help to simplify complex situations. With the help of flowcharts, diagrams, and charts it is possible to visualize the problem and find out the flows, their relationships, points of weakness and strength. By applying conceptual modeling we can start the initial part of computer modeling and as a result achieve precise solutions.

**System Dynamics:** Although systems dynamics tools have similarities with conceptual modeling in terms of simplifying complex problems to understand them better there are significant differences between them. In system dynamics computer models of complex situations are built and then they will be examined over time to study the behavior of model over time [19].

**Multi-criteria Analysis:** Multi-criteria Analysis (MCA) is one the most useful and practical tools in helping decision making processes when we have a complex situation which is necessary to choose between alternatives that are competing together. Above all, this tool is very useful in sustainability assessments where we are in need to analyze complex and inter-connected alternatives in environmental, social and economic dimensions. This method can be used in both qualitative and quantitative analysis.

**Risk Analysis and Uncertainty Analysis:** Rotmans defines risk as a "possibility of damage or loss that may happen because of an event or series of events" [20]. Risk analysis tries to find out these possible damages by identifying the risks and their probability of happening and help decision makers to take proper actions in order to diminish them or make appropriate mitigation actions.

**Vulnerability Analysis:** Vulnerability Analysis intends to find out the level of sensitiveness and resistant of the system toward changes and how capable are to cope with these changes. If vulnerability analysis proves that the system is vulnerable, then risk analysis will be executed [21].

**Cost Benefit Analysis:** Cost Benefit Analysis (CBA), has been used for the first time in order to help decision makers to make trade-off between the costs and benefits of the proposed investments by weighting the costs against the expected benefits [22]. In the area of sustainability this tool can be used in order to make the trade-offs for example, between the environmental benefits of an alternative comparing to its social costs [23]. The remarkable point in CBA is that using monetary units for expressing the expected benefits and similar issues can sometimes be a problem [24][25].

**Impact Assessment:** Impact assessment has been increasingly used for helping policy makers and legislators toward sustainability. This method has been increased for improving regulations in terms of effectiveness and efficiency [26].
<figure><img src="image_4.png" /><figcaption>**Figure 3:** Sustainability assessment tools and their focus area</figcaption></figure>

## 5 Gap Analysis

From what is discussed in the literature and interviews, there are number of existing gaps that have the potential of being future research topics toward sustainability development. Reducing the existing gaps will improve sustainability to a new era. The existing gaps are mainly related to sustainability development assessments and their applications.

One of the existing gaps between scholars and practices is that sustainability assessments are significantly time consuming to be implemented. There are varieties of reasons for this issue. The lack of a set of comprehensive indicators for sustainability development assessment might be supporting this gap. Availability of comprehensive indicators can enable assessment tools to consider all pillars of the triple bottom line concept in the measurements and consequently reducing the need
for executing different tools for measuring sustainability considering different aspects and then complementary tools, frameworks or methods to aggregate and compile the results of all measurements so it is possible to announce a unique set of conclusions to support decision makers regarding to sustainability decisions.

The second gap is that existing tools, methods and processes of sustainability development assessments are mostly specific ones which focus on special criteria, sector or sustainability aspect. This can be a challenge in different sectors or criteria because it may require time to find out about the most proper tool that serves the best for the intended purpose of sustainability measurements and evaluations. Although there have been efforts for finding proper solutions to bridge this gap but still this issue has not been eliminated completely. Another reason in this issue is the level of complexity of decisions vary from simple ones to very complex ones depending on how big a project or a company is and also how many variables are involved in assessments so the necessity of using different types of frameworks or tools can be an extra weight for making the assessments more complex.

# 6 Conclusion

Since sustainability has gained lots of attention as a result sustainability development assessments also became an important issue for companies and industrial sectors. There is large number of methods, regulations, frameworks and tools to reach this purpose.

Here in this paper the effort was to build a body of knowledge of the current methods and tools which are being used with the purpose of sustainability assessments in order to provide managers and policy makers in the industrial sectors clear information about the level of sustainability in which they are operating and hence helping them in the process of decision making. By investigating some gaps making comparisons between what is in the literature and what is in practice carrying interviews with experts in the area some research questions for future works are defined as below:

* Developing indicators for a comprehensive Rapid Sustainability Assessment Tool

* Introducing energy efficiency as a key enabler for sustainability assessments

* Integrating energy efficiency with manufacturing processes and sustainability developments

* Developing methods toward continuous energy efficient assessments

# 7 References

[1] World Business Council for Sustainable Development.: Signals of change: Business progress towards sustainable development (1997)

[2] Organisation for Economic Co-operation and Development (OECD).: Report on workshop on sustainability assessment methodologies, Amsterdam, Netherlands (2008)

[3] OECD.: An update of the oecd composite leading indicators. Short-term economic statistics division, statistics directorate (2002)
[4] Global Reporting Initiative (GRI).: The global reporting initiative, an overview. Global reporting initiative, Boston (2002)

[5] TBLI group: http://dl.dropbox.com/u/16037626/TBLI%20Group%20Presentation.pdf

[6] Warhurst, A.: Sustainability indicators and sustainability performance management. Report to the project: Mining, Minerals and Sustainable Development, Warwick, England (2002)

[7] Nijkamp, P., Lancker, E.: A policy scenario analysis of sustainable agricultural development options: a case study for Nepal. Impact assessment and project appraisal. 18(2): 111-124 (2002)

[8] OECD.: Towards sustainable development: Environmental indicators (1998)

[9] Chalmers University of Technology. Indicators for measuring the sustainability of urban water systems: a life cycle approach (2003)

[10] Veleva, V., Ellenbecker, M.: A proposal for measuring business sustainability: addressing shortcomings in existing frameworks. Greener Management International (2000)

[11] Global Reporting Initiative. https://www.globalreporting.org

[12] Meyer, F.M., Harger, J.: Definition of indicators for environmentally sustainable development. Chemosphere. 33(9): 1749-1775 (1996)

[13] Lindfors, L.G.: Nordic guideline on life-cycle assessment. Report # Nord 1995:20. Nordic Council of Ministers, Copenhagen, Denmark. (1995)

[14] Baumann, H., Gluch, P.: The life cycle costing (lcc) approach: a conceptual discussion of its usefulness for environmental decision making. Building and Environment. 39(5): 571-580 (2004)

[15] Brunner, P.H., Rechberger, H.: Practical Handbook of Material Flow Analysis. Boca Raton, FL. Lewis Publishers (2004)

[16] Herendeen, R.A.: Energy analysis and emergency analysis - a comparison. Ecological Modelling. 178: 227-237 (2004)

[17] Karlsson, S., Holmberg, J.: 'On designing socio-ecological indicators' in U. Svedin and B. Hagerhall Aniansson (eds). Society and Environment: A Swedish research perspective. (1992)

[18] Castells, N., Funtowicz, S., Gough, C.: Integrated assessment: an emerging methodology for complex issues. Environmental Modeling and Assessment. 3: 19-29 (1998)

[19] Caulfield, C.W., Maj, S.P.: A case for systems thinking and system dynamics. Proceedings of IEEE International Conference on Systems, Man and Cybernetics. 2793-2798 (2001)

[20] Rotmans, J.: Methods for IA: the challenges and opportunities ahead (1998)

[21] Kasperson, R., Matson, P.A. McCarthy, J.J., Turner, B.: A framework for vulnerability analysis in sustainability science. Proceedings of the national academy of the united states of america (2003)

[22] Johansson, Per-Olov.: Cost-benefit analysis of environmental change. Cambridge University Press (1993)

[23] Udo de Haes H.A., Triebswetter, U., Eder, P., Clift, R., Wrisberg, N.: Analytical tools for an environmental design and management in a systems perspective. Eco-efficiency in industry and science. Kluwer Academic Publishers (2002)

[24] Meadows, D.: Indicators and information systems for sustainable development. The Sustainability Institute. Hartland, Vermont (1998)

[25] Finnveden, G., Moberg, A.: Environmental systems analysis tools – an overview. Journal of Cleaner Production. 13(12): 1165-1173 (2005)

[26] European Commission. Impact assessment guidelines (2005)