1

A Framework for Research Inventory

of Manufacturing Sustainability Assessment

Ju Yeon Lee, Y. Tina Lee

Engineering Laboratory

National Institute of Standards and Technology

Gaithersburg, Maryland 20899, U.S.A

ABSTRACT

There have been many research papers focused on development of guidelines, indicators, metrics, 

methods, tools, and systems – all related to sustainability performance assessment. However, 

manufacturing companies have difficulty in finding papers that are relevant to their sustainability 

goals and conditions. A research inventory is a data repository for storing research papers in a manner 

that makes them easy to retrieve. To be successful, therefore, the inventory must have a system for 

managing, in a systematic manner, the results generated by a large number of researchers around the 

world. In this paper, we propose a framework for a research inventory that focuses on papers that 

report on research related to manufacturing sustainability assessment. This proposed framework 

consists of two parts: an operational definition and a classification scheme. The definition 

distinguishes between those papers that belong in the inventory and those that do not. The 

classification scheme is used to tag/code papers for retrieval. We demonstrate the use of this proposed 

framework using the Sustainable Manufacturing Toolkit of Organization for Economic Co-operation 

and Development.

KEYWORDS: research inventory, manufacturing sustainability, manufacturing sustainability 

assessment, classification scheme

1. INTRODUCTION

The need for manufacturing companies to pursue sustainability goes beyond its societal importance. It 

is becoming an important prerequisite to ensuring their long-term survival in the globally competitive 

market. Consequently, the number of manufacturing companies that demand the ability to assess

sustainability in their products and processes is increasing rapidly. This ability has been summarized 

in terms of sustainability accounting and impact analysis (Feng et al., 2010). Sustainability accounting

means that resource utilization, waste generation, and pollution emission from all activities in 

manufacturing should be evaluated and documented for sustainability performance evaluation. Impact 

analysis includes the impacts on people’s wellbeing, on the environment, and on the economy. Both 

should be evaluated against industry benchmarks using predefined, performance metrics based on 

sound, scientific research results. Many papers have been published describing such research results. 
2

Those results include guidelines, indicators, metrics, methods, tools, and systems –all related to 

sustainability performance assessment. However, it is often difficult to find papers that are relevant to

the sustainability goals and conditions of specific manufacturing companies.

 To overcome this difficulty, we recommend a research inventory. A research inventory is a 

repository of research papers and a system for managing those papers. The management system

performs four major functions: decide which research is relevant, implement a classification to “tag” 

papers, archive papers to a physical repository, and respond to queries for papers. These functions 

make it possible for users to get a better understanding of the research landscape, to determine the 

current state-of-the-art and state-of-the-practice, and to identify gaps and recommend new research. 

 This paper focuses on developing a research inventory for papers that address research 

approaches1relevant to manufacturing sustainability assessment (MSA). It addresses the first two 

management functions described above. It proposes a conceptual definition to decide relevant papers 

and a structural definition to classify those papers. A conceptual definition defines concepts and 

terminologies relevant to MSA. A structural definition defines a taxonomy or classification scheme of 

a research inventory. 

Figure 1. Research flow of main tasks and their relationships in the paper

 Together, these definitions make up our research inventory. The framework is described in the 

remainder of the paper. Section 2 investigates the definitions of sustainability, sustainability 

assessment, and sustainable manufacturing, and reviews the literature on classification schemes. 

 

1Approach is the method used or steps taken in setting about a task, problem, etc., or a means adopted in tackling a problem, job of work, 

and so on (Approach, 2012). In this paper, the term ‘approach’ is interpreted as a method, which is adopted from any type of means (e.g., 

tool, method, and methodology).
3

Section 3 defines manufacturing sustainability and MSA in an operational way. Section 4 proposes the 

classification scheme with its notation for the MSA approaches. The classification scheme is 

demonstrated with an existing MSA approach. Section 5 summarizes contributions and future work. 

The research method for this paper is depicted in Figure 1, which shows the research flow of main 

tasks and their relationships. 

2. BACKGROUND & LITERATURE REVIEW

A variety of research inventories have been developed in accordance with their target research field 

and scope. The University of North California Institute for the Environment (UNC, 2010) developed a 

sustainable research inventory, which provides a comprehensive inventory of sustainability-related 

research initiatives. The European Science Foundation developed a portal called MERIL (Mapping of 

the European Research Infrastructure Landscape) to achieve a comprehensive inventory of research

infrastructure2of major relevance in Europe across all scientific domains, accessible to the public 

through an interactive online portal (ESF, 2013). The International Society for the Psychoanalytic 

Study of Organizations provides an inventory for research that has been done by their society 

members (ISPSO, 2012). The Patient-Centered Outcomes Research Institute also started a project for 

a comprehensive inventory of clinical data and patient-powered research networks (PCORI, 2012).

 Further investigation reveals that limitations exist with implementations of these research 

inventories. First, most research inventories provide a simple list of research papers. They only have 

the index information (e.g., a title and authors) for registration, tracking, and retrieving. Second, the 

target research fields and scope of the inventories are not specified clearly. For example, even though 

the inventory is for sustainability research, the definition of sustainability research is not clearly 

defined. Lastly, it is difficult to find a commonly accepted framework to implement a research 

inventory in a systematic manner. This could have been the fundamental cause for the limitations to 

existing research inventory schemes.

 This section describes the definitions of sustainability, sustainability assessment, and sustainable 

manufacturing based on literature review. These definitions provide a basis to define the framework. 

Classification scheme for sustainability assessment is also discussed.

2.1 Defining Sustainability

The most widely accepted formal definition of sustainable development appeared in the 1987 World 

Commission on Environment Development (WCED) report (later published as a book, “Our Common 

 

2A European Research Infrastructure is a facility or (virtual) platform that provides the scientific community with resources and services to 

conduct top-level research in their respective fields. This research infrastructure can be single-sited or distributed or an e-infrastructure, and 

can be part of a national or international network of facilities, or of interconnected scientific instrument networks (ESF, 2013).
4

Future”) as development that meets the needs of the present without compromising the ability of future 

generations to meet their own needs (Brundtland Commission, 1987; Labuschagne and Brent, 2005; 

Baumgartner, 2011). There have been more than 100 definitions of sustainability since the WCED 

definition (MSA, 2008). The Forestry Commission of Great Britain defined sustainability by adding 

four objectives to the WCED definition: social progress which recognizes the needs of everyone;

effective protection of the environment; prudent use of natural resources; and maintenance of high and 

stable levels of economic growth and employment, as shown in Figure 2 (Forestry Commission of 

Great Britain, 2012). These objectives were generally referred to as three pillars or triple bottom line 

of sustainability (Earth Charter Initiative, 2000; Hardcastle and Waterman-Hoey, 2010). For instance, 

Beloff et al. (2004) defined sustainable development as a complex concept that encompasses the triple 

bottom line: the economic, environmental, and social factors that affect the ability of an organization 

to survive and grow.

Figure 2. Sustainability definition from the Forestry Commission of Great Britain

 According to the US National Research Council (US NRC, 1999), sustainability is the level of 

human consumption and activity, which can continue into the foreseeable future, so that the system

that provides goods and services to the humans persists indefinitely. The National Risk Management 

Research Laboratory of US Environmental Protection Agency suggested that sustainability occurs 

when we maintain or improve the material and social conditions for human health and the 

environment over time without exceeding the ecological capabilities that support them (Sikdar, 2003).

Australia's National Strategy for Ecologically Sustainable Development (1992) ecologically defined

sustainable development as using, conserving, and enhancing the community's resources so that 

ecological processes, on which life depends, are maintained, and the total quality of life, now and in 

the future, can be increased. According to Hediger (2000), sustainability is a normative concept which 

involves trade-offs among social, ecological, and economic objectives, and is required to sustain the 
5

integrity of the overall system. Lozano (2008) proposed the Two Tiered Sustainability Equilibria 

(TTSE) adding time aspect to economic, environmental, and social aspects to explain sustainability. 

Do (2010) defined sustainability as the development concept and approach that realize social justice, 

maintain natural environment, and pursue economic prosperity with 3P (People, Planet, and Profit) 

and 3E (Equity, Environment/Earth, and Economy). Feng et al. (2010) defined sustainability in 

development as an organization's ability to advance its economic state without compromising the 

environment and the social equity that provide the quality of life for all community residents, present 

or future. There are currently many other definitions of sustainability and sustainable development.

Glavic and Lukman (2007) clarified and classified the meaning and applications of 51 terms and their 

definitions related to sustainability for better understanding. To conclude, most definitions agree that 

the sustainability concept is designed to satisfy social, environmental, and economic goals despite 

various different purposes and perspectives.

2.2 Defining Sustainability Assessment 

The pervasive growth of interest over the past few decades in the idea of sustainability or sustainable 

development has brought with it challenges associated with impact assessment. Assessment was 

defined originally in the late 1960s and early 1970s to focus on the environmental impacts. Recently, 

however, the focus has expanded to take into account the three pillars of sustainability (Pope et al., 

2004). As a result, numerous assessment methods, across different disciplines, have been proposed. 

Buselich (2002) proposed that sustainability assessment as assessment of proposed initiatives 

(projects, policies and plans) in terms of sustainability to determine the conditions under which 

approval would be given. Hasna (2008) proposed adding a fourth dimension to the triple bottom line 

and that assessment be viewed as integrated assessment for assessing the social, environmental, 

technological and economic dimensions of projects, policies and programs (PPP). Ness et al. (2007) 

argued that sustainability assessment has increasingly become associated with the family of impact

assessment tools consisting of e.g., Environmental Impact Assessment (EIA), Strategic Environmental 

Assessment (SEA), or EU Sustainability Impact Assessment (SIA). Devuyst and Hens (2001) defined

sustainability assessment as a tool that can help decision-makers and policy-makers decide which

actions they should or should not take in an attempt to make society more sustainable. According to 

Verheem (2002), the aim of sustainability is to ensure that plans and activities make optimal 

contributions to sustainable development. Understanding the general concept of sustainability 

assessment helps us to define MSA. Although most definitions of sustainability assessment have been 

defined from a business perspective, there is a common idea that sustainability assessment should 

support a decision-making associated with social, environmental, and economic performance.

 There have been several research efforts on classifying the existing approaches for assessing 
6

sustainability; however, their focuses are not on the manufacturing. Ness et al. (2007) provided a 

categorization of sustainability assessment tools within a broader objective of lifting the 

understanding of sustainability assessment from a narrow, environmental focus to a wider

interpretation of sustainability. Kinderytė (2008) presented comparison criteria of methodologies for 

corporate sustainability assessment. Baumann and Cowell (1999) suggested an evaluative framework 

for comparing conceptual and analytical approaches used in environmental management based on 

their methodological features. These research efforts defined their own schemes for classification with

focus on environmental aspects of sustainability, not on MSA. 

 There also exist research efforts related to classification of sustainability assessment for the 

purpose of investigating existing research. They defined simple criteria to support their literature 

review. Pope et al. (2004) reviewed and categorized sustainability assessment approaches, and Hasna 

(2008) and Singh et al. (2012) reviewed sustainability assessment methods. Gunasekaran and 

Spalanzani (2012) reviewed and classified the literature available on Sustainable Business 

Development (SBD) in manufacturing and services. Finnveden and Moberg (2005) reviewed 

environmental systems analysis tools. Some criteria, which can be generally applied to sustainability 

assessment, are elicited from existing research efforts. The classification scheme proposed in this 

paper refers to those criteria to classify papers associated with MSA approaches.

2.3 Defining Sustainable Manufacturing 

Sustainability has been interpreted in many ways based on requirements from different application

domains and different objectives within those domains. One such domain is manufacturing, which can 

be considered as one of the most important domains for achieving the goal of sustainable development. 

Implementing sustainability in manufacturing will surely be one of the most positive contributions to 

sustainability in general (Rosen and Kishawy, 2012; Garetti and Taisch, 2012). For that reason, 

sustainable manufacturing has become more prominent in recent years. As with the preceding topics, 

there are a number of definitions of sustainability manufacturing. The US Department of Commerce 

defined that sustainable manufacturing is the creation of manufactured products that use processes 

that minimize negative environmental impacts, conserve energy and natural resources, are safe for 

employees, communities, and consumers and are economically sound (US DOC, 2009). The National 

Institute of Standards and Technology technically defined sustainable manufacturing as a systems 

approach for the creation and distribution (supply chain) of innovative products and services that: 

minimizes resources (inputs such as materials, energy, water, and land); eliminates toxic substances; 

and produces zero waste that in effect reduces greenhouse gases, e.g., carbon intensity, across the 

entire life cycle of products and services (Rachuri et al., 2010). According to the National Council for 

Advanced Manufacturing (NACFAM, 2009), sustainable manufacturing includes the manufacturing 
7

of “sustainable” products and the sustainable manufacturing of all products. The former includes 

manufacturing of renewable energy, energy efficiency, green building, and other green and social 

equity-related products, and the latter emphasizes the sustainable manufacturing of all products taking 

into account the full sustainability and total life cycle issues related to the products manufactured. The 

Organisation for Economic Co-operation and Development stated that sustainable manufacturing is 

all about minimizing the diverse business risks inherent in any manufacturing operation while 

maximizing the new opportunities that arise from improving your processes and products (OECD, 

2011). And the general principle of sustainable manufacturing is to reduce the intensity of materials 

use, energy consumption, emissions, and the creation of unwanted by-products while maintaining, or 

improving, the value of products to society and to organizations (OECD, 2009). The Lowell Center 

for Sustainable Production defined that sustainable production is the creation of goods and services 

using processes and system that are: non-polluting, conserving of energy and natural resources, 

economically viable, safe and healthful for workers, communities, and consumers, and socially and 

creatively rewarding for all working people (LCSP, 1998). According to the Intelligent Manufacturing 

Systems (IMS, 2011), sustainable manufacturing aims at developing innovative methods, practices,

and technologies in the manufacturing field for addressing world-wide shortages of resources, for 

mitigating excess environmental load and for enabling an environmentally benign life-cycle of 

products. The Institute of Manufacturing at University of Cambridge stated that “sustainable 

manufacturing is ... developing technologies to transform materials without emission of greenhouse 

gases, use of non-renewable or toxic materials, or generation of waste” (Allwood, 2005). The 

Sustainable Manufacturing Consulting defined sustainable manufacturing as a business practice of the 

industrial sector, which expands all the company’s processes and decisions into the social and natural 

environments it operates in and affects, with the explicit objective of reducing or eliminating any 

negative impact, while pursuing the desired level of technological and economic performance (LeahuAluas, 2009-2010).

 Despite various concepts of sustainable manufacturing, it is difficult to get practical understanding 

of the manufacturing sustainability and MSA concepts from them. That is because most sustainable 

manufacturing definitions are at a conceptual level and there exists significant difference between 

sustainable manufacturing and manufacturing sustainability. Generally speaking, sustainable 

manufacturing is a status that manufacturing is “sustainable”; in other words, sustainable 

manufacturing means that sustainability is already accomplished in manufacturing. The concept of 

sustainability in manufacturing is referred to as manufacturing sustainability in this paper. To 

determine whether or not the manufacturing is sustainable, it is required to assess its manufacturing 

sustainability. Section 3 addresses manufacturing sustainability and its assessment in more detail. 
8

3. THE OPERATIONAL DEFINITIONS OF MANUFACTURING SUSTAINABILITY

AND MANUFACTURING SUSTAINABILITY ASSESSMENT

It is clear from the preceding discussions, that there are no commonly agreed upon operational 

definitions of manufacturing sustainability and MSA. Since they are required for our research

inventory framework, we now provide such definitions to ground the premise for our work.

3.1 Requirements for Operational Definition

An operational definition is a statement that describes how a particular variable is to be measured or 

how an object or condition is to be recognized (Prentice Hall, 2001). It defines something (e.g., a 

variable, term, or object) in terms of the specific process or set of validation tests used to determine its 

presence and quantity. That is, one defines something in terms of the operations that count as 

measuring it (Shoemaker et al., 2004; Wikipedia-Operational Definition, 2012). The requirements for 

an operational definition are that

 It has criteria or variables

 It has operations

 It must be measurable or observable

 It must be practical

 It is important to set criteria or variables for a well-defined concept in an operational way. An 

operational definition translates the verbal concept into corresponding criteria or variables which can 

be measured. The concept can be referred to as a variable since it can respond to differences in the 

real world by taking on varying values, as specified in the operational definition (Watt and van den 

Berg, 1995). An operational definition contains one or more operations that measure the values of the 

criteria (Shoemaker et al., 2004; Wikipedia-Operational Definition, 2012). The most fundamental idea 

is that an operational definition must be measurable or observable. An operational definition aims to 

convert a concept into a measurement (Watt and van den Berg, 1995). More importantly, anyone can 

use the operations to repeat that measurement. It is this, in this sense, that an operational definition is 

practical. In summary then, an operational definition removes the ambiguity from a concept by 

actually measuring a variable related to that concept. 

3.2 Manufacturing Sustainability and Manufacturing Sustainability Assessment

In this section, we provide operational definitions of manufacturing sustainability and MSA. 

3.2.1 Operational Definition for Manufacturing Sustainability

Most existing definitions of sustainable manufacturing consider product, process, and system as main 

components (Jawahir, 2008; Jayal et al., 2010; ASME, 2011). We include these same concepts, called 
9

the MS’s 3P, in our definition: product design, process plan, and production system. We define the 

scope of these components as follows. Product design is the detailed specification of a manufactured 

item’s parts and their relationships to the whole (Product Design, 2012). Process plan is a collection of

relational models of processes, equipment, tooling, and the sequencing of operations required to 

produce a product based on the product design. Production system consists of production resources

that execute the processes contained in the process plan (Koho, 2010). Using this MS’s 3P, our 

operational definition of manufacturing sustainability is:

Manufacturing sustainability is a measure of manufacturing performance metrics of product design, 

process plan, and production system with respect to environment, economy, and society, when 

executing a process plan for a product design in a given production system. 

 The operational aspect of this definition has to do with the measurement of the impacts on 

environment, economy, and society. Figure 3 depicts the interrelationships among manufacturing 

sustainability, the MS’s 3P, and these impacts in order to understand manufacturing sustainability. In 

the following section, we address the assessment of these impacts.

Figure 3. Interrelationships among manufacturing sustainability, the MS’s 3P, and impacts
10

3.2.2. Operational Definition for Manufacturing Sustainability Assessment

The definition of MSA is proposed based on the MS definition:

Manufacturing sustainability assessment is to determine a value of the manufacturing sustainability

metric, which is a balanced performance of product design, process plan, and production system with 

respect to environmental, economic, and social aspects of sustainability. 

Figure 4. Definition of MSA

 Figure 4 depicts the relationships among the MS’s 3P (i.e., PD, PP, and PS) and three pillars of 

sustainability (i.e., En, Ec, and So) in an operational manner. Table 1 shows all of the potential impact 

assessments that can be performed. Each row depicts one of the MS’s 3P and each column depicts one 

of the pillars. Additionally, the table shows aggregated rows (PDMS, PPMS, PSMS), aggregated columns 

(EnTotal, EcTotal, SoTotal) and a variable called MSPoint. MSPoint represents an aggregated balance among 

all of the impacts. This balance is achieved through a series of weights WS. These weights can vary 
11

from one company to another.

Table 1. Various sustainability impacts explained with the MS’s 3P and three pillars of sustainability

En Ec So Total

PD PDEn PDEc PDSo PDMS

PP PPEn PPEc PPSo PPMS

PS PSEn PSEc PSSo PSMS

Total EnTotal EcTotal SoTotal MSPoint

 Computing the variables in the total column is usually quite difficult, since the metrics associated 

PD, PP, and PS are usually quite different. Thus, we can represent these computations formally as 

functions, Fi, as shown below. 

MSPoint = F(WPD*PDMS, WPP*PPMS, WPS*PSMS) [E1]

PDMS = F1(WPDEn*PDEn, WPDEc*PDEc, WPDSo*PDSo) [E2]

PPMS = F2(WPPEn*PPEn, WPPEc*PPEc, WPPSo*PPSo) [E3]

PSMS = F3(WPSEn*PSEn, WPSEc*PSEc, WPSSo*PSSo) [E4]

 If the metrics can be converted and normalized to a single metric, such as cost, then the functions 

Fi, can be written as a weighted linear combination of the individual row values as shown below. 

MSPoint = WPD*PDMS + WPP*PPMS + WPS*PSMS [E5]

PDMS = WPDEn*PDEn + WPDEc*PDEc + WPDSo*PDSo [E6]

PPMS = WPPEn*PPEn + WPPEc*PPEc + WPPSo*PPSo [E7]

PSMS = WPSEn*PSEn + WPSEc*PSEc + WPSSo*PSSo [E8]

 Calculating the column values (EnTotal, EcTotal, SoTotal) will be conducted in the same way, if the 

values in each column have the same metric. They too can be calculated using a weighted linear 

combination of the individual column values as shown below. 

 

EnTotal = WEnPD*PDEn + WEnPP*PPEn + WEnPS*PSEn [E9]

EcTotal = WEcPD*PDEc + WEcPP*PPEc + WEcPS*PSEc [E10]

SoTotal = WSoPD*PDSo + WSoPP*PPSo + WSoPS*PSSo [E11]

 Note, as before, there is no simple formula for computing MSPoint using PDMS, PPMS, and PSMS.
12

4. CLASSIFICATION SCHEME FOR MANUFACTURING SUSTAINABILITY 

ASSESSMENT APPROACH

A classification scheme is required to establish the fundamental structure for research inventory of 

papers associated with MSA approaches. This paper presents a set of requirements for the 

classification scheme. The scheme to classify the MSA approaches is developed based on the

requirements and definition of MSA in section 3. This section presents the requirements, the scheme,

and the notation of the proposed classification. An implementation of the classification scheme is 

demonstrated through using an existing approach of MSA. 

4.1 Requirements for Classification Scheme

5W1H, which stands for Why, Who, When, Where, What, and How, has been broadly employed in 

journalism to gather information (Chung et al., 2009). 5W1H can be used also to classify and navigate 

information with intuitive classification axes and retrieval keys (Ikeda et al., 1998). Hong and 

Davison (2009) used 5W1H as a classification method for their experiment on question answering in 

discussion boards. Huang et al. (2011) proposed a scheme based on the context of 5W1H in 

ubiquitous computing to establish a database for environmental monitoring factors. Ikeda et al. (1998)

proposed a method by which 5WlH information is used to classify and navigate Japanese language 

texts. 

We will use 5W1H to establish a set of requirements for classifying MSA approaches. How we 

use them is shown below.

 What: What aspects or facets of manufacturing sustainability does it assess?

 When: When is it needed to assess manufacturing sustainability?

 Where: Where is the scope targeted for assessing manufacturing sustainability?

 Who: Who has interest in assessing manufacturing sustainability?

 Why: Why does it assess manufacturing sustainability?

 How: How does it assess manufacturing sustainability?

What aspects or facets of manufacturing sustainability does it assess? This question is related to

the contents, which are assessed by the approach in terms of manufacturing sustainability (Singh et al., 

2012). When is it needed to assess manufacturing sustainability? It asks about a particular situation or 

purpose, which needs MSA. Where is the scope targeted for assessing manufacturing sustainability?

It is literally about the scope of assessment (Ness et al., 2007). Defining a specific scope is important 

to classify the approaches for MSA. Who has interest in assessing manufacturing sustainability? This 

question can be interpreted as an interest unit of a stakeholder in manufacturing from an 

organizational perspective. Why does it assess manufacturing sustainability? It is about the purpose of 
13

MSA. How does it assess manufacturing sustainability? This question asks about the method of MSA

(Hasna, 2008; Singh et al., 2012).

4.2 Proposed Classification Scheme

Figure 5 shows the 5-level, proposed classification scheme. Level 0 is the top level. It uses our 

operational definition of MSA to determine whether or not a paper is eligible to be included in the 

inventory. Level 1, 2, and 3 are criteria to define a generic classification scheme, and level 4 is

categorical variables specialized from the generic criteria for the MSA approaches. Note, that this 

classification is meant to provide a list of potential key-words, in a prescribed order, that can be used 

to classify papers for retrieval.

Figure 5. 5-level classification scheme for the MSA approaches

Level 1, Level 2, and Level 3

Level 1 comprises three criteria: content, method, and result. Content is formed from the ‘what’, 

‘where’, and ‘who’ questions. The answers to these questions fall into two categories: scope and 

object. Scope is the boundary covered or targeted by the paper and has two sub criteria: Product Life 

Cycle (PLC) and Organizational Unit (OU). PLC identifies the product-life-cycle stage and OU
14

identifies the organizational unit level the paper addresses. Object identifies the sustainability areas 

assessed by the paper; it has two sub criteria: Three Pillars of Sustainability (TPS) and Manufacturing 

Sustainability Components (MSC). TPS is about three pillars of sustainability that are essential to 

assess manufacturing sustainability according to the proposed MSA definition. MSC refers to the 

MS’s 3P, which is proposed as components of manufacturing sustainability in the proposed definition.

Method is about technical features of the MSA approach and is based on answers to the ‘how’

question. Method has two sub criteria: technology and Assessment Process (AP). Technology has two

sub-criteria: Level Of Technology (LOT) and Use Of Technology (UOT). LOT identifies the maturity 

level of the paper’s particular approach in terms of technology and UOT provides information on the 

type of technology used for the assessment. AP is related to the business process used to assess 

manufacturing sustainability. In order to operationally understand the MSA approach, it is required to 

know which step in the business process the approach can cover. 

Result is formed from answers to the ‘why’ and ‘when’ questions; it has two sub criteria: Purpose 

Of Assessment (POA) and Presentation Of Assessment result (PAR). POA and PAR identify the 

purpose and the desired result type of the MAS approach respectively.

Level 4

All of the subcategories described above combine to form level 3 in the classification. These 

subcategories can be treated as categorical variables, which are referred to as enumerations at level 4. 

Each categorical variable contains a set of possible values that might be defined based on consensus 

or common understanding. In this paper, values of the categorical variables are assigned based on the 

proposed MSA definition and a comprehensive study of MSA.

The categorical variables of PLC correspond to the various stages of the life cycle. They are PreManufacturing (PM), Manufacturing (M), Use (U), or Post-Use (PU) (Jaafar et al., 2007). PM usually 

includes raw material extraction and processing; PU is mainly related to disposal. The categorical 

variables of OU include Supply chain (S), Company (C), Factory (F), ProducT (PT), ProcesS (PS), 

Work cell (W), or Machine tool (M) (Feng et al., 2010; Reich-Weiser et al., 2008). The categorical 

variables of TPS and MSC in the object criterion refer to the proposed MSA definition. TPS is 

Environmental (En), Economic (Ec), or Social (So), and MSC is Product Design (PD), Process Plan 

(PP), or Production System (PS). 

The categorical variables of LOT refer to the Technology Readiness Levels (TRL), (TRL1,...,

TRL9) (US DoD, 2011). TRL is a general measure to assess the maturity of evolving technologies

even though TRL levels are not specialized for the technology for sustainability assessment

(Wikipedia-TRL, 2012). The variables of UOT are related to Scaling (S), Normalization (N), 

Weighting (W), or Aggregation (A) (Singh et al., 2012). The AP variables refer to the OECD 
15

Sustainable Manufacturing Toolkit, which defines 3 main steps: Prepare (P), Measure (M), and 

Improve (I) (OECD, 2011). 

The categorical variables of POA take in values associated with Decision-Making (DM), 

Reporting/Communication (RC), or Identifying/Ranking (IR). Eventually, the results of any MSA are 

used to support some kind of a decision-making process. As a result, many papers are expected to be 

about decision making. Frequently, the results of an assessment or decision must be communicated

and forwarded to another entity. The Global Reporting Initiative (GRI, 2010-2011) provides a set of 

protocols for this purpose. Sustainability assessment can also be used for identifying or ranking the 

level of sustainability against a predetermined set of benchmarks (Kinderytė, 2008; Baumann and 

Cowell, 1999). The categorical variables of PAR are Composite Index (CI), IMpact (IM), or INdicator 

(IN) (Kinderytė, 2008). The sustainability indicator values are directly calculated by sustainability 

metrics while the impact values are determined based on the sustainability indicator values. The 

composite index is an aggregated total score of manufacturing sustainability based on sustainability 

indicator or impact values. 

4.3 Notation for Classification Scheme 

The proposed classification scheme will provide the fundamental structure of our research inventory. 

That is, it will provide the basis for storing and retrieving papers from the inventory. This section 

proposes a math-based notation to express the classification information for this purpose. Assume that 

C is a nonempty finite set of criteria in the classification scheme. Ci and Cj are sets of i and j

respectively, which are criteria in the classification scheme. The relationships among all sets can then 

be represented as equations from N1 to N2. 

The top level criteria, CMSA, can be represented as a union set of 3 union sets that exist at level 1. 

Each set of content, method, and result is expressed by a union set of union sets or sets of their 

elements. For example, Ccontent is a union set of CScope and CObject where CScope is a union set of CPLC

and COU, and CObject is a union set of CTPS and CMSC. Each set of PLC, OU, TPS, MSC, LOT, UOT, 

AP, POA, and PAR can be represented by the elements of their categorical variables.

CMSA = CContent ∪ CMethod ∪ CResult = {c|c ∈ CContent ⋁ CMethod ⋁ CResult} [N1]

where, Ci ∩ Cj = ∅ {i ≠ j ∧ Ci, Cj ∈ {CContent , CMethod , CResult}}

CContent = CScope ∪ CObject = {c|c ∈ CScope ⋁ CObject} [N1.1]

where, Ci ∩ Cj = ∅ {i ≠ j ∧ Ci, Cj ∈ {CScope, CObject}}

CSope = CPLC ∪ COU = {c|c ∈ CPLC ⋁ COU} [N1.1.1]

where, Ci ∩ Cj = ∅ {i ≠ j ∧ Ci, Cj ∈ {CPLC, COU}}
16

CObject = CTPS ∪ CMSC = {c|c ∈ CTPS ⋁ CMSC} [N1.1.2]

where, Ci ∩ Cj = ∅ {i ≠ j ∧ Ci, Cj ∈ {CTPS, CMSC}}

CMethod = CTechnology ∪ CAP = {c|c ∈ CTechnology ⋁ CAP} [N1.2]

where, Ci ∩ Cj = ∅ {i ≠ j ∧ Ci, Cj ∈ {CTechnology, CAP}}

CTechnology = CLOT ∪ CUOT = {c|c ∈ CLOT ⋁ CUOT} [N1.2.1]

where, Ci ∩ Cj = ∅ {i ≠ j ∧ Ci, Cj ∈ {CLOT, CUOT}}

CResult = CPOA ∪ CPAR = {c|c ∈ CPOA ⋁ CPAR } [N1.3]

where, Ci ∩ Cj = ∅ {i ≠ j ∧ Ci, Cj ∈ {CPOA, CPAR}}

CMSA = {{{CPLC⋁COU}⋁{CTPS⋁CMSC}} ∨ {{CLOT⋁CUOT}⋁CAP} ∨ {CPOA⋁CPAR}} [N2]

where,

CPLC = {PM, M, U, PU}

COU = {S, C, F, PT, PS, W, M}

CTPS = {En, Ec, So}

CMSC = {PD, PP, PS}

CLOT = {TRL1, TRL2, TRL3, TRL4, TRL5, TRL6, TRL7, TRL8, TRL9}

CUOT = {S,N, W,A}

CAP = {P, M, I}

CPOA = {DM, RC,IR}

CPAR = {CI,IM, IN}

4.4 Demonstration of the Proposed Classification Scheme

In this section, we demonstrate how to use the proposed classification scheme and its notation on a 

paper entitled, “The OECD Sustainable Manufacturing Toolkit.” This paper provides a set of 

internationally applicable, common, and comparable indicators to measure the environmental 

performance of manufacturing facilities of any size and in any sector or country (OECD, 2011). This 

paper was analyzed using the feature analysis method and its meta-information was expressed using 

our classification scheme. 

 The feature analysis method is the most prominent and popular comparison/evaluation approach 

that has been used. In its screening mode, it can be performed by a single person who is responsible 

for both generating the evaluation criteria and assessing the methodologies. The evaluation is solely 

based on his/her understanding of the methodologies according to their documentation (Dam, 2003). 
17

Figure 6 shows the results of the features analysis method applied to the OECD paper. The values 

in the various cells are explained as follows (OECD, 2011):

 PLC. The OECD toolkit covers the manufacturing (M) and use (U) stages of product life cycle. 

The toolkit mainly focuses on the manufacturing stage, but it also considers energy consumption 

and greenhouse gas emissions from the use of the manufactured products. Therefore, CPLC of the 

OECD toolkit is represented as {M, U}.

 OU. The indicators in the toolkit have been developed for the production activities of a single 

facility. That is, the OECD toolkit covers from work cell (W), through process (PS) and product 

(PT), to factory (F) levels. The toolkit additionally explains that the toolkit’s coverage can be 

extended to include the boundary to company (C) and supply chains (S). However, OECD

indicators do not cover detailed sustainability data at the machine tool (M) level. Thus, COU is 

represented as {S, C, F, PT, PS, W}.

 TPS. Since the toolkit considers only the environmental aspect (En) of sustainability, CTPS of the 

OECD toolkit is represented as {En}.

 MSC. The toolkit considers production system (PS) as well as product (PD) and production 

process (PP). Some indicators on operations, such as water or energy intensity, are calculated 

based on the data of production process and overhead at the facility level. Therefore, CMSC is

represented as {PD, PP, PS}.

 LOT. The technical level of the OECD toolkit is at TRL8 that is determined based on the current 

status: it was internationally published by OECD; it supports a guide document and web portal;

but it still cannot fully support its desired goal and certain functions. CLOT of the OECD toolkit is 

represented as {TRL8}. 

 UOT. The toolkit explains how to measure (S) and normalize (N) its indicators, and what data is 

needed for those processes. Thus, CUOT is represented as {S, N}.

 AP. The OECD toolkit presents 7 steps, which are categorized into 3 main steps: prepare (P), 

measure (M), and improve (I). The guide provides the detailed explanation about each step of the 

toolkit. Thus, CAP of the OECD toolkit is represented as {P, M, I}. 

 POA. The indicators addressed in the toolkit mainly assist decision-making (DM) and internal 

management including reporting/communication (RC). It is possible to compare or rank (IR) the 

targets through a normalization factor. CPOA is represented as {DM, RC, IR}.

 PAR. The OECD toolkit is mainly composed of its own indicators. The results are provided with 

each value of indicators (IN) separately. The guide explains the way to analyze cost and benefits 

in terms of improvement, but not about a composite index (CI) or values of impacts (IM). Thus, 

CPAR is represented as {IN}. 
18

Figure 6. Implementation of OECD Sustainable Manufacturing Toolkit in research inventory

In summary on this application, it was possible to express important meta-information of the 

OECD Sustainable Manufacturing Toolkit with the scheme proposed for classifying the MSA 

approaches. The classification result could be easily implemented into the research inventory using 

the notation presented in section 4.3.

5. CONCLUSION

This paper has presented a framework for a research inventory containing papers related to MSA. The 

proposed framework primarily consists of two parts: an operational definition of MSA and a scheme 

to classify the related papers. The operational definitions of manufacturing sustainability and MSA

were proposed in consideration of the requirements defined for an operational definition. This paper 

has suggested the classification scheme based on its requirements stated by 5W1H. In addition, the 

notation was defined to represent the classification information in a systematic and computational way 

for retrieving, understanding, and using papers from the inventory. Lastly, the classification scheme 

was demonstrated by applying it to the existing approach, the OECD Sustainable Manufacturing 

Toolkit.

The framework proposed in this paper provides conceptual and structural foundations of the 

research inventory of papers that describe various MSA approaches. Such a research inventory would 

1) make research papers accessible and available, 2) provide a general understanding of research

papers, 3) investigate the state of the art of research papers, and 4) identify gaps and recommend a 

research roadmap. Eventually, this inventory enables manufacturing industries to improve the 
19

sustainability of their manufacturing processes by leveraging appropriate MSA approaches provided 

by the inventory.

 This paper includes information that can adequately represent the approaches of MSA in the 

classification scheme. In that sense, criteria in the scheme are focused on specific features of the MSA 

approaches. The classification scheme can be improved by including more general attributes on the 

approaches (e.g., data availability or flexibility) in the future.

ACKNOWLEDGMENT

The authors like to thank Swee Leong and Prof. Chin-Sheng Chen. They provided valuable inputs and 

recommendations on this paper. Special thanks go to Dr. Albert Jones for his support and technical 

lead. 

6. REFERENCES

American Society of Mechanical Engineers (ASME), 2011. ASME Sustainable Products and 

Processes Strategic Plan [online]. Available from: http://files.asme.org/Committees/K&C/ 

TCOB/CRTD/30032.pdf [Accessed 10 November 2012].

Allwood, J., 2005. What is Sustainable Manufacturing?. Sustainable Manufacturing Seminar Series, 

Institute of Manufacturing, University of Cambridge [online]. Available from: 

http://www.ifm.eng.cam.ac.uk/ [Accessed 26 August 2013].

Approach [online]. Available from: http://dictionary.reference.com/browse/approach [Accessed 29 

November 2012].

Baumann, H., Cowell, S.J., 1999. An evaluative framework for conceptual and analytical approaches 

used in environmental management. Greener Management International: The Journal of Corporate

Environmental Strategy and Practice, 26, 109-122.

Baumgartner, R.J., 2011. Critical perspectives of sustainable development research and practice. 

Journal of Cleaner Production, 19(8), 783-786.

Beloff, B., Tanzil, D., Lines, M., 2004. Sustainable development performance assessment. 

Environmental Progress, 23(4), 271-276.

Brundtland Commission, 1987. Our Common Future-From One Earth to One World. Oxford 

University Press, Oxford, England.

Buselich, K., 2002. An Outline of Current Thinking on Sustainability Assessment. Background paper 

prepared for the Western Australian State Sustainability Strategy. Institute for Sustainability and 

Technology Policy, Murdoch University, Western Australia.

Chung, S., Won, D., Baeg, S.H., Park, S., 2009. Service-oriented reverse reengineering: 5W1H 

model-driven re-documentation and candidate services identification. Proceedings of IEEE 

International Conference on Service-Oriented Computing and Applications (SOCA), 1-6, Taipei, 
20

Taiwan.

Dam K.H., 2003. Evaluating and Comparing Agent-Oriented Software Engineering Methodologies. 

Master Thesis of Applied Science in Information Technology. School of Computer Science and 

Information Technology of RMIT University, Australia.

Devuyst, D., Hens, N., 2001. How Green is the City? Sustainability assessment and the management 

of urban environments. Columbia University Press, New York.

Do, N.C., 2010. The Definition of Sustainable Manufacturing. KEDM PLM [online]. Available from: 

http://fvortal.cimerr.net/plm/122?category=26 [Accessed 28 August 2013].

Earth Charter Initiative, 2000. The Earth Charter [online]. Available from: 

http://www.earthcharterinaction.org/content/pages/Read-the-Charter.html [Accessed 3 July 2012].

Ecologically Sustainable Development Steering Committee, 1992. National Strategy for Ecologically 

Sustainable Development [online]. Available from: http://www.environment.gov.au/about/esd/ 

publications/strategy/index.html [Accessed 12 October 2012].

European Science Foundation (ESF), 2013. The MERIL portal [online]. Available from: 

http://portal.meril.eu/converis-esf/publicweb/startpage [Accessed 27 January 2013].

Feng, S.C., Joung, C.B., Li, G., 2010. Development Overview of Sustainable Manufacturing Metrics. 

Proceedings of the 17th CIRP International Conference on Life Cycle Engineering.

Finnveden, G., Moberg, A., 2005. Environmental systems analysis tools – an overview. Journal of 

Cleaner Production, 13(12), 1165-1173.

Forestry Commission of Great Britain, Sustainability [online]. Available from: 

http://www.forestry.gov.uk/forestry/edik-59fmzf [Accessed 5 September 2012].

Garetti, M., Taisch, M., 2012. Sustainable manufacturing: Trends and research challenges. Production

Planning & Control, 23 (2-3), 83-104.

Glavic, P., Lukman, R., 2007. Review of sustainability terms and their definitions. Journal of Cleaner 

Production, 15(18),1875-1885.

Global Reporting Initiative (GRI), 2010-2011. Sustainability Reporting Guidelines Version 3.1 

[online]. Available from: http://www.globalreporting.org [Accessed 28 August 2013].

Gunasekaran, A., Spalanzani, A., 2012. Sustainability of manufacturing and services: Investigations 

for research and applications. International Journal of Production Economics, 140(1), 35-47.

Hardcastle, A., Waterman-Hoey, S., 2010. Advanced Materials Manufacturing, Sustainability and 

Workforce Development. Washington State University Extension Energy Program [online]. 

Available from: http://www.energy.wsu.edu/ [Accessed 28 October 2012].

Hasna, A.M., 2008. A review of sustainability assessment methods in engineering. The International 

Journal of Environmental, Cultural, Economic & Social Sustainability, 5(1), 161-176.

Hediger, W., 2000. Sustainable development and social welfare. Ecological Economics, 32(3), 481-
21

492.

Hong, L., Davison, B.D., 2009. A Classification-based Approach to Question Answering in 

Discussion Boards. Proceedings of ACM SIGIR conference on Research and development in 

information retrieval, 171-178. New York, USA.

Huang, Y.P., Chang, H.C., Lin, C.C., 2011. Systematic design of environmental monitoring interface 

by Bayesian classification. Proceedings of 2011 International Conference on System Science and 

Engineering, 43-48, Macau, China.

Ikeda, T., Okumura, A., Muraki, K., 1998. Information Classification and Navigation Based on 5W1H 

of the Target Information. Proceedings of the 17th International Conference on Computational 

Linguistics, 1, 571-577, Montreal, Quebec, Canada.

Intelligent Manufacturing Systems (IMS), 2011. Maintenance for Sustainable Manufacturing (M4SM), 

White paper of the IMS M4SM MTP initiative [online]. Available from: http://www.ims.org/

[Accessed 10 October 2012].

International Society for the Psychoanalytic Study of Organizations (ISPSO), 2012. ISPSO Research 

Inventory [online]. Available from: http://www.ispso.org/the-field:research-projects [Accessed 6 

January 2013].

Jaafar, I.H., Venkatachalam, A., Joshi, K., Ungureanu, A.C., De Silva, N., Dillon Jr, O.W., Rouch,

K.E., Jawahir, I.S., 2007. Product Design for Sustainability: A New Assessment Methodology and 

Case Studies. In Environmentally Conscious Mechanical Design (Ed M. Kutz), John Wiley & 

Sons, Inc., 25-65, NJ, USA.

Jayal, A.D., Badurdeen, F., Dillon, O.W., Jawahir, I.S., 2010. Sustainable manufacturing: Modeling

and optimization challenges at the product, process and system levels. CIRP Journal of 

Manufacturing Science and Technology, 2(3), 144-152.

Jawahir, I.S., 2008. Beyond the 3R’s: 6R Concepts for Next Generation Manufacturing: Recent 

Trends and Case Studies. Symposium on Sustainability and Product Development, IIT, Chicago,

USA [online]. Available from: http://mmae.iit.edu/symposium/downloads/pres/Jawahir.pdf

[Accessed 17 February 2012].

Kinderytė, L., 2008. Analysis and Comparison of Methodologies for Corporate Sustainability 

Assessment. Environmental Research, Engineering and Management, 4(46), 66-75.

Koho, M., 2010. Production system assessment and improvement: a tool for make-to-order and 

assemble to-order companies. Doctoral dissertation, Tempere University of Technology (TUT), 

Publication 885.

Labuschagne, C., Brent, A.C., 2005. Sustainable project life cycle management: the need to integrate 

life cycles in the manufacturing sector. International Journal of Project Management 2005, 23(2), 

159-168.
22

Leahu-Aluas, S., 2009-2010. Sustainable Manufacturing: An Overview for Manufacturing Engineers. 

Sustainable Manufacturing consulting [online], Available from: 

http://sustainablemanufacturing.biz/ [Accessed 29 August 2013].

Lowell Center for Sustainable Production (LCSP), 1998. What is Sustainable Production? [online]. 

Available from: http://www.sustainableproduction.org/abou.what.php [Accessed 22 May 2012].

Lozano, R., 2008. Envisioning sustainability three-dimensionally. Journal of Cleaner Production, 

16(17), 1838-1846.

Manufacturing Skills Australia (MSA), 2008. Sustainable manufacturing - Manufacturing for 

sustainability [online]. Available from: http://www.mskills.com.au/ShopProduct.aspx?pid=42

[Accessed 10 September 2012].

National Council for Advanced Manufacturing (NACFAM), 2009. Sustainable Manufacturing 

[online]. Available from: http://www.nacfam.org/Default.aspx [Accessed 28 August 2013].

Ness, B., Urbel-Piirsalu, E., Anderberg, S., Olsson, L., 2007. Categorising tools for sustainability 

assessment. Ecological Economics, 60(3), 498-508.

Organisation for Economic Co-operation and Development (OECD), 2009. Overview of the OECD 

Sustainable Manufacturing Toolkit Prototype [online]. Available from: 

http://www.oecd.org/industry/industryandglobalisation/44280332.pdf [Accessed 4 July 2012].

Organisation for Economic Co-operation and Development (OECD), 2011. OECD Sustainable 

Manufacturing Toolkit [online]. Available from: http://www.oecd.org/innovation/green/toolkit/

[Accessed 15 August 2012].

Patient-Centered Outcomes Research Institute (PCORI), 2012. Comprehensive Inventory of Research 

Networks [online]. Available from: http://www.pcori.org/ [Accessed 9 February 2013].

Pope, J., Annandale, D., Morrison-Saunders, A., 2004. Conceptualising Sustainability Assessment. 

Environmental Impact Assessment Review, 24(6), 595-616.

Prentice Hall, 2001. Prentice Hall Science Explorer Inquiry Skills Activity Book. Prentice Hall, 

California.

Product Design [online]. Available from: http://www.businessdictionary.com/definition/productdesign [Accessed 20 November 2012].

Rachuri, S., Sriram, R.D., Narayanan, A., Sarkar, P., Lee, J.H., Lyons, K.W., Kemmerer, S.J., 2010. 

Sustainable Manufacturing: Metrics, Standards, and Infrastructure. Workshop Report of National 

Institute of Standards and Technology, NISTIR 7683.

Reich-Weiser, C., Vijayaraghavan, A., Dornfeld, D.A., 2008. Metrics for Sustainable Manufacturing. 

Proceedings of the 2008 International Manufacturing Science and Engineering Conference, Illinois,

USA.

Rosen, M.A., Kishawy, H.A., 2012. Sustainable Manufacturing and Design: Concepts, Practices and 
23

Needs. Sustainability, 4(2), 154-174.

Shoemaker, P.J., Tankard, J.W., Lasorsa, D.L., 2004. How to build social science theories. SAGE 

Publications, Inc.

Sikdar, S. K., 2003. Sustainable development and sustainability metrics. American Institute of 

Chemical Engineers Journal, 49(8), 1928-1932.

Singh, R.K., Murty, H.R., Gupta, S.K., Dikshit, A.K., 2012. An overview of sustainability assessment 

methodologies. Ecological Indicators, 15(1), 281-299.

University of North California (UNC) Institute for the Environment, 2010. Sustainable Research 

Inventory [online]. Available from: http://www.ie.unc.edu/index.cfm [Accessed 14 January 2013].

US Department of Commerce (DOC), 2009. How does Commerce define Sustainable Manufacturing?

[online]. Available from: http://trade.gov/index.asp [Accessed 28 May 2012].

US Department of Defense (DoD), 2011. Technology Readiness Assessment (TRA) Guidance [online]. 

Available from: http://www.acq.osd.mil/chieftechnologist/publications/docs/TRA2011.pdf

[Accessed 17 November 2012].

US National Research Council (NRC), Board on Sustainable Development, 1999. Our Common 

Journey: A Transition Toward Sustainability. Washington, DC: National Academy Press.

Verheem, R., 2002. Recommendations for Sustainability Assessment in the Netherlands. Proceedings 

of the 22nd Annual Conference of the International Association for Impact Assessment.

Watt, J.H., van den Berg, S., 1995. Research Methods for Communication Science. Allyn and Bacon, 

Needham Heights, MA.

Wikipedia, The Free Encyclopedia, Operational Definition [online]. Available from: 

http://en.wikipedia.org/wiki/Operational_definition [Accessed 19 November 2012].

Wikipedia, The Free Encyclopedia, Technology Readiness Level (TRL) [online]. Available from: 

http://en.wikipedia.org/wiki/Technology_readiness_level [Accessed 30 November 2012].