# Maturity Models and Tools for Enabling Smart Manufacturing Systems: Comparison and Reflections for Future Developments

Anna De Carolis, Marco Macchi, Boonserm Kulvatunyou, Michael P. Brundage, Sergio Terzi

Brundage, Sergio Terzi

► To cite this version:

Anna De Carolis, Marco Macchi, Boonserm Kulvatunyou, Michael P. Brundage, Sergio Terzi. Maturity Models and Tools for Enabling Smart Manufacturing Systems: Comparison and Reflections for Future Developments. 14th IFIP International Conference on Product Lifecycle Management (PLM), Jul 2017, Seville, Spain. pp.23-35, 10.1007/978-3-319-72905-3_3 . hal-01764203

HAL Id: hal-01764203

https://inria.hal.science/hal-01764203

Submitted on 11 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Maturity Models and Tools for Enabling Smart Manufacturing Systems: Comparison and reflections for future developments

Anna De Carolis<fnref n="1" />, Marco Macchi<fnref n="1" />, Boonserm Kulvatunyou<fnref n="2" />, Michael P. Brundage<fnref n="2" />, Sergio Terzi<fnref n="1" />

**Abstract.** One of the most exciting new capabilities in Smart Manufacturing (SM) and Cyber-Physical Production Systems (CPPS) is the provisioning of manufacturing services as unbundled "apps or services", which could be significantly more flexible and less expensive to use than the current generation of monolithic manufacturing applications. However, bundling and integrating heterogeneous services in the form of such apps or composite services is not a trivial job. There is a need for service vendors, cloud vendors, manufacturers, and other stakeholders to work collaboratively to simplify the effort to "mix-and-match" and compose the apps or services. In this regard, a workshop was organized by the National Institute of Standards and Technology (NIST) and the Open Applications Group Inc. (OAGi), with the purpose to identify – through parallel sessions – technology and standard needs for improving interoperability and composability between services. The workshop was organized into five working session. This paper documents evidences gathered during the “Smart Manufacturing Systems Characterization” (SMSC) session, which aims at establishing a roadmap for a unified framework for assessing a manufacturer’s capability, maturity and readiness level to implement Smart Manufacturing. To that end, the technology maturity, information connectivity maturity, process maturity, organizational maturity, and personnel capability and maturity, have been identified as critical aspects for Smart Manufacturing adoptions. The workshop session culminated at providing a coherent model and method for assisting manufacturing companies in their journey to smart manufacturing realizations. This paper shows three different maturity models and tools that, thanks to their complementarity, enable one to reflect on the different perspectives required by SMSC. These models and tools are usable together for assessing a manufacturing company’s ability to initiate the digital transformation of its processes towards Smart Manufacturing. Therefore, based on their comparison, the ultimate purpose of the research is to come up with a set of coherent guidelines for assessing a manufacturing system and its management practices for identifying improvement opportunities and for recommending SM technologies and standards for adoption by manufacturers.

**Keywords:** Smart Manufacturing Systems Characterization; maturity model; manufacturer’s capabilities; Industry 4.0; Smart manufacturing readiness

<footnote n="1">¹ Politecnico di Milano, Via Lambruschini 4/b, Milano, Italy</footnote>

<footnote n="2">² National Institute of Standards and Technology (NIST), Gaithersburg, MD, USA</footnote>
# 1 Introduction

With the introduction of Smart Manufacturing (SM), manufacturers are faced with new, advanced technologies that offer the potential to dramatically improve their manufacturing systems. Many definitions of SM have been proposed. Price Waterhouse Cooper identifies eleven digital technologies that are SM enablers [1]: mobile devices, IoT platforms, location detection technologies, advanced human-machine interfaces, advanced authentication and fraud detection, 3D printing, smart sensors, advanced algorithm for big data analytics, multilevel customer interaction, wearable augmented reality, and cloud computing. It is interesting to observe the wide scope of applications enabled by combining these technologies including supply chains, business models, and designs of products and services.

SM focuses on the end-to-end digitalization of all physical assets and integration into digital ecosystems with value chain partners [1]. A SM research project within the European union indicates that application of digital technologies in manufacturing will have three key impacts [2]: (i) full integration of product and asset life-cycle both within and outside the factory from cradle to grave; (ii) full integration of all the stakeholders in the value-network (i.e. suppliers and customers). The entire manufacturing system will be connected based on shared vision, standards, and service-oriented integration mechanisms that defy traditional, rigid functional hierarchies to create dynamic structures from their articulated functions; (iii) new business models based on new value-added services enabled by these technologies.

The application of new digital technologies in manufacturing leads to the rise of new, complex enterprise challenges [3][4][5]. Integrating so many different technologies inevitably leads to increased complexity of the whole manufacturing system, which might limit the obtainable advantages. For this reason, we assume that the impact of smart manufacturing technology introduction may differ depending on the maturity level of a company's capabilities. Before starting the transition towards SM or Industry 4.0, manufacturing companies should define their transformation roadmap according to the maturity level of their capabilities [6]. This requires proper methodologies for maturity or readiness assessment with respect to SM. The aim is to support companies in finding their own path to adopting SM technologies.

Even though such assessment methodologies are emerging, there is no established approach or framework. This paper describes three different, but complementary tools for analyzing the readiness of manufacturing systems and environments from an SM perspective. These are: DREAMY (Digital Readiness Assessment Maturity model), SMSRL (Smart manufacturing readiness level), and MOM (Manufacturing Operations Management) Capability Maturity Model. Based on a comparison of these methods, we propose to establish a set of guidelines for maturity assessment to support the transition towards SM.
# 2 Smart Manufacturing Systems Characterization (SMSC) and Maturity Models

## 2.1 Smart Manufacturing Systems Characterization

Smart manufacturing systems rely on new information technologies in supply chains, in product development, in business to shop floor integration, in operations of smart products, and in production equipment [7]. SM is a convergence of new technologies and related capabilities brought from multiple areas and multiple business lifecycles. In order to guide manufacturers coping with the adoption of such complex systems, NIST and international researchers identified requirements for manufacturing systems characterization based on work carried out during the NIST/OAGI Workshop 2016 [8]. Smart manufacturing systems characterization will help identify and prioritize opportunities for improvement of manufacturing systems by providing recommendations on which SM technologies and standards to implement [8].

Manufacturers need to adopt a progressive introduction of SM applications, systems, and hardware based on a composition of different technologies [6]. The introduction of new technologies depends on understanding the actual readiness of the manufacturer to deploy the new technologies in its manufacturing system(s). Manufacturers should perform periodic assessments to monitor the maturation process towards SM. Manufacturing systems characterization is focused on the assessment of a manufacturer's capabilities, and readiness level to implement SM technologies and applications. The maturity of a company's manufacturing systems is a key indicator for success in adopting SM technologies. A maturity model is a critical tool to perform a characterization of existing manufacturing systems.

## 2.2 Maturity Models and Tools

It is appropriate to provide a definition of maturity for this paper since the understanding of maturity can vary even within the same field of expertise [9].

Maturity can be defined as “*the state of being complete, perfect or ready*” [10][11][12]. Another slightly different perspective on the concept of maturity is the one given by Maier et al. [9], who stated that the process of bringing something to maturity means bringing it to a state of full growth. In other words, maturity implies an evolutionary progress from an initial to a desired or normally occurring end stage [13]. This last definition, which stresses the process toward maturity, introduces another important concept, which is the one of *stages of growth* or *maturity levels*.

Before reaching a state of “full growth”, an entity (an organization as well as a human being) must encounter different stages of growth or maturity levels. In particular, the stages an organization passes through have three main distinctive properties [14]: (i) they are sequential in nature; (ii) they occur in a hierarchical progression that is not easily reversible; and (iii) they involve a broad range of organizational activities and structures. We can state that maturity models can be used as tools for determining manufacturers’ readiness level and capabilities within an SM perspective.
Maturity models in literature have different characteristics. Fraser et al. [16] pre-
sented a first clear classification per typology of maturity models. In particular, they
distinguish three types of maturity models: (i) Maturity grids; (ii) Likert-like question-
naires; and (iii) CMM-like models.

The maturity grids typically illustrate maturity levels in a simple and textual manner, structured in a matrix or a grid. As Fraser et al. stated, maturity grids are of a moderate complexity and they do not specify what a particular process should look like. Maturity grids only identify some characteristics that any process and every enterprise should have in order to reach high performance processes [9]. On the other hand, the Likert-like questionnaires are constructed by “questions”, which are no more than statements of good practices. A responder to the questionnaire has to score the related performance on a scale from 1 to n. A hybrid model can be defined as a combination of the questionnaire approach with the maturity grid definition [16]. Finally, the CMM-like models (Capability Maturity Model) identify the best practices for specific processes and measures the maturity of organizations in terms of how many practices are implemented [9]. Their architecture is more formal and complex compared to the first two. CMM models are composed of process areas organized by common features, which specify key practices to address a series of goals. Typically, the CMM-like models exploit Likert questionnaires to assess the maturity. The framework for defining maturity models have been improved successively by the Capability Maturity Model Integration (CMMI), which expands the scope of the original CMM beyond software development maturity [17].

Although a number of different types of maturity models have been proposed in lit-
erature, they share some common proprieties [6] [16]: (i) Maturity levels (typically
from three to six); (ii) a “descriptor” for each level, which gives a meaningful name to
each level; (iii) a generic description of the characteristics of each level; (iv) a number
of dimensions or “process areas”; (v) a number of elements or activities for each process
areas; and (vi) a description of each activity, that has to be performed at each maturity
level.

The terms ‘readiness’ and ‘maturity’ are relative and related. We define the term ‘smart manufacturing readiness’ as the capability or maturity of a manufacturing company **to deploy** smart manufacturing concepts, and the term ‘smart manufacturing maturity’ as how well a manufacturing company **has employed** smart manufacturing concepts or its smart manufacturing capability. Following such definitions, certain maturity models can be viewed as part of smart manufacturing readiness assessment. For example, the manufacturing operation management (MOM) maturity model is a smart manufacturing readiness assessment. On the other hand, Industrie 4.0 Readiness [25], although calling itself readiness, is more of a smart manufacturing maturity. In the following section, three different tools for assessing manufacturers’ readiness or maturity levels **to deploy** SM concepts are described.

2.3 DREAMY (Digital REadiness Assessment MaturitY model)

The Digital Readiness Assessment MaturitY model is a tool with two objectives. The
primary objective is to assess a manufacturing company’s readiness level for starting
the digital transitioning process, which is an aspect of smart manufacturing concepts [10–12]. For this reason, DREAMY has the form of a maturity model based on the principles of the CMMI framework [18,19] as shown in Table 1. The secondary objective is to identify a manufacturing company’s strengths, weaknesses, opportunities, and create a roadmap for investments in digitalization and transitioning to smart manufacturing [6].

<table><caption>**Table 1.** Definition of DREAMY Maturity Levels (taken from [20])</caption><thead><tr><th></th><th></th></tr></thead><tbody><tr><td>ML 1 Initial</td><td>The process is poorly controlled or not controlled at all, process management is reactive and does not have the proper organizational and technological "tools" for building an infrastructure that will allow repeatability / usability / extensibility of the utilized solutions.</td></tr><tr><td>ML2 Managed</td><td>The process is partially planned and implemented. Process management is weak due to lacks in the organization and/or enabling technologies. The choices are driven by specific objectives of single projects of integration and/or by the experience of the planner, which demonstrates a partial maturity in managing the infrastructure development.</td></tr><tr><td>ML3 Defined</td><td>The process is defined with the planning and the implementation of good practices and management procedures. The management of the process is limited by some constraints on the organizational responsibilities and / or on the enabling technologies. Therefore, the planning and the implementation of the process highlights some gaps/lacks of integration, information exchange, and ultimately interoperability between applications.</td></tr><tr><td>ML4 Integrated and interoperable</td><td>The process is built on information exchange, integration, and interoperability across applications; and it is fully planned and implemented. The integration and the interoperability are based on common and shared standards within the company, borrowed from intra- and/or cross-industry de facto standards, with respect to the best practices in industry in both perspectives of the organization and enabling technologies.</td></tr><tr><td>ML5 Digital-oriented</td><td>The process is digitally-oriented and is based on a solid technology infrastructure and on a high potential growth organization, which supports – through pervasive integration and interoperability – speed, robustness and security in information exchange, in collaboration among the company functions and in the decision making.</td></tr></tbody></table>

To define the DREAMY architecture, it was fundamental to identify the relevant manufacturing operational processes, within which value-added activities are performed, and that are strategic for the digital transition to SM [20]. To make the architecture as general as possible, manufacturing operational processes were grouped in five main areas: 1) Design and Engineering; 2) Production Management; 3) Quality Management; 4) Maintenance Management; 5) Logistics Management. Each process area can be considered as a self-contained module and therefore it is possible to add or remove areas as needed based on certain industrial situations. Cutting across the process areas is the Digital Backbone, within which all the information exchange processes across the process areas are considered [20]. The digital readiness of a manufacturing company is then defined through a scale of maturity levels. These levels provide a snapshot of the company’s current abilities. The maturity levels are based on the principles from the CMMI framework [19] [18]. The CMMI maturity levels provide a generic starting point. These maturity levels have been adapted in order to gather the definitions,
and so the semantic, of the digital readiness levels for the DREAMY model [20] (see **Table 1**).

According to the maturity level definitions in Table 2, a manufacturing company’s digital readiness needs to be evaluated along the four dimensions as shown in **Fig. 1**: *Process, Monitoring and Control, Technology, and Organization* [20].

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** DREAMY (Adapted from [20])</figcaption></figure>

The DREAMY model in its current form can be used for descriptive purposes. That is, maturity indexes for each process can be calculated to reflect the as-is situation of a
manufacturing company [20]. With further analysis, strengths, weaknesses, and opportunities (prescription [6]) for smart manufacturing adoption can be derived. Going forward, the model can be enhanced such that the strengths, weaknesses, and opportunities information can be automatically generated. The “factory” is the unit of the analysis considered by the model. However, thanks to the modular structure of the model, future work can be done to include other value-added process areas, such as Supply Chain Management, Sales, Marketing, Customer care, and Human Resource Management to extend the scope of the analysis. In addition, *Skills of Personnel* should be considered as another analysis dimension when assessing company capabilities, because specialized skills are needed to deploy smart manufacturing systems.

## 2.4 SMSRL

Smart Manufacturing System Readiness Level (SMSRL) is an index that measures a manufacturing company's readiness for employing smart manufacturing concepts with the assumption that smart manufacturing is essentially the intensive use of information and communication technologies to improve manufacturing system performance [21]. The SMSRL index bases its readiness model on the factory design and improvement (FDI) activity model [22,23]. FDI consists of four high-level activities as shown in Fig. 2. Each activity has one further level of decomposition consisting of processes that should be regularly performed for continuously improving factory operational performances. The information flow between activities and the software functions supporting each activity are captured in the activity model.

<figure><img src="image_3.png" /></figure>
<caption>**Fig. 2.** Factory Design and Improvement Activity Model (from [23])</caption>

The figure shows software functions grouped into five categories entering the bottom of each activity box SCM (supply chain management), ERP (enterprise resource planning), DM (digital manufacturing), PLM (product life cycle management), and MES (manufacturing execution system). The more tasks performed, the more they are assigned with a responsible person, software functions deployed, and digital information used, the more ready a factory is for the deployment of smart manufacturing concepts. The contribution of these aspects and dimensions to the smart manufacturing readiness is illustrated **Fig 3**. Differing ways of computing readiness index are used for C1 to C4. C1 uses the CMMI index qualification. C2 and C3 uses counting measures, while C4 uses incidence matrix-based similarity measure along with an incidence scoring scheme based on the technology used to enable the information flow. They are viewed independently or averaged into a single SMSRL index.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** SMSRL Maturity Dimensions [21]</figcaption></figure>

Like other models, the output from an SMSRL assessment is largely descriptive. After an assessment, a company can use the model to prescribe goals to improve the readiness, but the model has not yet included guidelines for achieving those goals. The FDI activity model underlying the assessment focuses on factory improvement tasks, not day-to-day factory operation tasks, and has weaknesses on supply chain and logistics operations. The IT maturity dimension in the SMSRL assessment evaluates software functions utilized by a manufacturing companies as part of the *readiness* assessment. Some of these software functions may be considered smart manufacturing capabilities causing the SMSRL readiness index to overlap the smart manufacturing *maturity* assessment. Descoping of the FDI may be necessary to avoid this overlap. Consequently, the SMSRL assessment will benefit from alignment and harmonization with other assessment methods described in this paper.
## 2.5 MOM Maturity

MESA (Manufacturing Enterprise Systems Association) created the Manufacturing Operations Management Capability Maturity Model (MOM/CMM) to evaluate the maturity of manufacturing enterprises' manufacturing facilities [24]. The objective is to determine the policy, procedure, and execution of a manufacturing operation management to be organized, robust, and repeatable. In other words, MOM/CMM does not provide a measure of sophistication of the physical production, but a measure of the capability to streamline operations, particularly in response to abnormal events. The MOM/CMM focuses on four main *process areas*:

1. Production Operations Management

2. Inventory Management

3. Quality Test Operations Management

4. Maintenance Operations Management

Each *process area* consists of multiple *activities*: scheduling, dispatching, execution management, resource management, definition management, data collection, tracking, and performance analysis [24]. Each *activity* can have a maturity level from level 0 to level 5. The maturity levels are characterized in Table 2.

The higher the level of maturity, the more likely an efficient organization and fewer problems at the manufacturing operations management level. The maturity levels can be also applied across different aspects, such as roles and responsibilities, succession plans and backups, policies and procedures, technology and tools, training, information integration, and KPIs. The model, in its raw form, can be time and resource consuming to complete with 832 questions and lacks improvement strategies based on the results. However, the model can provide a benchmark for comparison to others in their industry and can aid in understanding where to make improvements. Future work will simplify the questionnaire and map improvement strategies to the results.

<table><caption>**Table 2.** MOM Maturity Level Definitions</caption><tr><td>Level 0</td><td>There has been no evaluation performed.</td></tr><tr><td>Level 1</td><td>Procedures for activities and their executions are at initial stage and not documented or formally managed.</td></tr><tr><td>Level 2</td><td>Procedures for some activities are documented and executed with possibly repeatable results in the normal situation.</td></tr><tr><td>Level 3</td><td>Procedures for activities are defined with documented standards for all activities whose executions are possibly supported by software tools and better handling of abnormal situations.</td></tr><tr><td>Level 4</td><td>Procedures for activities are defined and documented across all organizational groups; and their executions are repeatable and monitored with software tools supports.</td></tr><tr><td>Level 5</td><td>Procedures for activities are focused on continuous improvement and optimization.</td></tr></table>
## 3 Models Comparison: Building a Framework for SMSC and Road-Mapping its Development

From the review of the different methods described in the previous chapters (DREAMY, SMSRL, MOM maturity models), it is possible to state that they are complementarity in the overall scope of Smart Manufacturing. The MOM maturity model, which focuses on day-to-day factory operation tasks, can be complemented by SMSRL, which focuses more on assessing the maturity of factory improvement tasks. Neither MOM nor SMSRL includes product life cycle and business processes in their scope of analysis, so they can be complemented by DREAMY, which offers a business processes-oriented view also on product life cycles phases. According to their *different but complementary* objectives, DREAMY, SMSRL, and MOM models might be used by manufacturing companies with *different but complementary* purposes, i.e., descriptive and prescriptive, and descriptive and comparative respectively.

The table below summarizes the three models showing their objectives, clarifying their focus, and describing their structures.

<table><caption>**Table 3.** Comparison of DREAMY, SMSRL and MOM models</caption><thead><tr><th>Element</th><th>DREAMY</th><th>SMSRL</th><th>MOM</th></tr></thead><tbody><tr><td>Objective(s)</td><td>1. To assess a manufacturing company readiness level for starting the digital transformation process<br>2. To identify strengths and weaknesses and related opportunities manufacturers can gather from the digital transformation, with the final aim to help them in defining a roadmap for prioritizing investments</td><td>To assess a manufacturing company's readiness to employ data-intensive technologies for its performance management.</td><td>To determine level of an organization's capability to have mature, robust, and repeatable manufacturing operations [24].</td></tr><tr><td>Focus</td><td>Manufacturing company / Product and Factory Life Cycles</td><td>Maturity of performance improvement tasks/processes, availability of software supports, maturity of information sharing capability, and availability of responsible personnel</td><td>Manufacturing Operations Management (MOM) processes</td></tr><tr><td>Analysis Dimensions</td><td>Process / Execution, Monitoring and control, Organization, Technology</td><td>Organization, IT, Performance Management (process execution), and Information Connectivity</td><td>Process / Execution</td></tr><tr><td>Process Areas</td><td>Product and asset design and engineering, Production management, Quality management, Maintenance management, Logistics management, Digital Backbone</td><td>(Change) Requirement developments, Basic (rough) design of a new or a change requirement, Detail design, and Test</td><td>Production Operations Management, Inventory Management, Quality Test Operations Management, Maintenance Operations Management</td></tr><tr><td>Maturity levels</td><td>5 (1-5)</td><td>6 (0-5)</td><td>6 (0-5)</td></tr></tbody></table>
<table><tr><td>Inspiring framework</td><td>CMMI</td><td>Factory Design and Improvement Activity model</td><td>ISA-95 Enterprise Control Activities</td></tr><tr><td>Assessment methods</td><td>Interview / case study</td><td>Self-assessment</td><td>Self-assessment</td></tr><tr><td>Model purpose</td><td>Descriptive and prescriptive</td><td>Descriptive and comparative</td><td>Descriptive and comparative</td></tr><tr><td>Questions / Answers' type</td><td>Questions with normative answers</td><td>Yes/No Question, Scoring Question</td><td>Yes/No Questions</td></tr><tr><td>Number of questions</td><td>About 200 scoring questions</td><td>242 scoring and at least ~123 Yes/No questions</td><td>832 Yes/No Questions</td></tr></table>

## 4 Conclusions

The “recipe” for smart manufacturing involves combining different “ingredients” to obtain the best results in terms of performance improvements. Companies planning to build SM systems must possess basic capabilities. What are these capabilities and how can they be measured? This paper carries out some reflections of smart manufacturing system characterization (SMSC), showing three different tools for assessing manufacturing companies their ability to start the digital transitioning process. With this comparison, it is now possible to reflect on the different perspectives required by SMSC and on future developments expected for such type of tools.

First of all, the evidence from the literature and from the thoughts that emerged during the workshop organized by NIST and OAGi [8] show several perspectives on capabilities required in terms of organization, process execution and technology. Models and tools for assessing enterprise readiness to embrace SM should consider all these different perspectives to be effective. Therefore, we may expect that current and emerging models and tools may address additional perspectives. It is worth remarking that these models and tools should not be used solely during the assessment phase. Instead, they should be enhanced to support the prescription phase of improvements. In addition, with sufficient improvement data, benchmarks can be developed to provide evidence of return-on-investment for smart manufacturing systems adoption. This would accelerate overall industry adoption of SM. Finally, further studies should deal more with principles, providing an abstract view on the founding concepts to adequately address differences between other “readiness” and “maturity” models to suggest the most appropriate tool to use in each of the digitalization roadmapping phases.

## 5 References

[1] P. GMIS, Industry 4.0: Building the Digital Industrial Enterprise, 2016. https://www.pwc.com/m1/en/publications/documents/middle-east-industry-4-0-survey.pdf.

[2] E.R. for C. in M. SCorPiuS, Validated sCorPiuS Vision, (2016) 1–17.

[3] J. Jager, O. Schollhammer, M. Lickefett, T. Bauernhansl, Advanced complexity management strategic recommendations of handling the “Industrie 4.0” complexity for small and medium enterprises, in: 49th CIRP Conf. Manuf. Syst. (CIRP-CMS 2016), 2016.

[4] W. Elmaraghy, H. Elmaraghy, T. Tomiyama, L. Monostori, Complexity in engineering design and manufacturing, CIRP Ann. - Manuf. Technol. 61 (2012) 793–814. doi:10.1016/j.cirp.2012.05.001.

[5] J. Jäger, A. Kluth, A. Schatz, T. Bauernhansl, Complexity patterns in the advanced complexity management of value networks, in: Procedia CIRP, 2014: pp. 645–650.
doi:10.1016/j.procir.2014.01.070.

[6] A. De Carolis, M. Macchi, E. Negri, S. Terzi, Guiding manufacturing companies towards digitalization: A methodology for supporting manufacturing companies in defining their digitalization roadmap, 2017.

[7] D. Brandl, SMART MANUFACTURING SYSTEM CHARACTERIZATION SESSION, (2016).

[8] NIST/OAGi, Drilling down on Smart Manufacturing - Enabling Composable Apps, (2016).
https://www.nist.gov/news-events/events/2016/04/nistoagi-workshop-drilling-down-smart-manufacturing-enabling-composable (accessed April 18, 2017).

[9] A.M. Maier, J. Moultrie, P.J. Clarkson, Assessing organizational capabilities: Reviewing and guiding the development of maturity grids, IEEE Trans. Eng. Manag. 59 (2012) 138–159.
doi:10.1109/TEM.2010.2077289.

[10] J.A. Simpson, E.S.C. Weiner, The Oxford English Dictionary, 1989.

[11] T. Mettler, A Design Science Research Perspective on Maturity Models in Information Systems, Design. 41 (2009) 1–13. doi:10.2174/97816080506351100101.

[12] H. Karkkainen, A. Silventoinen, Different Approaches of the PLM Maturity Concept and Their Use Domains - Analysis of the State of the Art, in: 12th IFIP WG 5.1 Int. Conf. PLM 2015, Springer International Publishing Switzerland 2016, Doha, Quatar, 2015: pp. 89–102. doi:10.1007/978-3-319-33111-9_9.

[13] T. Mettler, P. Rohner, Situational maturity models as instrumental artifacts for organizational design, Proc. 4th Int. Conf. Des. Sci. Res. Inf. Syst. Technol. - DESRIST '09. Artic. No. 22. (2009) 1–9. doi:10.1145/1555619.1555649.

[14] P. Gottschalk, Maturity levels for interoperability in digital government, Gov. Inf. Q. 26 (2009) 75–81. doi:10.1016/j.giq.2008.03.003.

[15] M. Van Steenbergen, R. Bos, S. Brinkkemper, I. Van De Weerd, W. Bekkers, The design of focus area maturity models, in: CEUR Workshop Proc., 2010: pp. 17–19. doi:10.1007/978-3-642-13335-0_22.

[16] P. Fraser, J. Moultrie, M. Gregory, The use of maturity models/grids as a tool in assessing product development capability, in: IEEE Int. Eng. Manag. Conf., 2002.

[17] C.P. Team, Capability Maturity Model\{textregistered\} Integration (CMMI SM), Version 1.1, C. Syst. Eng. Softw. Eng. Integr. Prod. Process Dev. Supplier Sourc. (CMMI-SE/SW/IPPD/SS, V1.1). (2002).

[18] M. Macchi, L. Fumagalli, S. Pizzolante, A. Crespo, J.F. Marquez, G. Fernandez, Towards eMaintenance: maturity assessment of maintenance services for new ICT introduction, in: APMS 2010 Int. Conf. Adv. Prod. Manag. Sys-Tems, Cernobbio, Italy, 2010.

[19] M. Macchi, L. Fumagalli, A maintenance maturity assessment method for the manufacturing industry, J. Qual. Maint. Eng. 19 (2013) 295–315. doi:10.1108/JQME-05-2013-0027.

[20] A. De Carolis, M. Macchi, E. Negri, S. Terzi, A maturity model for assessing the digital readiness of manufacturing companies, 2017.

[21] K. Jung, B. Kulvatunyou, S. Choi, M.P. Brundage, An Overview of a Smart Manufacturing System Readiness Assessment, in: Springer-Verlag Berlin Heidelb., 2016.

[22] K. Jung, S. Choi, B. Kulvatunyou, H. Cho, K.C. Morris, A reference activity model for smart factory design and improvement, Prod. Plan. Control. (2016). doi:10.1080/09537287.2016.1237686.

[23] B.. Kulvatunyou, Factory Design and Improvement (FDI) Activity Model, (accessed May 2017) .
https://www.nist.gov/services-resources/software/factory-design-and-improvement-fdi-activity-model.

[24] D. Brandl, MESA MOM Capability Maturity Model Version 1.0, (2016).

[25] Lichtblau, K. et al. Industrie 4.0 Readiness, VDMA, Available online: https://www.industrie40-readiness.de/?lang=en