# The IoT Technological Maturity Assessment Scorecard:
A Case Study of Norwegian Manufacturing Companies

Bjørn Jæger, Lise Lillebrygfjeld Halse

► To cite this version:

Bjørn Jæger, Lise Lillebrygfjeld Halse. The IoT Technological Maturity Assessment Scorecard: A Case Study of Norwegian Manufacturing Companies. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2017, Hamburg, Germany. pp.143-150, 10.1007/978-3-319-66923-6_17. hal-01666214

HAL Id: hal-01666214

https://inria.hal.science/hal-01666214

Submitted on 18 Dec 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# The IoT Technological Maturity Assessment Scorecard:
A case Study of Norwegian manufacturing companies

Bjørn Jæger and Lise Lillebrygfjeld Halse

Molde University College, Specialized University in Logistics, Molde, Norway

{bjorn.jager, lise.l.halse}@himolde.no

**Abstract.** The accelerated use of technologies has led to what is termed the fourth industrial revolution, or Industry 4.0. It is based on machinery, robots, production lines, items and operators connected via the Internet to each other and to back-end systems, as a part of the Internet of Things (IoT). In this paper, we propose a new IoT Technological Maturity Assessment Scorecard that can assist manufacturers in adopting IoT-technologies. To demonstrate the Scorecard, we present a case study applying the scorecard in four Norwegian manufacturing companies.

**Keywords:** Industry 4.0, Internet of Things, Maturity model, Manufacturing

## 1 Introduction

International competition, global sourcing of production and financial crisis calls for a new level of excellence in manufacturing. A fourth industrial revolution is envisioned based on innovations in technologies, smart materials and manufacturing operations. The revolution includes initiatives termed Industry 4.0, the Industrial Internet, Factories of The Future, and Cyber Physical Systems. A driving force for this development is the accelerated use of Internet of Things (IoT)-technologies [1]. The challenge for manufacturers is the escalating technological change, and they need a tool to measure and analyze their current technology level, assisting them in giving directions for adopting new technologies. A suitable tool for this is a maturity model together with a scorecard to assess the maturity level of a company. Maturity models serve as a reference to implement improvements while assessing one's own capabilities compared with others. Maturity assessment has been successfully applied in other industries, most notably the software industries [2]. There has been a growing interest in maturity models within many domains [2-4]. However, models for how to evaluate the technology maturity level of manufacturing companies are scant, and even more so are tools to support the use of models in decision-making. In this study, we present a model for measuring the IoT-technology maturity level of manufacturing companies, followed by results from a case study of using the model for four Norwegian manufacturing companies.
# 2 Literature review

## 2.1 Maturity Models

In general, maturity models outline a path to maturity, involving development stages building on each other until maturity is reached [3, 4]. In order to ensure research rigor, we follow the Comprehensive Research Framework for Maturity Model Research presented by Wendler [3]. This framework states that maturity model development should be an iterative process where each cycle consists of the three steps: Model Development, Model Application, and Model Validation. Model Development follows the seven research guidelines of design science research. Model Application and Model Validation are conducted by applying the model for assessing four manufacturers, using interviews and a survey for validation purposes. De Bruin et al. [7] has proposed a methodology that consists of six phases for development of maturity models (Fig. 1).

<figure><img src="image_2.png" /><figcaption>Fig. 1. Six phases of developing maturity model. (Adopted from de Bruin et al. [5])</figcaption></figure>

## 2.2 Defining a Thing in the Internet of Things (IoT): the 4.0-enabled-object

A crucial issue is to define what distinguishes IoT-technologies from earlier generations of technologies, namely mechanical-, electrical-, computer-technologies. Taken literally, Internet of Things means things connected to form a network. Using capitalized “Internet” signals that we mean the Internet communication network [6]. However, the literature is lacking a clear definition of what a “thing” actually is. To characterize a “thing”, different terms are typically used. Some examples are “Smart Object”, “Smart Thing”, “Smart Product”, and “Intelligent Product”. In order to develop the IoT maturity model we need to define what we mean by a “thing” in IoT. Intuitively, a characteristic is that the “thing” must have the ability to communicate via the Internet. Either directly if the thing has Internet Protocol (IP) capabilities embedded, or in case the “thing” does not have IP capabilities, via a device that connect the “thing” to the Internet. Thus, for the purposes of this research we define the “thing” as a “4.0-enabled-object” with some characteristics that separate it from any-thing. I.e. a “4.0-enabled-object” should have some properties that separate it from objects in general, and from a “3.0-enabled-object” in particular. It should be an extension of a 3.0-enabled-object, so we start by identifying the characteristics of a 3.0-enabled-object. This implies an object that exhibits properties that was not in the 2.0 area. Following the analysis of the Fraunhofer institute [https://www.fraunhofer.de/], we define the PLC (Programmable Logical Controller) as the central 3.0-object. The PLC was designed for controlling manufacturing machinery and equipment. The PLC contained all three elements of a computer in one unit, namely the computer memory, processing capability and Input/Output (I/O) communication facilities. The PLC is
thus the core component of the IoT-technologies. However, some additional require-
ments need to be included. According to Porter and Heppelmann [1], all smart, con-
nected products from home appliances to industrial equipment shares three core ele-
ments. These three core elements are; physical components (comprising the product's
mechanical and electrical parts), “smart” components (comprising the sensors, micro-
processors, data storage, controls, software, embedded operating systems, etc.) and
connectivity components (comprising the ports, antennas, protocols enabling wired or
wireless connections with the product). While the smart components enhance the
capabilities and the value of the physical components, the connectivity components
enhance the capabilities and value of the smart components. In addition, the connec-
tivity components enable some of the capabilities to exist beyond the physical product
itself [1]. Based on this, we define a 4.0-enabled-object as an object with three char-
acteristics:

1. Embedded PLC-element. I.e. an electronic component with computer memory, processing capabilities and I/O communication facilities.

2. Associated global unique identifier. An IP-address if the object has IP-communication capabilities. Otherwise a globally unique identifier must be assigned, e.g. by GS1 following the AutoID standards which is typically used for RFID-tags.

3. **Global connectivity.** Wherever the object is, a two-way communication with the object must be possible. In practice, the object needs to be connected to Internet directly or via some middleware software. A non-IP object, like an object with an embedded RFID-tag, needs an intermediate device to connect it to Internet.

2.3 The automation pyramid

The IT-systems related to manufacturing are traditionally classified using the automa-
tion pyramid, as shown in Figure 2.

Note that some manufacturers might not have MES and/or SCADA layers. Humans operating the ERP-system might control the shop floor assets directly. The shop floor consists of assets like CNC-machines, industrial robots and transportation systems interacting with IT-systems that encompass supervision, control and data acquisition (SCADA) capabilities. The MES level interact with the SCADA level below, and the ERP-level above it. The systems at each level can cover decision-making or simply act as a dispatching system for the level above. ERP includes long-term strategic and tactical planning integrated with business-related functions such as customer order handling and available-to-promise (ATP) checks.

The command communication direction in the automation pyramid is traditionally
vertical, i.e. top – down, with lower levels responding to orders from above. The lev-
els might be connected electronically, or human interaction is required to transfer
information between the levels. More recently, the assets on the shop floor are 4.0-
enabled-objects with the ability to communicate horizontally creating some local
intelligence, i.e. they communicate among themselves without involving levels above.
Furthermore, IT-systems are internal to the manufacturing company if they operate in
a closed enterprise environment, and external if they can communicate in an open
environment involving several enterprises. 4.0-enabled objects can communicate to
each other, either indirectly via IT-systems in the automation pyramid, or directly
with each other.

<figure><img src="image_3.png" /><figcaption>Fig. 2. The Automation Pyramid with three IT-system levels above the shop floor.</figcaption></figure>

3 The IoT Technological Maturity Model

Our research follows the five first phases in Figure 1 since the Maintain phase re-
quires a longitudinal study.

**Level 1: 3.0 Maturity:** On level 1, an organization has reached maturity of the third revolution. The first characteristic is some use of “Track and Trace” technology, such as RFID and/or barcodes in the production and/or warehouse environment, but with limited functionality. The second characteristic, is that the organizations have implemented an ERP system to collect, store, manage and interpret data from business activities, such as product planning, manufacturing, inventory, marketing/sales, shipping and payment. The third characteristic is an initial automatization of the production and/or warehouse environment using at least one robot. The ERP level is typically operated manually with limited electronic communication with the levels below. The IT-systems are internal to the organization.

**Level 2: Initial to 4.0 Maturity:** Having one 4.0-enabled-object is the entry requirement. As a second characteristic, robots, machines and IT-systems communicate vertically. The third characteristic imply that assets and/or products can be remotely programmed, accessed, and managed by for instance the use of a PC, tablet, or a smart phone connected to the Internet.

**Level 3: Connected:** The organization have between two and nine 4.0-enabled-objects within the manufacturing assets and/or the products, with vertical communication. The 4.0-enabled objects communicates indirectly via a control system that is
internal or external to the organization. Cloud computing is a way of supporting verti-
cal communication if control systems are operated from a remote system via Internet.
I.e. a platform for connecting devices and sensors in IoT. Cloud computing can be
defined as being "able to access files, data, programs and 3rd party services from a
Web browser via the Internet, hosted by a 3rd party provider", following Kim [7]. A
second characteristic is that at least one operation within the production and/or ware-
house environment is automated.

**Level 4: Enhanced:** The assets and/or products can communicate horizontally and directly within a closed environment. Assets and/or products are internally connected. A second characteristic is that some operations in the production and/or warehouse environment have been automated. Meaning that for instance, robots, machines and IT-systems are connected into a production network, performing the production of e.g. standard parts. Alternatively, robots and/or automated transport carriers have fully automated at least a specific part of the inbound and/or outbound warehouse operations.

**Level 5: Innovating:** A first characteristic is that organizations need to have an internal supply chain control with an increasingly number of IoT-objects (at least ten) within the assets and/or the products. In addition, these IoT-objects need to have the ability of horizontal communication (e.g. robot-to-robot) and vertical communication (e.g. robot-to-Internet). A second characteristic is that the 4.0-enabled- objects are further developed and equipped with advanced features. More specifically, that the objects at this level have self-awareness capabilities, which means that the objects have the ability to know its own status and structure, as well as any changes to it, and its history [8]. A third characteristic is that the production and/or warehouse environment is extensively automated, e.g. the production and/or warehouse environment increasingly use robots replacing the manual workforce. A fourth characteristic involves organizational understanding of the importance of, as well as interacting to achieve, standardization (data standards, wireless protocols, technologies). Without standardization, the communication between asset-to-asset and product-to-product becomes hard, especially communication beyond organizational boundaries. Thus, standardization and interoperability both can be regarded as two especially central elements organizations should be engaged in at this level, since standards are needed for interoperability both within, and between various domains. Interoperability is defined as the ability of a system to interact with other systems, without application of special effort for integration, e.g. customization of interfaces, etc. Interoperability is established on the physical level; when assembling and connecting manufacturing equipment, on the IT-level; when exchanging information or sharing services, and on the business level; where operations and objectives have to be aligned [9].

**Level 6: Integrated:** There is an increasingly number of IoT-objects among both assets and products, and the organization has further implemented IoT-technology with 4.0-enabled-objects communicating directly with humans internally in their or-
ganization. This feature passes beyond self-awareness at the previous level, and in-
cludes the IoT-objects ability to use the information gathered in order to manage its
own life cycle, including services, self-repair and resources. It also includes the ability
to learn from experiences and the ability to improve operations [8]. A third character-
istic is that the production and/or warehouse environment are highly automated, in-
volving robots that perform a high degree of the production and/or warehouse opera-
tions, further replacing the manual workforce. A fourth characteristic, is that the con-
nected robots, machines and products constantly and increasingly are exchanging
various types of information. Consequently, the volume of the generated data and the
processes that is involved in the handling of the data becomes critical and important
to manage. Data management is a crucial aspect within IoT, and organizations at this
level have a deep focus on the exchanged data with a plan and strategy for further
data management. The organizations understand what information they need in order
to create as much value as possible [10].

**Level 7: Extensive:** The range of the organization is extended from being merely internal, to embracing the organizations external network. A second characteristic is that the production and warehouse environment are highly automated, meaning that robots and machines performs a high degree of both production and warehouse operations, replacing a high degree of the manual work operations. A third characteristic is that organizations move from Data Management towards Big Data Management and extensive Data Analysis. Big Data is the result of an extensive implementation of new technology, and the enormous amount of data that arises from the internal and external communication, and the monitoring and measuring of objects (e.g. a robots and/or a products performance), in the business environment. Consequently, Big Data Management, which is the organizations administration and governance of great volumes, of both structured and unstructured data, becomes of crucial importance at this level. The aim of Big Data Management is to extract big data to gain business insights, which further means to ensure a high level of data quality and accessibility for business intelligence and Big Data analytics applications. The fourth main characteristic is developed from the third characteristic, namely that organizations at this level are actively engaged in Data Analysis, with the inspection, cleaning, transforming and modelling of data from sensors, machine-to-machine, and networks, in order to discover useful information and support business conclusions and decision-making [10].

**Level 8: 4.0 Maturity:** This is the final and optimal level on the maturity scale repre-
senting maturity of the fourth industrial revolution. A characteristic is the vision of
optimal IoT-technology use in which all objects are connected to the Internet and
seamlessly integrated, where objects communicate with other objects using common
architectures, interoperability and open standards, enabling human intervention. A
second main characteristic is that the production and warehouse environments are
optimally automated, having manual work only because it is considered most appro-
priate. A third characteristic is that Business Intelligence and continuous improvement
characterizes the organizations. Continuous improvement is enabled by continuous
monitoring of real-time performance discovering design problems that testing failed to reveal. Further, smart factories will emerge, where the new capabilities of smart, connected machines are reshaping operations at manufacturing plants on their own, and where machines are linked together in manufacturing networks. In these smart factories, networked machines automate and optimize production. The key enabler for such a smart environment are seen to be Business Intelligence, which can be described as a set of techniques and tools for transformation of raw data - into meaningful and useful information for the purposes of analysis of business [1]. Thus, at this level, organizations have become predictive, meaning that organizations can forecast what can happen in the future, from the basis of Big Data management. For instance, predictive analytics can identify consumers buying behavior, which organizations can use for marketing trends, as well as production and capacity planning. It is believed that new business processes and models arise, since the smart, connected machines and products create new production requirements and opportunities. The final product assembly might be moved to the customer site, where the final step will be loading and configuring software or the product itself might be delivered as a service.

# 4 Case study of four Norwegian Manufacturing Companies

An in-depth study of four major companies was carried out to develop and refine the model in the development phase, and the final model was used to assess the companies. The four companies assessed in this case study were 1) a furniture manufacturer, 2) an industrial pipe manufacturer, 3) a ship equipment manufacturer and 4) a shipyard. Data was collected through in-depth interviews, meetings and discussion in workshops with the case companies. We found that all companies complied with the characteristics of level 1 maturity, except of company 3 that did not use RFID or barcodes. All companies complied with all requirements associated with level two without exceptions. Regarding level 2, all companies had at least two IoT objects with the ability to communicate vertically, and at least one activity being automated. However, only company 1 and 3 used remotely programming (tablet, smart phones) and access and management of assets or products. Thus, we consider all companies to have score that corresponds to a level 3 maturity. Only company 2 did comply to the requirements at level 4 by having more than two IoT objects with the ability to horizontally communicate. Moreover, this company applies extended automated production.

# 5 Limitations

This study is limited to look specifically at the IoT-technologies, while leaving out the consequences of their use on business process change, smart materials and smart manufacturing, which also are the ingredients of the Internet of Things in general, and complementary approaches like cyber-physical system, future factories, and Industry 4.0. Validation of the model covering its use in practice is ongoing work that requires longitudinal studies of which results will be reported at a later stage.
# 6 Conclusions

In this study, we have developed an IoT Technology Maturity Scorecard, reflecting the evolution of the use of IoT-technologies along a maturity scale with eight levels. It represents a presumed evolution path of the use of IoT Technologies by manufacturing companies. It may serve as a tool for management supporting the adaption of such technologies. The model can be a reference frame to implement an approach for improvements, and assessment of one's own IoT technology maturity level as well as being used in benchmarking against other manufacturing companies. We have demonstrated the IoT Technological maturity scorecard on four globally competitive Norwegian manufacturing companies, covering different manufacturing industries. Our findings show that three of these companies have a score corresponding to level three, "Connected", at the maturity scale, while one of the companies reached level four, "Enhanced", out of eight levels. Hence, these companies have potential to improve their maturity level significantly in their transition toward the fourth industrial revolution. The scorecard provide a useful tool for managers in their efforts in developing their organization in this direction.

**Acknowledgments.** We thank the Manufacturing Network 4.0 project for support of the research presented in this paper.

# References

1. Porter, M.E. and J.E. Heppelmann, *How smart, connected products are transforming companies*. Harvard Business Review, 2015. 93(10): p. 96.

2. Wendler, R., *The maturity of maturity model research: A systematic mapping study*. Information and software technology, 2012. 54(12): p. 1317-1339.

3. Lasrado, L.A., R. Vatrapu, and Andersen, *Maturity Models Development In Is Research: A Literature Review*. IRIS Selected Papers of the Information Systems Research Seminar in Scandinavia, 2015. 6(6).

4. Kühnle, H. and B. G., *Foundations & Principles of Distributed Manufacturing. Elements of Manufacturing Networks*, in *Cyber-Physical Production Systems and Smart Automation*. 2015, Springer Series in Advanced Manufacturing.

5. De Bruin, T., et al., *Understanding the main phases of developing maturity assessment model*, in *Australasian Conference on Information Systems (ACIS), November 30 - December 2 2005, Australia.*, B. Campbell, J. Underwood, and D. Bunker, Editors. 2005: New South Wales, Sydney.

6. Ashton, K., That 'internet of things' thing. RFiD Journa, 2009. 22(7): p. 97-114.

7. Kim, W., *Cloud Computing: Today and Tomorrow*. Journal of Object Technology, 2009. 8(1).

8. Perez Hernandez, M.E. and S. Reiff-Marganiec. *Classifying smart objects using capabilities*. in *SMARTCOMP - 2014 International Conference on Smart Computing*. 2014. IEEE.

9. IEC, *Factory of the Future*, in *White Paper*. 2015.

10. Tan, K.H., et al., *Harvesting big data to enhance supply chain innovation capabilities: An analytic infrastructure based on deduction graph*. International Journal of Production Economics, 2015. 165: p. 223-233.