# Performance Measurement in Sensorized Sociotechnical Manufacturing Environments

Emrah Arica, Manuel Oliveira, Christos Emmanouilidis

► To cite this version:

Emrah Arica, Manuel Oliveira, Christos Emmanouilidis. Performance Measurement in Sensorized Sociotechnical Manufacturing Environments. IFIP International Conference on Advances in Production Management Systems (APMS), Aug 2018, Seoul, South Korea. pp.263-268, 10.1007/978-3-319-99707-0_33. hal-02177859

HAL Id: hal-02177859

https://inria.hal.science/hal-02177859

Submitted on 9 Jul 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Performance Measurement in Sensorized Sociotechnical
Manufacturing Environments

Emrah Arica<fnref n="1" />, Manuel Oliveira<fnref n="1" /> and Christos Emmanouilidis<fnref n="2" />

**Abstract.** Industry 4.0 entails the digitization of the shopfloor operations combining technologies such as internet of things-enabled sensing, cyber-physical systems, data analytics, augmented reality, and wearable devices and robots that transform the manufacturing environment into a workplace of human-machine interactive symbiosis. With the digitization of the manufacturing environment, new opportunities emerge concerning performance measurement as new sources of real-time data become available, including data collated from the operator on the shopfloor. Traditionally, the human dimension had been disjoint from the situation analysis of shopfloor performance that drives evidence based decision making. This paper presents the features and advantages of performance measurement in human-workplace interactive manufacturing where detailed data on human performance is provided by sensors and utilized to improve the performance goals. The paper is concluded with a discussion on the impact of context information management for interactive manufacturing workplaces, as a means of delivering more informed situational awareness, a critical enabler for human-machine interaction, as well as for handling complexity in disparate data sources.

**Keywords:** Performance Measurement, Manufacturing, Human-Machine Interaction

## 1 Introduction

Performance measurement contributes to identify the deviation from the targets, provide mechanisms for both top-down and bottom-up communication, and assess the competitive priorities and operational capacity of the firm [1]. Achieving these functions largely rely on capturing timely and accurate status information from the manufacturing environment composed of a complex ecosystem of processes, workers, machines, tools, robots, information systems and technologies. Such complexity also involves uncertainty and flexibility in the manufacturing processes, as well as a degree of decision-making autonomy on the shop floor, typified as sociotechnical manufacturing environments [2, 3]. In today's industry 4.0 sociotechnical manufacturing environments, a great deal of real time data can now be captured from machines and processes by information systems and technologies such as sensors. Sensorizing the workers is

<footnote n="1">¹ SINTEF Technology and Society, Department of Economics and Technology Management,
Trondheim, Norway</footnote>

<footnote n="2">² Cranfield University, School of Aerospace, Transport and Manufacturing, Cranfield, UK
{emrah.arica | manuel.oliveira}@sintef.no
christosem@cranfield.ac.uk</footnote>
far behind this digitalization pace due to various reasons, such as lack of focus on hu-
man factor in performance measurement systems and privacy concerns. However, re-
search indicates that human factor is an important mediator to achieve good perfor-
mance results [4, 5]. As such, the correlation between the human performance and other
performance dimensions should be better understood. Further research is therefore
needed to study how workplaces with sensorized workers capturing detailed real time
human data can be incorporated into performance measurement systems.

This paper proposes a conceptual framework for sensorized manufacturing environ-
ment and discusses its impact on performance measurement. A literature review is con-
ducted, highlighting the scarcity of resources on the influence of industry 4.0 technol-
ogies on performance measurement. To the best of our knowledge, the potential per-
formance impact of sensorizing humans has not been sufficiently discussed previously.
Furthermore, the paper proposes a framework that can be utilized to develop adaptable
and interactive workplaces with enhanced efficiency and worker well-being.

## 2 Manufacturing Performance Measurement

A manufacturing performance measurement system traditionally focuses on flow ori-
entation with short throughput times and maximizing the internal efficiency in terms of
both financial (e.g. cost) and non-financial measures (e.g. labor productivity) [6].
Alignment of financial and non-financial measures has gained importance with the in-
troduction of balanced scorecard [7] and its derivations [8]. Many authors argue that
non-financial performance measures have indirect but important impact on financial
performance. One way of realizing the positive influence of non-financial measures on
financial performance is through lean manufacturing practices [4]. This necessitates the
incorporation of human factor into performance measurement frameworks and consid-
eration of human resource-oriented metrics. However, traditional manufacturing per-
formance measurement systems are criticized as they largely focus on technical
measures and the human factor is largely ignored [9]. The pioneering empirical study
of [5] on 163 plants indicates the necessity of joint optimization of both socially- and
technically-oriented practices for achieving good performance results. Tracking, meas-
uring, and improving human-oriented performance dimensions such as employee in-
volvement, training, and information and feedback are closely linked with success in
overall efficiency measures (e.g. productivity) and financial performance [5, 10]. Elim-
inating production wastes alone does not improve productivity, quality, or delivery per-
formance, but reveals them [4]. There is need for involving shop-floor employees
through information and actionable feedback to address the root causes of problems.
As such, sociotechnical manufacturing environments require the convergence of social
(e.g. reduced worker accidents, increased skills) and technical (e.g. increased produc-
tivity and quality) performance dimensions and objectives for optimized and improved
company performance.

Research shows the positive correlation of employing enterprise information sys-
tems in manufacturing with shop floor performance dimensions [11]. Although manu-
facturing technologies and information systems have been utilized to manage and im-
prove performance, there are still challenges in timely information acquisition and
timely feedback [11]. The recent applications of Internet of Things (IoT) technologies (e.g. radio frequency Identification tags, embedded computing devices and sensors, actuators and mobile devices), along with IoT-oriented computing architectures are providing enablers for addressing such challenges. They are not only enabling more adaptive and reactive performance measurement systems, but also lead to the evolution of performance measurement systems by enabling timely information availability and accessibility. Relatively few recent studies focus on the latter aspect to develop smart performance measurement systems. Kumaraguru et al. [12] proposes a continuous performance measurement system integrated with real-time analytics. Hwang et al. [13] suggested an IoT-based performance management model that define the performance indicators consistent with the ISA95 automation standard, using manufacturing information systems such as MES. However, these studies are at the very early stages and consider the use of IoT technologies merely on objects and locations such as machines, production lines, and inventory areas. The focus is mainly on machine-oriented overall efficiency measures such as OEE, and on minimizing the deviation between targeted and actual performance. The impact of human performance and the interaction between different measures is not considered. The next section presents a performance measurement framework for sensorized manufacturing environment, where real time data can be captured both from humans and machines, enabling involvement of human factor in performance management frameworks.

## 3 Performance Measurement in Manufacturing Environments with Sensorized Workers

In Industry 4.0 manufacturing environments, the human worker can be directly senso-
rized by wearable devices, yielding more data in addition to what is accessible by means
of information systems that support processes, machines, and robots. This will permit
to assess the worker’s state across multiple dimensions, including satisfaction, engage-
ment, stress level, capabilities, skill, performance and experience. Motion sensors can
detect movement and gestures that contribute to the definition of the psycho-physio-
logical status. Through wearable devices, workers receive the support to bridge cogni-
tive gaps they are experiencing. Examples of captured parameters are: breath/heart rate,
skin conductance and movement of relevant body segments. From the analysis of the
signals representing the multiple dimensions, it can be decided if an immediate action
should be taken or if the detected conditions are not so serious as to require a long-term
re-design of the workplace to improve the interaction between workers and the produc-
tion system. This can help detecting when deviations occur that require interventions
for physical and cognitive enhancements, making sense regarding the context if the
underlying circumstances and can lead to improving operator performance and wellbe-
ing. Furthermore, the symbiosis between workers and workplaces will be facilitated,
making the interaction between the two comfortable and efficient. A conceptual frame-
work for such manufacturing environment is depicted in Figure 1 and described below.
<figure><img src="image_2.png" /><figcaption>Fig. 1. A Conceptual framework for sensorized manufacturing environment</figcaption></figure>

The process starts with the capture of signals from different sources, including the operator on the shopfloor. The collated signals are used to create in the system a situation awareness of the work being carried out by the worker by means of digital representation of reality. The system then uses machine learning (eg: case based reasoning or similar approaches) to detect when deviations from the norm, including potential safety and ergonomics problems or cognitive deficits (e.g. lack of attention, tiredness, reduction of productivity) of workers are detected. The system triggers potential short-term interventions that address the nature of a deviation. Two examples are shown:

* An exoskeleton actuator is used to alleviate the physical stress and thereby mitigate the tiredness of the worker of handling heavy equipment or/and parts;

* An augmented reality actuator is triggered when signals indicate a decline in their attention, thus the system will reinforce with visual cues their focus of attention.

The interventions triggered are very context dependent, thus when the system is unable to find a resolution to an identified deviation with an existing intervention (eg: exoskeleton or augmented reality), the system registers the failure and triggers a long term intervention where a human stakeholder (eg: engineer) is required to use a set of tools for mining the data collated to determine the root cause of the deviation and devise a new short-term intervention, alternatively they may redesign the product, process or workplace. Such human-workplace interactive sensorized manufacturing environment are suggested to bring the following performance measurement system improvements:

*Successful utilization of proactive leading performance indicators:* The quality of commonly-used indicators (e.g. number of accidents) is questioned since they are historical in nature, reporting on activities that occurred in the past [9]. Such indicators are classified as lagging indicators in literature [14]. Lagging indicators are output oriented and aim to provide a snapshot of analysis of the historical data. Leading indicators aim to monitor if the tasks are being performed on track and to provide early warning signs for taking a proactive action to avoid unwanted events from occurring. An example of a leading indicator is the frequency of stress peaks of the workers, which provides a warning for a future work accident. A proactive action in this case could be taking a
break. While leading indicators enable taking corrective actions towards performance objectives, lagging indicators primarily reveals the performance results.

*Better understanding of causal relationships between measures:* There is highly limited possibility to understand the cause-effect relationships of measures in today's performance measurement systems. This is usually done with trial and error approach, with lack of evidences. Capturing detailed real-time performance information from workers can also provide evidence-based links between various performance dimensions such as health, productivity, and quality. For example, it can now be better understood how the stress and fatigue levels of the workers correlates with sickness and absenteeism. How does the stress level influence the productivity? Is the worker achieving the daily quote at the cost of his/her health? How is the level of stress the person is feeling when getting the daily productivity quote? How is the quality influenced by the stress levels? Better understanding of such causal relationships also enables better alignment of different performance dimensions and better decisions. For example, reducing the daily productivity quote to reduce the absenteeism can on the long run increase productivity.

# 4 Discussion and Conclusion

This paper focused on the potential impact of employing industry 4.0 technologies, with emphasis on sensors and wearables, on performance measurement. Considerable challenges need to be addressed when introducing sensorized solutions for human performance measurement. First, situational awareness needs to be established, so that measurements are analysed, and recommendations or action triggers are initiated on the basis of the right context. Secondly, the complexity and disparity of data sources need to be managed, filtering out irrelevant data, while reinforcing attention via cues which are contextually relevant. Finally, concerns regarding the privacy and security of data related to human activity must be addressed. The latter should be considered at the design stage and relevant guidelines need to be followed to address privacy issues of human-centered manufacturing environments for all phases of data lifecycle management (Mannhardt et al. [15]). The first two challenges require further research on context information management, which is now recognized as of high importance in modern enterprise information systems and as such it is also relevant to the assessment of human performance in workplaces [16]. The typology of high level context identified in such information systems, namely user, environment, system, social, and business/service context is also relevant for human performance measurement. Further work is nonetheless needed to model this type of application-specific context with a particular emphasis on ‘User Context’ to capture key elements of human physical and sentiment situational awareness. For example, detecting worker stress during work activities has different contextual interpretation depending on the circumstances under which the worker operates. The context of operating under heavier than usual workload and tight constraints is quite different from that of a worker operating under low work load conditions and therefore any stress detection should be interpreted differently. Such analysis should then provide further guidance not also regarding the sensorization choices
but also the sensor and context fusion approaches which would be applicable to perfor-
mance measurement in sensorized sociotechnical manufacturing environments.

References

1. Nuthall, L.: Supply chain performance measures and systems. Gower handbook of supply chain management. Pp. 248-66, Gower Publishing, Burlington, VT (2003).

2. Wiers, V.C.S.: The relationship between shop floor autonomy and APS implementation success: evidence from two cases. Production Planning & Control 20(7), 576-585 (2009).

3. Arica, E., Haskins, C. and Strandhagen, J.O.: A framework for production rescheduling in sociotechnical manufacturing environments. Production Planning & Control, 27(14), 1191-1205 (2016).

4. Fullerton, R.R., Wempe, W.F.: Lean manufacturing, non-financial performance measures, and financial performance. International Journal of Operations & Production Management 29(3), 214-240 (2009).

5. Cua, K.O., McKone, K.E. and Schroeder, R.G.: Relationships between implementation of TQM, JIT, and TPM and manufacturing performance. Journal of operations management, 19(6), 675-694 (2001).

6. Jonsson, P., M. Lesshammar: Evaluation and improvement of manufacturing performance measurement systems-the role of OEE. International Journal of Operations & Production Management 19(1), 55-78 (1999).

7. Kaplan, R.S., Norton, D.P.: Using the balanced scorecard as a strategic management system, Harvard business review, Boston (1996).

8. Hoque, Z.: 20 years of studies on the balanced scorecard: trends, accomplishments, gaps and opportunities for future research. The British accounting review 46(1), 33-59 (2014).

9. Gomes, C.F., Yasin, M.M., and Lisboa, J.V.: A literature review of manufacturing performance measures and measurement in an organizational context: a framework and direction for future research. Journal of Manufacturing technology management 15(6), 511-530 (2004).

10. Birdi, K., et al.: The impact of human resource and operational management practices on company productivity: A longitudinal study. Personnel Psychology 61(3), 467-501 (2008).

11. Abdel-Maksoud, A., D. Dugdale, and R. Luther: Non-financial performance measurement in manufacturing companies. The British Accounting Review 37(3), 261-297 (2005).

12. Kumaraguru, S., Kulvatunyou, B., and Morris, K. C. Integrating Real-Time Analytics and Continuous Performance Management in Smart Manufacturing Systems. In Advances in Production Management Systems. Innovative and Knowledge-Based Production Management in a Global-Local World, 175-182. Springer Berlin Heidelberg (2014).

13. Hwang, G., et al.: Developing performance measurement system for Internet of Things and smart factory environment. International Journal of Production Research 55(9), 2590-2602 (2017).

14. Muchiri, P., et al.: Development of maintenance function performance measurement framework and indicators. International Journal of Production Economics 131(1), 295-302 (2011).

15. Mannhardt, F., Petersen, S. A., Duarte de Oliveira, M. F.: Privacy Challenges for Process Mining in Human-centered Industrial Environments. In Intelligent Environments (IE) (2018).

16. El Kadiri, S., Grabot, B., Thoben, K. D., Hribernik, K., Emmanouilidis, C., Von Cieminski, G., & Kiritsis, D.: Current trends on ICT technologies for enterprise information systems. Computers in Industry 79, 14-33 (2016).