<figure><img src="image_1.png" /></figure>

HAL
open science

# Mobile Personalised Support in Industrial Environments: Coupling Learning with Context - Aware Features

Nikos Papathanasiou, Dimitris Karampatzakis, Dimitris Koulouriotis, Christos Emmanouilidis

<figure><img src="image_2.png" /></figure>

## To cite this version:

Nikos Papathanasiou, Dimitris Karampatzakis, Dimitris Koulouriotis, Christos Emmanouilidis. Mobile Personalised Support in Industrial Environments: Coupling Learning with Context - Aware Features. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2014, Ajaccio, France. pp.298-306, 10.1007/978-3-662-44739-0_37. hal-01388265

HAL Id: hal-01388265

https://inria.hal.science/hal-01388265

Submitted on 26 Oct 2016

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Mobile Personalised Support in Industrial Environments: Coupling Learning with Context - Aware Features

Nikos Papathanasiou¹,², Dimitris Karampatzakis², Dimitris Koulouriotis¹, and Christos Emmanouilidis²

¹Democritus University of Thrace, Greece
jimk@pme.duth.gr

²ATHENA Research & Innovation Centre, Greece
{npapatha, dkara, chrisem}@ceti.athena-innovation.gr

**Abstract.** The human response time to events in a manufacturing environment depends both on the available skills and competencies of technical staff but also on the extent to which actionable and task-relevant content is readily available when and where is needed. Relevance itself is determined by the task situation context, which in turn is influenced by many factors. This paper presents the development of a context-aware mobile support system for personalised assistance in industrial environments. Combining the individual strengths of learning and content management systems with the ubiquity of delivering relevant content to users carrying NFC (Near Field Communication) enabled mobile devices, the system aims at both enhancing personnel competences as well as their work efficiency. The developed solution is customised to serve an industrial maintenance-support application scenario, wherein the relevant context is determined through location and asset identification, as well as through task and user profiling, offering practical on the spot mobile support.

**Keywords:** NFC, mobile support, e-maintenance, context awareness.

## 1 Introduction

The continuous increase in the inherent complexity of modern industrial production environments exerts greater pressure on personnel, required to operate in a safe and efficient manner to meet overall production and task-specific goals. This is especially the case for personnel involved in maintenance tasks, where high technical skills and competences are sought. Specific sectors, involving safety critical processes, such as aerospace or nuclear industries, have reached a level of maturity which is not tolerant to human errors or unforeseen events. But while safety concerns can be of paramount importance in such cases, a much wider potential impact on improving technical personnel response time in most typical manufacturing sectors has been less well explored [1]. Maintenance staff is required to possess a wide, multi-disciplinary and constantly up-to-date skill set in order to efficiently perform their everyday work tasks. On the individual worker level, a steadily updated set of competences can improve work opportunities in a
global volatile job market. Technology-enhanced solutions should support agile production and work roles so as to meet challenges in a fast changing manufacturing environment [2].

The complexity of modern production lines may be overwhelming for both new employees and older generation ones, thus preventing them from unfolding their true capacities. An obvious improvement would be the provision of on-site support for their work activities, providing the right supporting content to the right personnel at the right time. The appropriate alignment of supporting content and services to the real needs of a specific situation would be a key enabler for making a mobile support solution to be of practical use. This paper presents a context-aware mobile support system, which employs NFC identification and performs basic context data acquisition, such as location, time and user information, as well as a Learning Management System (LMS) for content handling and user roles management. The developed solution provides personalised support to industrial maintenance personnel, thus enabling them to improve their task response time, while also supporting their overall job-related skills development.

The paper is structured as follows. Section 2 outlines the need for providing documentation support in industrial environments, highlighting past efforts and challenges. The importance of involving context identification and context-aware adaptation mechanisms is then discussed and the incorporation of Near Field Communication (NFC) is identified as an indirect method for supporting context identification in industrial settings. Section 3 presents an integrated solution that fuses the benefits of Content Management Systems (CMS) and LMS with NFC-enabled context identification mechanisms, in order to provide fast, intuitive and on the spot support to technical teams, thus supporting their work efficiency. Finally, Section 4 presents the main conclusions of the present work and highlights further research and development directions.

## 2 Contextualised Documentation Support in Industrial Environments

The readily access to technical documentation and practical guidelines is considered critical in order to support maintenance technicians during their job. This information is equally important to be accurate and closely related to the task assigned to technical staff. Employing mobile computers, wireless networks and identification-supporting technologies can facilitate the integration and delivery of task-relevant content on the spot, by applying context adaptation based on different context parameters. A key requirement for this context-driven adaptation is a method to acquire and fuse context information.

The concept of context was introduced in computing to provide an abstraction of the factors that need to be taken into account in delivering content to users, thereby personalising service delivery. An application can be considered as context-aware if it uses context to provide relevant content and services to the user [3]. What is deemed to be relevant depends on several factors, which may be classified in broad categories, such as user, system, environment, service and even social context [4]. In handling context, many issues can be taken into account. A basic context management loop includes acquisition of primary context data, their fusion for context identification or reasoning, and finally the dissemination of the identified context to the respective context-aware applications [5], which are responsible for the contextually-relevant presentation of content and services to
the user. Context data acquisition and processing can be supported by a modelling abstraction, a form of context widget or context engine that persistently collects and fuses context information, in order to disseminate it to any application that has been declared as interested for this context [6]. This continuous data acquisition on a hardware level is usually performed by networked sensors, which collect raw data from the environment and transmit them to the aforementioned widget to extract useful context information. An alternative to this continuous context information acquisition is to trigger this procedure based on user actions. This explicit initiation of context data acquisition has the benefits of lower power consumption and reduced volume of unprocessed context data [7].

Context-aware service delivery presupposes an efficient way of acquiring and understanding context. In the simplest case, context is often linked to localisation and identification, both which can be served by NFC technology. NFC is a wireless communication technology between coupled inductive devices which is regulated by standards such as ISO/IEC 14443 and ISO/IEC 18092. According to these standards, the range of the antenna is short, usually less than 10 cm and the operational frequency is 13.56 MHz. The NFC standardized data rate of 424 kbps is limited compared to other protocols but it is still an efficient means of transferring credentials, short messages or initiating relationships [8]. The format of the NFC exchanged data is the NFC Data Exchange Format (NDEF), which is common to all NFC devices. An NFC message contains one or more records of the following types: simple text, URIs, smart posters or signatures [9]. NFC communicating devices can be active or passive. An active device can act as initiator, whereas a passive device that can emulate a NFC card, using the RF field of an active device to communicate [10]. NFC identification offers a deterministic input modality as opposed to the probabilistic sources such as speech recognition which makes it appropriate for a deterministic and usually time-crucial industrial environment. NFC identification poses little overhead on the power consumption of the mobile device, thus rendering it suitable to be continuously executed. Embedded in low-cost mobile devices, NFC is essentially the evolution of short range RFID technology and has become a de facto standard, facilitating the emergence of new, context-aware applications [11].

Certain industrial sectors with relatively high demands in human technical expertise and relatively complex maintenance procedures were the first which attempted to introduce technology-enhanced support for personnel. In the aerospace sector, the replacement of workcards containing detailed guidelines on how to perform each maintenance task, with computer-based documentation systems was one of the first e-maintenance applications [12]. Part of their initial concerns was to verify the quantitative and qualitative benefits of the new systems. A method to offer digital support material to technical staff operating at the shop floor that has been suggested by many authors is the use of Augmented Reality (AR) to superimpose computer-generated information on top of the real world environment. A major issue with AR is tracking, namely the accurate determination of the user's location, gaze direction, proximity and head pose estimation in reference to the surrounding objects and environment. Two prevalent solutions that have been proposed for this problem are either pre-applied tracking markers, or pre-built CAD models of the machinery positioning. Other methods that can be used for tracking but demand highly specialized hardware are strapped down systems with accelerometers and gyroscopes, ultrasonic pulse acoustic trackers or electromagnetic trackers. AR implementations require complex
equipment and hardware intensive procedures, a substantial amount of preparatory work, requiring previous knowledge of the exact machinery placement [13] and/or sophisticated image processing and rendering capabilities.

While AR usage is well-justified in certain cases, when generic supporting solutions with minimal customisation requirements are sought, there is a considerable potential in investing in the development of technician-centric applications that employ mobile devices and exploit the NFC identification technology coupled with content management offered by a customised Learning Management System, in order to provide on-the-spot context-aware learning and support to maintenance technicians. This work outlines the development of a mobile and context-aware supporting solution that instead of traditional workcards, employs e-support entries that are able to combine different interfacing options offered by modern mobile devices, as tablets, to provide an intuitive and interactive experience. The innovative point of the developed solution is the use of a Learning Management System as the base for the development of a new module that takes advantage of the offered user management and content editing capabilities and combines it with NFC identification for basic level context data acquisition, so as to provide context-adaptive mobile learning and task support. In previous research [14], the structure of the LMS based e-support system was described. The present work focuses on the context-awareness capabilities that are realized by processing stored knowledge and information acquired by NFC identification.

## 3 Building Context-Aware Mobile Support for Asset Management

In order to provide any kind of context-adaptive service it is necessary to receive some input from the environment that when appropriately fused can guide this adaptation. An industrial setting may impose particular limitations in relation to non-industrial environments. For example, GPS is rarely supported inside a building that may be shielded against electrical interference and noise levels may be too high to support voice recognition. Conversely, the developed solution involves specific identification capabilities, either for machine parts or for the involved technicians, combined with an LMS as the base platform for learning and content management (Fig. 1).

<figure><img src="image_4.png" /><figcaption>Fig. 1. Schematic Representation of E-Support Functionality.</figcaption></figure>

A simple format for the stored information is considered sufficiently flexible and easy to process; therefore the selected NFC record type for this implementation is “Simple Text”. The native mobile support application scans this information and parses the data. For
the deployment of the mobile application an Android Nexus device was selected mainly
due to the open-source tools available and NFC identification capabilities. An example of
an NFC tag that is used is “asset:246” where the number 246 refers to the code of an asset.
The information within the tag is transferred through web services to a Learning
Management System on a Web Server that is mainly responsible for:

* Authenticate users and authorize user requests,

* Locate the right support content to send to the mobile device,

* Provide an environment for editing and handling this content.

The selected LMS is Moodle (www.moodle.org), known for customization and
expandability flexibility. Aiming to reduce power consumption on the mobile device, a
lightweight NFC identification component with minimal processing requirements is
developed, instead of a comprehensive context widget. Thus, a large part of reasoning and
processing is being delegated to the corresponding web server component. The latter is a
custom-built Moodle module, responsible for handling content delivery service requests
through web services and storing information derived from the mobile device such as new
photos, voice comments, user bookmarks and interactions. This material can be used by
authorized users to further enrich the system’s content. The support information is
delivered to the technician through intuitive and user-friendly mobile application interfaces,
employing a simple and intuitive interaction model, as shown in Fig. 2.

<figure><img src="image_5.png" /><figcaption>Fig. 2. E-support mobile application and Interaction overview diagram.</figcaption></figure>

The communication between the mobile native application and the Moodle module is
bi-directional, offering a two-fold benefit. First, the technician's feedback, acquired as
annotated photos and voice comments, is stored to the server and can be used to further
enrich the e-support content by the creation of new support entries. Second, the technician's
profile is constantly updated, so as to have an integrated overview of all the tasks that have
been dealt with when relevant support is requested. The coupling of a native mobile e-
support application with a Learning Management System for the handling of all the
recorded support content has some unique benefits. The LMS is literally a fully functional
content management system dedicated to learning, which provides also a full set of user
management capabilities. It is easy and intuitive to consider technicians as trainees with
different levels of access to the learning content and the e-support entries as e-learning objects. These objects follow a predefined structure, enabling easier support information recording and retrieval. The following sequence diagram shows the basic NFC identification, web service requests and content retrieval actions (Fig. 3).

<figure><img src="image_6.png" /><figcaption>Fig. 3. Mobile e-Support with NFC Identification Sequence Diagram.</figcaption></figure>

The structure of the stored e-support data was designed to meet industrial requirements, based both on personal meetings with the industrial project partner, namely a Lifts manufacturing industry (Kleemann Lifts SA, http://www.kleemannlifts.com) and on a subset schema of the MIMOSA standard [15]. The internal requirements set by Moodle for the development of new plug-ins were followed so as to be able to install and communicate with other Moodle components. The structure was completed with the aim to be flexible for future content additions, regarding new technical documentation or input from authorised staff with expertise on a specific type of machinery. A critical design factor was the ability to integrate the mobile e-support system to larger comprehensive e-maintenance systems, providing a wide spectrum of web-service-based interconnectivity capabilities both on the Moodle module and on the native mobile application.

The diagram in Fig. 4 depicts the overall information model. The e-support table stores the e-support component installation data, the user comments table stores information provided by the technicians and the entries table stores all the basic support content, which may be accompanied by relevant files, such as images, pdf files or videos. The FMECA (Failure Modes, Effects and Criticality Analysis) table stores input from an external processing application enhancing the module interconnectivity within an integrated e-maintenance platform [16]. The remaining tables target the categorization and easier extraction of the content. The many-to-many relation between entries and other tables enables content reusability. The actual implementation required some extra tables, ensuring compatible with Moodle component installation and future platform upgrades.
<figure><img src="image_7.png" /><figcaption>Fig. 4. Mobile e-Support Database Schema.</figcaption></figure>

## 4 Conclusions and Further Work

The complexity of modern industrial asset facilities and the rapid adoption of technological changes require from technical personnel to maintain an up to date often multi-disciplinary skills-set. To reduce staff response time to events occurring at the shop floor, methods for the storage, management and delivery of task-relevant support content and services are needed. In this paper, the provision of context-aware maintenance support through mobile devices is presented. The mobile support system utilizes NFC identification to acquire basic location and asset information and employs a customized module within a Learning Management System for user and content management. Information acquired via NFC suffices to cover explicitly three out of five minimum parameters that are necessary to understand context, namely who, where and when [5]. The other two, namely why and what, are derived from the user interaction with the mobile device. The processing of such information along with task profiling are performed by a dedicated LMS-customized module. A joint benefit of the LMS-based content management and the NFC-based context-awareness approach is the overall concealment of the technical communication and context processing details from the end-user, facilitating system acceptance, through the use of simple and user-friendly interfaces. Another benefit is the enrichment of the stored data about learners and their context, which is a key step for the implementation of Learning Analytics as part of further research. A mobile support system that combines an LMS, NFC identification and context-awareness, is an efficient and cost-effective basis to offer on-the-spot maintenance support, assisting maintenance personnel to improve their competencies as well as their work speed and efficiency.

## Acknowledgements

Mobile support requirements and relevant content were offered by Kleemann Lifts and Mr Aggelos Papadopoulos contribution to this end is gratefully acknowledged. The project received financial support through GSRT grant 09SYN-71-856 (WelCOM project).
References

1. Ab-Samat, H., et al., Reduction of response time during machine breakdown: a case study in semiconductor industry. International Journal of Logistics Systems and Management, 2013. 16(2): p. 167-185.

2. Meixner, G., N. Petersen, and H. Koessling, User interaction evolution in the SmartFactory, in Proceedings of the 24th BCS Interaction Specialist Group Conference. 2010, British Computer Society: Dundee, United Kingdom. p. 211-220.

3. Abowd, G., et al., Towards a Better Understanding of Context and Context-Awareness, in Handheld and Ubiquitous Computing, H.-W. Gellersen, Editor. 1999, Springer Berlin Heidelberg. p. 304-307.

4. Emmanouilidis, C., R.-A. Koutsiamanis, and A. Tasidou, Mobile guides: Taxonomy of architectures, context awareness, technologies and applications. Journal of Network and Computer Applications, 2013. 36(1): p. 103-125.

5. Perera, C., et al., Context Aware Computing for The Internet of Things: A Survey. Communications Surveys & Tutorials, IEEE, 2014. 16(1): p. 414-454.

6. Dey, A.K., Understanding and Using Context. Personal Ubiquitous Comput., 2001. 5(1): p. 4-7.

7. Bravo, J., et al. From implicit to touching interaction: RFID and NFC approaches. in Human System Interactions, 2008 Conference on. 2008.

8. Trottmann, U., NFC-Possibilities and Risks. Network, 2013. 35.

9. Igoe, T., D. Coleman, and B. Jepson, Beginning NFC: Near Field Communication with Arduino, Android, and PhoneGap. 2014: O'Reilly Media. 246.

10. Vauclair, M., NFC, in Encyclopedia of Cryptography and Security, H.A. van Tilborg and S. Jajodia, Editors. 2011, Springer US. p. 840-842.

11. Garrido, P.C., et al. A Model for the Development of NFC Context-Awareness Applications on Internet of Things. in Near Field Communication (NFC), 2010 Second International Workshop on. 2010.

12. Drury, C.G., S. C. Patel, and P.V. Prabhu, Relative advantage of portable computer-based workcards for aircraft inspection. International Journal of Industrial Ergonomics, 2000. 26(2): p. 163-176.

13. Zhu, J., S.K. Ong, and A.Y.C. Nee, An authorable context-aware augmented reality system to assist the maintenance technicians. The International Journal of Advanced Manufacturing Technology, 2013. 66(9-12): p. 1699-1714.

14. Papathanasiou, N., et al., Context Aware E-Support in E-Maintenance, in Advances in Production Management Systems. Competitive Manufacturing for Innovative Products and Services, C. Emmanouilidis, M. Taisch, and D. Kiritsis, Editors. 2013, Springer Berlin Heidelberg. p. 574-581.

15. MIMOSA - Machinery Information Management Open Systems Alliance.; Available from: http://www.mimosa.org/.

16. Pistofidis, P., Emmanouilidis, C., Papadopoulos, A., and Botsaris, P.N., (2014), Modeling the Semantics of Failure Context as a means to offer Context-Adaptive Maintenance Support, Second European Conference of the Prognostics and Health Management Society, PHME 2014, 8-10 July 2014, Nantes, France, (PHM Society).