# Virtual Reality for the Training of Operators in Industry
## 4.0

Henrik Schroeder, Axel Friedewald, Chris Kahlefendt, Hermann Lödding

► To cite this version:

Henrik Schroeder, Axel Friedewald, Chris Kahlefendt, Hermann Lödding. Virtual Reality for the Training of Operators in Industry 4.0. IFIP International Conference on Advances in Production Management Systems (APMS), Sep 2017, Hamburg, Germany. pp.330-337, 10.1007/978-3-319-66923-6_39. hal-01666216

HAL Id: hal-01666216

https://inria.hal.science/hal-01666216

Submitted on 18 Dec 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Virtual Reality for the Training of Operators in Industry

## 4.0

Henrik Schroeder¹, Axel Friedewald¹, Chris Kahlefendt¹, Hermann Loedding¹

¹Hamburg University of Technology, Hamburg, Germany

henrik.schroeder@tuhh.de

**Abstract.** The time for maintenance operations is often restricted due to external circumstances. When conducted with inexperienced operators, additional uncertainties can arise and in case of a delay, high follow-up costs emerge. By using Virtual Reality (VR), operators may practice their skills in advance. This paper analyzes the training content and describes which parts of a generic operating cycle have the potential to be supported by VR as training technology. Furthermore, a training procedure and a resulting system architecture that allows to work efficiently with VR as a training technology are presented.

**Keywords:** Virtual Reality · Virtual Technologies · Training · Education · Industry 4.0 · Qualification · Flexibility · Individual Production · Smart Factory · On-the-Job Training

## 1 Introduction

As timeslots for complex maintenance tasks are often restricted, companies need experienced operators to fulfill these essential tasks in a short time. This is among others a characteristic of the machine industry, where operators may service an engine only during the planned maintenance time and in case of a delay, further loss-of-production costs or overtime payments can emerge. To avoid this, service operators usually rely on their experience or use paper based instructions for preparation. A problem with these utilities is that users cannot easily identify complex, three-dimensional subassemblies [1]. Neither can users train their motor skills. To gain experience when executing those operations for the first time or after a long period, a training method is needed, that enhances cognitive skills as well as motor skills – e.g. for movements in cramped areas. The training method shall enable the operator to interact with the prospective work object even without the training object being present, giving him the possibility to train at any place desired.

Virtual Reality (VR) imitates a fictional situation for the user who can manipulate any object with no restriction to weight, size, time, location or with any danger to his well-being. It was already applied in several training showcases, e.g. [2]. Still, up-to-date literature does not sufficiently describe the parts of a task that companies may train with VR. Therefore, the goal of this paper is to identify the benefits VR provides as a
training technology and to reduce the effort of the trainer to prepare a training. Further-
more, this article shall propose an adequate information flow to increase the training
outcome of complex manufacturing tasks.

2 Current State of Research

2.1 Training Content of Complex Manufacturing Tasks

To define suitable training courses, the training content and the target competence lev-
els have to be defined. The training content is mainly determined by the intended out-
come [3] – in this case a complex manufacturing task. These tasks may be described by
the dimensions complexity, connectivity, non-transparency, dynamics and multiple tar-
gets [4]. To define which parts of a task may profit from VR based training, this paper
uses a generic operating cycle developed by Tietze et al. It consists of the five generic
steps information gathering and processing, material and tool allocation, component
and site preparation, execution, feedback and post-processing [5]. These steps may be
split into sub-processes focusing on different capability factors.

As the pure content of a task is not sufficient to clarify the outcome, the target com-
petences have to be specified in a proper way [3]. In the context of maintenance tasks,
especially cognitive and motor skills are relevant. Referring to the oxford dictionary, a
cognitive skill is the ability to acquire and understand knowledge [6] and a motor skill
is the ability to produce the required muscular movements [7]. Anderson et al. [8] pre-
sent a revised taxonomy of Bloom for cognitive skills – from remembering to creating.
Klemme et al. [9] present the taxonomy of Dave for psychomotor skills – from imitating
to naturalization. The different levels of the taxonomy can be measured by specific
indicators concerning potential, activity and output [10].

## 2.2 Training Technology

Many complex manufacturing tasks are usually required to meet certain standards of
time and quality. As described in the introduction, today's training methods prepare the
operator with written instructions and technical drafts. The preparation of these instruc-
tions is often time consuming and it usually takes a long time for the operator to find
and understand the necessary information. Furthermore, written instructions provide no
possibility to train motions for training purposes – not to mention necessary repetitions
as crucial elements of successful trainings [11].

VR is a combination of a computer platform, scene data, an input and an output
device and a VR software [12]. The input device enables the communication between
the user and the software. It may work by a tracking of the user's motions. This com-
bination gives the user the impression to be physically present in the virtual environ-
ment [12]. It can be realized among others in form of a VR Cave or with VR glasses.
As it is a promising technology to solve the problems mentioned above, several authors
describe VR based training scenarios – e.g. [13, 14]. Vaughan et al. [2] even present
existing VR training systems used in five different sectors – medicine, industry and
commerce, collaboration, serious games and rehabilitation. Their paper stresses the
positive effects and the practicability of each concept. Still, none of the papers considers
the implicit knowledge of experienced operators in the training. Rather, the authors
focus on explicit knowledge – e.g. written instructions. Furthermore, the concepts do
not consider the potential to partially compensate physical training with a tracking of
motions in VR – only cognitive skills are under examination.

## 2.3 Training Procedure

Simplified, training processes can be divided into preparation, execution and evaluation (post-processing) [15]. In this context, the preparation mainly consists of the definition of the target competences, the definition of the resulting training technology and the authoring of a proper training scenario. For the execution and the evaluation, Haase et al. [13] propose four different strategies for VR-training: The discovery mode (no guidance, no evaluation), the presentation mode (full guidance, full evaluation), the guided mode (defined guidance and evaluation) and the free mode (no guidance, final evaluation). Even though Haase et al. explicitly exclude the training of motor skills, no restrictions disable the use of their training modes for this concept.

# 3 Combining Work Tasks and Training Aspects

This chapter connects a complex work tasks with the relevant aspects of a VR-based
training. In this paper, a maintenance scenario of an engine is selected. Particularly, the
operator shall clean the charge air cooler with a diameter of approx. 2 meters weighing
about 450 kg. To handle the components the operator has to apply additional structures
and a crane in non-ergonomically positions.

1. Information Gathering and Processing: The operator reads paper instructions for preparation. Thereby, he pictures the correct motions. Once on site, the operator compares the instructions, his perception and reality. He may repeat this, whenever a new part comes into sight during the disassembly process. This step contains only cognitive skills as knowledge and imagination. No motions are yet required. As the instructions provide most of the information necessary, the operator has to reach the level of applying for cognitive skills in terms of Blooms taxonomy.

2. Material and Tool Allocation: Prior to the planned maintenance stop, the operator chooses the required material and his tools and places them in a suitable manner. Therefore, this step requires cognitive skills in form of knowledge of the correct tools up to the level of applying. This is not necessarily described by the instructions. The motor skills of gathering the material can be neglected as this paper assumes that operators already have the motor skills for any sub-task of this step.

3. Component and Site Preparation: Again, prior to the planned maintenance stop, the operator gathers the necessary components and places them in a suitable manner. This step includes the correct use of the crane in a cramped area. Therefore, this step mainly consists of cognitive skills of knowing where to find the components up to the level of applying and motor skills for the correct use of the crane up to the level of manipulation.
4. Execution: The operator fulfills the tasks in accordance to his perception of the in-
structions. On demand, the operator goes back to the paper instructions and gathers
further information. Usually, the operator is under time pressure during this step. At
large, this step requires cognitive skills for applying the gathered information and
motor skills for the correct execution up to the level of manipulation.

5. Feedback and Post-Processing: When production is re-initiated, the operator gathers the tools and components, cleans the workspace and documents his actions. Hence, only cognitive skills up to the level of applying are required.

At large, the training content requires the training technology to train cognitive skills up to the level of applying and motor skills up to the level of manipulation. While the cognitive skills of course can be taught in principle in advance via paper instructions, videos, 3D-PDFs and VR, the motor skills need physical practice. For evaluating to what degree VR can compensate the physical training, this paper provides the setup of a general training concept.

Generally, this concept shall provide benefits in terms of a higher profitability for
the training as well as a higher quality and a lower execution time for the actual task.
Thereby, the concept has to fulfill certain requirements. Contrary to the preparation of
a training scenario, different operators with diverse skills perform the training. While
the degree of difficulty is already defined by the task, the training software has to in-
clude different levels of assistance to meet the diverse requirements of the operators.
Furthermore, as the time for the execution of the training is limited, the operator needs
his feedback as soon as possible. When conducted with human aid, additional waiting
times would have to be accepted. Therefore, at best the software shall conduct a fully
automated evaluation based on the individual competence level immediately.

4 Implementing Training Procedures in Virtual Reality

**4.1 Fundamental Concept**

As three persons participate in the training process – the trainer as the editor of the training scenario, the student as the executor and the trainer as the final evaluator – the software shall provide the necessary views for each role. These views also refer to the simplified training process – one to support the preparation, one the execution and one the evaluation (Fehler! Verweisquelle konnte nicht gefunden werden.).

The first part is the preparation of the training. Therefore, the trainer uses the editor view and the function *Editing* to issue the *Template*. It includes the written instructions in form of steps in the VR-environment as the explicit knowledge. Then, a skilled person executes the steps in the VR-environment using the function *Recording*. This may be the trainer or an experienced operator. The function records and saves the actions in the *Solution* – in an unpretentious manner, this represents the implicit knowledge.

Second part is the execution of the training by the student. On the one-hand-side, he
has to follow the instructions of the maintenance plan, which are generally not highly
detailed and on the other hand has to perform a specific motion sequence. While the
student follows the same instructions as the experienced operator before, the software records and saves the students actions. Thus, he creates the *Result*.

Finally, the function *Evaluating* derives any discrepancy between the experienced operator and the student. The trainer may define minimum requirements that the student has to accomplish in order to pass. The software then creates the *Evaluation* containing the evaluated results of the student and the grade. Fig. 1 summarizes this information flow and the according system architecture.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Information Flow and System Architecture</figcaption></figure>

As the implementation of a training – especially with VR – involves additional costs for hard- and software as well as the additional time for the training, the following chapters describe the setup and the expected benefits of each phase individually.

## 4.2 Preparation of the Training Content

Purpose of this phase is to enable the trainer to include both explicit and implicit knowledge and at the same time keep the necessary effort to a minimum. As stated in the previous chapter, the trainer includes the written instruction in a suitable format for the software. Then, he executes these instructions in VR. The software shall include the motions of the operator and thus allow the combination of explicit knowledge of the instructions and the implicit knowledge of the trainer in an unpretentious manner. There are two options to record the motions: (a) to develop standardized templates, adapt them to any new situation and compare their counterparts for evaluation (b) to divide any task into the smallest steps possible and compare them for evaluation. Option (a) results in a high amount of work, as the templates have to cover any of the existing situations. In addition, a VR software specialist must carry out this work instructed by the trainer. Option (b) results in less work, as the specialist must not create a new template for any given situation. It is therefore the favored option.
Current instructions require the trainer to transfer the 3D-file to a technical 2D-draft.
A positive consequence of the concept is that this part vanishes as the transmission from
a 3D-file to VR is completely automated. On the negative side is the trainer who spends
additional time as he carries out the instructions for the *Solution*.

4.3 Execution of the Training

Purpose of the execution is to teach the student all necessary skills defined by the
trainer. He therefore executes the written instructions the same way the trainer did in
the phase of preparation. As more or less experienced operators may participate in the
training during this phase, the execution needs to provide different levels of assistance.
The trainer may realize this either via (a) an adaption of the task or (b) an adaption of
the aids. Option (a) results in a new training scenario as described in the previous chap-
ter. When choosing option (b), the trainer may disable hints or instructions for the steps.
The more experienced an operator becomes, the fewer hints are included by the trainer
and thus, the more difficult a training becomes until the operator has gained the desired
competences.

Trainings are more likely remembered when the student is involved rather than when he only observes a visual presentation, e.g. written instructions or 3D-PDFs [16]. Thus, using this concept has the potential to improve trainings and therefore reduces mistakes and working time of the work task. In addition, VR has the potential to partially compensate physical training by tracking the user's motions and therefore support the training of the required motor skills. This potential is not yet quantified.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Remote Control and Virtual Reality</figcaption></figure>
For a first proof of concept, the developed prototype uses the application programming interface of the software VDP by the ESI group. The developed module receives and saves the virtual coordinates of any object from the VDP to create the *Solution* and the *Result*. Next, the module compares the student's coordinates with the ones of the *Solution* and derives consequences. For a better comfort, the user may control the situation via smartphone or tablet within the immersive area of the Cave (Fig. 2).

## 4.4 Evaluation of the Training Results

The last phase is the measurement of the acquired competences. Therefore, the software describes and detects errors, rates solutions and determines the new competence level of the student.

In order to describe and detect errors, general criteria for the evaluation are defined. These are the time and the number of mistakes [17]. For an adequate feedback, it is necessary to further define the mistakes. For that, Berndt et al. propose the presence, the position and orientation of each part and the completeness of each working step [18]. These measurable criteria can then be connected to the acquired competences [10]. The software has knowledge of the coordinates of any object in each step. Thus, it can calculate the differences of the *Result* and the *Solution* for each criterion and therefore detect errors.

The function *Evaluating* rates the differences of each step. This is based on the calculated differences and on the defined tolerances that the trainer may define for each criterion, each step and for each operator individually— e.g. an object shall be placed within a diameter of 5cm. In practice, the developed prototype generates the *Evaluation* automatically. As no human actions are required, no additional waiting time or effort has to be considered. The student can repeat the training with a full understanding of his previous mistakes and therefore improve his capabilities.

Finally, the software derives the acquired competence level based on the defined tolerances and on the errors of the student – e.g. the student is capable to perform a task in a given time. This is based on rules defined by the trainer for passing a new competence level – e.g. only a certain kind of error is detected in the training procedure.

# 5 Conclusion and Future Research

This paper outlines a concept for a VR-based training of a complex maintenance task. Therefore, it analyzes the generic operating cycle and determines the trainable parts for VR. These are especially cognitive and motor skills up to the level of applying respectively manipulation. Subsequently, the paper derives a concept with the three training phases preparation, execution and evaluation. It describes the resulting information flow and the expected benefits of each phase. At last, it proves the feasibility of the concept by presenting a basic prototype.

As this is only an early approach towards a comprehensive training concept, future research will cover a more detailed view on the generic operating cycle, the critical parts of the work task and the compensation of physical training with VR. In addition, it will extend the current results towards VR glasses and Augmented Reality.
## 6 References

1. Halata PS, Friedewald A, Hillmer A (2015) Augmented_Reality-gestützte Arbeitsunterlagen für die Unikatfertigung. Go3D

2. Vaughan N, Gabrys B, Dubey VN (2016) An overview of self-adaptive technologies within virtual reality training. Computer Science Review 22: S. 65–87

3. Biggs JB, Tang C (2009) Teaching for quality learning at university: What the student does, 3. ed. McGraw-Hill education. McGraw-Hill, Maidenhead

4. Dörner D, Kreuzig HW (1983) Problemlösefähigkeit und Intelligenz. Psychologische Rundschau (34): S. 185-192

5. Tietze F, Lödding H (2014) Analyse der Arbeitsproduktivität in der Unikatfertigung: Eine Grundlage für zielorientierte Verbesserungsprozesse in der Unikatfertigung. Industrie Management: S. 62-66

6. Oxford Dictionary (2017) Definition of cognition in English. https://en.oxforddictionaries.com/definition/us/cognition

7. Oxford Dictionary (2017) Definition of motor in English. https://en.oxforddictionaries.com/definition/motor

8. Anderson LW (ed) (2009) A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives, Abridged ed.. Longman, New York

9. Klemme B, Andres J (eds) (2012) Lehren und Lernen in der Physiotherapie: 41 Tabellen. Physiofachbuch. Thieme, Stuttgart

10. Nedeß C, Friedewald A, Kanieß U et al. (2007) Reporting Strategies for a Competence-oriented Management Information System. 13th International Conference on Computer Applications in Shipbuilding (Papers Vol. I): S. 61-69

11. Green N, Green K (2012) Kooperatives Lernen im Klassenraum und im Kollegium: Das Trainingsbuch, 7. Auflage. Klett Kallmeyer, Seelze

12. Runde C (2007) Konzeption und Einführung von virtueller Realität als Komponente der digitalen Fabrik in Industrieunternehmen. IPA-IAO Forschung und Praxis, vol 455. Jost-Jetter, Heimsheim, Stuttgart

13. Haase T, Winter M, Bluemel E (2009) VR-basierte Qualifizierung technischer Fachkräfte im industriellen Einsatz. In: Lukas U von (ed) Go-3D 2009: Go for Innovations: Tagungsband der Konferenz Go-3D 2009, Rostock, 31. August 2009. Fraunhofer-Verl., Stuttgart, S. 73-86

14. Ordaz N, Romero D, Gorecky D et al. (2015) Serious Games and Virtual Simulator for Automotive Manufacturing Education & Training. Procedia Computer Science 75: S. 267-274

15. Merkens H (2010) Unterricht: Eine Einführung, 1. Aufl. VS Verl. für Sozialwiss, Wiesbaden

16. Dale E (1969) Audiovisual Methods in Teaching: 3rd ed. Dryden Press and Holt, Rinehart, and Winston, New York

17. Baren Jv, IJsselsteijn W (2004) Presence Measurement Compendium

18. Berndt D, Sauer S (2012) Visuelle Assistenzsysteme in der Montage verhindern Ausfälle. http://www.maschinenmarkt.vogel.de/visuelle-assistenzsysteme-in-der-montage-verhindern-ausfaelle-a-360481/index3.html