# Improving the Safety of Using Didactic Setups by
## Applying Augmented Reality

Srdjan Tegeltija, Vule Reljić, Ivana Šenk, Laslo Tarjan, Branislav Tejić

► To cite this version:

Srdjan Tegeltija, Vule Reljić, Ivana Šenk, Laslo Tarjan, Branislav Tejić. Improving the Safety of Using Didactic Setups by Applying Augmented Reality. IFIP International Conference on Advances in Production Management Systems (APMS), Aug 2020, Novi Sad, Serbia. pp.11-18, 10.1007/978-3-030-57997-5_2. hal-03635688

HAL Id: hal-03635688

https://inria.hal.science/hal-03635688

Submitted on 8 Apr 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Improving the Safety of Using Didactic Setups by
## Applying Augmented Reality

Srdjan Tegeltija¹ [0000-0002-9307-445X]✉, Vule Reljić¹[0000-0001-6496-5651], Ivana Šenk¹[0000-0002-8296-7885], Laslo Tarjan¹[0000-0001-9731-1304], Branislav Tejić¹[0000-0003-4088-6136]

¹ University of Novi Sad, Faculty of Technical Sciences, 21000 Novi Sad, Serbia

srkit@uns.ac.rs

**Abstract.** The application of didactic setups is one of the commonly used teaching methods at schools and universities nowadays. By using the didactic setups, students are introduced to real components and real processes that take place in the industry. However, the students often need to use the didactic setups outside of regular teaching activities and at times when teaching staff is unavailable. This paper presents a solution for improving the safety of using the didactic setups by applying an augmented reality solution. The presented solution was implemented on a didactic setup for control of a three-phase asynchronous motor. In addition to improving the safety, the proposed solution allows the students to autonomously use the didactic setups beyond regular teaching hours.

**Keywords:** didactic setup, remote control, augmented reality.

## 1 Introduction

In order to provide training for the use of various equipment, there are numerous different solutions aiming to provide simple, fast and safe training for people. One of the possible ways is that the trainees learn solely from literature [1, 2]. In such a setting, the users are introduced to the equipment and its use through printed literature (textbooks, books, operating instructions, etc.) or through literature in a digital format (PDF, audio and video, etc). By following the descriptions and instructions, the users are introduced to the equipment and its operation. The obvious advantage of this type of training is simplicity. However, its main disadvantage is that it is completely theoretical, and lacks hands-on trainings which are essential for real acquisition of knowledge and skills. After such a training, the trainees need some time to become familiar with the real equipment, which might additionally pose a potential safety risk. Another disadvantage of this type of training is that certain instructions may be difficult to read and incomprehensible to the trainees without experimenting with the actual equipment. Also, there may be errors in the text that may lead the users to the wrong conclusions, which also poses a safety risk (for example, electric shock, injuries due to contact with actuators, etc). Another training possibility is to apply didactic
setups [3–5]. The didactic setups serve as system models with actual parts of equip-
ment, which are used to simulate the systems and processes that are executed in real
industrial and non-industrial systems under controlled and laboratory conditions. The
main advantage of this type of training is the hands-on experience of working with
real components of the system. The didactic setups are often created by the producers
of the equipment to make it easier and faster for the users to familiarize themselves
with the equipment, or by companies in order to train new employees as quickly as
possible. After such a training, the trainees can apply the acquired knowledge directly
in real systems because they have the hands-on experience with working with real
components during the training. A disadvantage of this type of training is that there
are some, although minimal, safety risks. Even though the didactic setups simulate
the operating processes in the controlled laboratory conditions, there is a possibility of
electric shock if the trainees would accidentally touch metal parts which could get
connected to the power supply, as well as injuries from physical contact with actua-
tors, or burns from handling high temperature fluids (hot water or steam), etc. Con-
sequently, in order to minimize the safety risks, at least one responsible person must
be present to supervise the training process (instructor, laboratory assistant, etc).
Thus, the availability of didactic setups for the users depends on the availability of the
responsible personel for supervising the training as well as on the availability of the
laboratories in which the didactic setups are located. Further disadvantage of this type
of training is that didactic setups cannot be used by a large number of users at the
same time, i.e. there is a limit on the number of the trainees per training. In recent
years, following the development and popularization of virtual and augmented reality
technologies, training systems which utilize these technologies have been developped
[6–8]. The advantage of this type of training is that, similarly to hands-on trainings,
the users are introduced to a model of the system with real system components, but in
a virtual environment, so there is no possibility of physical injury. In addition, a large
number of trainees can attend the training at the same time, and there is a possibility
of conducting the trainings from remote locations. The disadvantage of this type of
training is that the virtual reality systems can cause side effects on the user such as a
headache, blurred vision and nausea, so planned breaks are recommended during the
training at regular time intervals. Another disadvantage of this type of training is that
it employs emerging technologies whose cost is still relatively high, and many organi-
zations may not have the resources to make the initial investment, although the costs
are usually justified in the long run. To overcome the disadvantages of the individual
training types, the trainings for users who will work in more complex industrial and
non-industrial environments often utilize a combination of multiple training methods.

Educational institutions such as schools and universities commonly use dedicated
didactic setups as a training method in order to provide hands-on experience to the
students. By using the didactic setups, students are introduced to real components as
well as to the models of real systems and processes under controlled laboratory condi-
tions. A disadvantage of using such didactic setups at schools and universities is that
they are available to pupils and students only during regular teaching activities with
the presence of teaching staff (professors, assistants, laboratory assistants). However,
the students often need to use the didactic setups outside of regular teaching hours,
including times when the teaching staff is not available for them. This paper presents a concept for a solution that increases safety of using the didactic setups as well as enables autonomous use of didactic models by pupils and students beyond regular teaching hours by applying augmented reality technology.

## 2 The Proposed Solution

The proposed solution is based on a combination of using didactic setups and the augmented reality concept. The augmented reality is a concept that expands a user's view of the world with computer-generated text, images and sound. The idea behind the augmented reality concept is not new. The concept was created in the 1960s, and its foundations were laid by Ivan Sutherland through the development of a system with a head mounted display which was used to show a simple wireframe cube overlaid onto the real world [9]. This technology has long been neglected due to the limitations of hardware and software resources (large computers with low performances, cameras and displays with large dimensions but low resolutions, etc.) [4]. With the emergence and widespread use of smartphones with built-in high-speed processors, high-resolution displays and cameras, the augmented reality concept has gained in popularity and became implemented in a number of fields, such as industry, medicine, education, and other areas [10–17]. In the meantime, a variety of glasses that allow augmented reality to be displayed have been developed [18]. All this leads to new possibilities of implementing the augmented reality concept in various areas.

In this paper we propose a solution for using the augmented reality concept in order to enhance the existing didactic setups used for teaching students through hands-on experiences with a model of a system that includes real industrial equipment. The proposed solution is based on using a server application, a smartphone with a dedicated user application providing the augmented reality features and the remotely controlled didactic setup. The concept diagram of the proposed solution is shown in Fig. 1.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** The concept diagram of the proposed solution for improving the safety of using the didactic setups by applying augmented reality: server, smartphone and the didactic setup.</figcaption></figure>
The server application is designed to store the user profiles and the didactic setup data. When creating the user profile, each user is assigned adequate access rights for the didactic setup along with dedicated timeslots when they can access it. The didactic setup data includes the 3D model of the setup as well as the descriptions of its functions required for using the augmented reality options in the smartphone application. The smartphone application allows the user to log on to the system, and based on their defined access rights it grants or denies the user control of the didactic setup. After the user has logged on, a connection between the smartphone application and the didactic setup is established, in order to establish communication and enable sending commands to the didactic setup, as well as downloading data from the didactic setup to display them on the smartphone through augmented reality. Communication between the smartphone application and the didactic setup can be established either via Bluetooth or via Wi-Fi communication, depending on the actual equipment of the didactic setup.

# 3 The Realized Solution

As a proof of concept, an experimental system was developed and realized for use in hands-on teaching activities. This experimental system serves as an upgrade of a didactic setup for control of a three-phase asynchronous motor, previously developed at the Faculty of Technical Sciences in Novi Sad. The didactic setup consists of a programmable logic controller (PLC), type Siemens Simatic S7-1200, frequency converter type Danfoss FC51, a three-phase asynchronous motor, command buttons and switches, signal lamps and auxiliary components (fuses, power supply, etc) [4]. The command buttons are connected to the PLC, and allow executing basic commands such as starting or stopping the three-phase asynchronous motor, changing the direction of rotation of the motor, or changing (reducing or increasing) its speed. Following the user commands, the PLC sends commands to the frequency converter, which controls the direction and speed of rotation of the three-phase asynchronous motor. In addition, the PLC controls the signal lamps that show the current operation state of the didactic setup to the users. The used didactic setup is shown in Fig. 2.

In order to provide remote control of the didactic setup, a dedicated Android smartphone application was developed and implemented by using the Vuforia augmented reality library and Unity development environment [18]. By starting the smartphone application, the user logs on to the system with their credentials, which were previously assigned to them by an authorized person. This allows the control of the didactic setup only to the users having the adequate access rights for the specific time slot. When the user selects the didactic setup, the communication between the smartphone application and the didactic setup is established. The server computer, the didactic setup, and the smartphone are all connected to a local computer network via a WiFi access point. In the next step, the didactic setup data (its 3D model and the descriptions of its functions) become visible to the user. By directing the camera of the smartphone towards the didactic setup, the image processing algorithms implemented in the Vuforia library recognize the didactic setup and, implementing the concept of
augmented reality, the information on the didactic setup and commands for its control are displayed on the screen of the smartphone. Fig. 3 shows the flow diagram for the user side of the developed solution.

<figure><img src="image_3.png" /><figcaption>Fig. 2. The didactic setup for control of a three-phase asynchronous motor</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3. The flow diagram of the developed solution</figcaption></figure>

The control of the didactic setup by using a smartphone and augmented reality was realized through the use of virtual buttons. Fig. 4 shows the smartphone application with virtual command buttons "S", "R", "-" and "+". The virtual control button "S" is used to start and stop the three-phase asynchronous motor. The virtual control button
"R" is used to change the direction of rotation of the three-phase asynchronous motor. Virtual command buttons "-" and "+" are used to reduce or increase the speed of the three-phase asynchronous motor. Virtual buttons do not physically exist but are displayed to the user's smartphone screen as augmented reality. By activating the virtual control buttons, the smartphone sends adequate commands to the didactic setup. This way, the control of the didactic setup (starting and stopping the motor rotation, changing the motor rotation direction, changing the motor rotation speed) without physically touching the control buttons on the didactic setup is enabled.

<figure><img src="image_5.png" /><figcaption>Fig. 4. User smartphone application with virtual command buttons.</figcaption></figure>

In order to allow sending commands to the didactic setup, the existing PLC program of the didactic setup for controlling the three-phase asynchronous motor has been modified. The server application was implemented in C# programming language in Microsoft Visual Studio development environment [19]. The server application allows easy creation and modification of user profiles, as well as transferring the data about the didactic setup upon request. For further improvement of the concept as well as increase of general security, the implementation of a system for identification of didactic setups (for example 2D barcodes [20]) and a more advanced system for identification of users (for example access control as a service for software security [21]) is planned.

## 4 Conclusion

This paper presents a solution for improving the safety of using didactic setups by applying the concept of augmented reality. This system enables the control of the didactic setup from a dedicated smartphone application through augmented reality, which enables the control without physical contact of the user and the didactic setup. Such a solution improves safety as during its intended use there is no possibility of
injury due to electric shock or contact with actuators. The advantage of the proposed and realized solution is that students can control the didactic setup without supervision of the teaching staff with minimal risks, assuming that they follow the safety regulations for handling the machines in the laboratory. In such a setting, the physical buttons on the didactic setup can be completely disabled, or the access to the buttons or even to the whole didactic setup can be physically disabled (e.g. by locking the control cabinet or putting the safety barrier between the students and the didactic setup), as the solution allows control of the didactic setup by using only the smartphone application and the virtual command buttons. It is important to note that even if full control of the didactic setup is available through the smartphone application, a physical emergency stop button has to be available to the students at all times for unexpected situations. A preliminary assessment and validation of the developed solution was conducted in collaboration with five students. All five students gave a positive assessment of the realized solution regarding the safety aspect, with only a single comment that it is not always easy to simultaneously keep a phone directed at the didactic setup and command virtual buttons. To confirm this preliminary assessment, an extensive user study is planned, which would include a larger number of students, in order to assess the applicability and usability of the presented solution. The study would also extend the developed solution to include additional didactic setups located at the Faculty of Technical Sciences in Novi Sad. As further development of the system it is planned to include the use of augmented reality smartglasses to facilitate the manipulation of the setup by the users and improve usability (instead of holding the smartphone directed toward the didactic setup, the students could control the setup simply by looking through the smartglasses).

## Acknowledgment

This research has been supported by the Ministry of Education, Science and Technological Development, Government of the Republic of Serbia, through the project: "Innovative scientific and artistic research from the FTN activity domain".

## References

1. Bhutkar, G., Rajhans, N., Konkani, A., Dhore, M.: Usability issues of user manuals provided with medical devices. Br. J. Healthc. Comput. Inf. Manag. (2009).

2. Møller, M.H.: Usability Testing of User Manuals. Commun. Lang. Work. 2, 51–59 (2013).

3. Cunha, B.G.P., Mendes, S.T., Dutra, P.M., Rodrigues, T.V., Nunes, L.M., Machado, F.M.F., da Silva Martins, C.A.P.: DidacTronic: A low-cost and portable didactic lab for electronics: Kit for digital and analog electronic circuits. In: 2016 IEEE Global Humanitarian Technology Conference (GHTC). pp. 296–303 (2016).

4. Ostojic, G., Stankovski, S., Tarjan, L., Senk, I., Jovanovic, V.: Development and implementation of didactic sets in mechatronics and industrial engineering courses. Int. J. Eng. Educ. 26, 2 (2010).
5. Stankovski, S., Tarjan, L., Skrinjar, D., Ostojic, G., Senk, I.: Using a didactic manipulator in mechatronics and industrial engineering courses. IEEE Trans. Educ. 53, 572–579 (2009).

6. Van Wyk, E., De Villiers, R.: Virtual reality training applications for the mining industry. In: Proceedings of the 6th international conference on computer graphics, virtual reality, visualisation and interaction in Africa. pp. 53–63 (2009).

7. Gavish, N., Gutiérrez, T., Webel, S., Rodriguez, J., Peveri, M., Bockholt, U., Tecchia, F.: Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interact. Learn. Environ. 23, 778–798 (2015).

8. Hasan, R.B., Aziz, F.B.A., Mutaleb, H.A.A., Umar, Z.: Virtual reality as an industrial training tool: A review. J. Adv. Rev. Sci. Res. 29, 20–26 (2017).

9. Billinghurst, M., Kato, H., Poupyrev, I., others: Tangible augmented reality. ACM SIGGRAPH ASIA. 7, (2008).

10. Marescaux, J., Rubino, F., Arenas, M., Mutter, D., Soler, L.: Augmented-reality--assisted laparoscopic adrenalectomy. Jama. 292, 2211–2215 (2004).

11. Fraga-Lamas, P., Fernández-Caramés, T.M., Blanco-Novoa, Ó., Vilar-Montesinos, M.A.: A review on industrial augmented reality systems for the industry 4.0 shipyard. Ieee Access. 6, 13358–13375 (2018).

12. Yuen, S.C.-Y., Yaoyuneyong, G., Johnson, E.: Augmented reality: An overview and five directions for AR in education. J. Educ. Technol. Dev. Exch. 4, 11 (2011).

13. Kounavis, C.D., Kasimati, A.E., Zamani, E.D.: Enhancing the tourism experience through mobile augmented reality: Challenges and prospects. Int. J. Eng. Bus. Manag. 4, 10 (2012).

14. Fischer, J., Neff, M., Freudenstein, D., Bartz, D.: Medical Augmented Reality based on Commercial Image Guided Surgery. In: EGVE. pp. 83–86 (2004).

15. Yovcheva, Z., Buhalis, D., Gatzidis, C.: Smartphone augmented reality applications for tourism. E-review Tour. Res. 10, 63–66 (2012).

16. Pettersen, T., Pretlove, J., Skourup, C., Engedal, T., Lokstad, T.: Augmented reality for programming industrial robots. In: The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings. pp. 319–320 (2003).

17. Peng, F., Al-Sayegh, M.: Personalised Virtual Fitting for Fashion. Int. J. Ind. Eng. Manag. 5, 233–240 (2014).

18. Cherdo, L.: The 8 best augmented reality smartglasses in 2020, https://www.aniwaa.com/buyers-guide/vr-ar/best-augmented-reality-smartglasses/, last accessed 2020/03/26.

19. Microsoft: Visual Studio tutorials | C#, https://docs.microsoft.com/en-us/visualstudio/get-started/csharp/?view=vs-2019, last accessed 2020/03/26.

20. Tarjan, L., Šenk, I., Kovač, R., Horvat, S., Ostojić, G., Stankovski, S.: Automatic identification based on 2D barcodes. Int. J. Ind. Eng. Manag. 2, 151–157 (2011).

21. Dašić, P., Dašić, J., Crvenković, B.: Applications of access control as a service for software security. Int. J. Ind. Eng. Manag. 7, 111–116 (2016).