İşbirlikçi robotların haptik arayüzlerle teleoperasyonu
Haptic teleoperation of cooperating robots
Yazar: ÖMER FARUK ARGIN
Danışman: DOÇ. DR. ZEKİ YAĞIZ BAYRAKTAROĞLU
İstanbul Teknik Üniversitesi / Lisansüstü Eğitim Enstitüsü / Mekatronik Mühendisliği Ana Bilim Dalı
Derece: Doktora, 2021
Konu: Mekatronik Mühendisliği = Mechatronics Engineering
Dizin: Robotik = Robotics

Özet:
Robotlar günlük hayatımızda ve üretim  alanında katlanarak  artan bir şekilde kullanılmaktadır. Gelecekte daha da akıllanarak insanlar ile birlikte çalışacak ve diğer robotlarla işbirliği içinde çalışarak akıllı fabrikalar oluşturacaklardır. Robotlar bazı uygulamalarda, insan operatörünün kontrolünde ve uzaktan teleoperasyonla kontrol edilmektedir. Uzakta çalışan robot bilinmeyen ve dinamik bir çevreyle etkileşime girmektedir. Bazı uygulamalarda daha hassas ve kararlı teleoperasyon gerçekleştirebilmek için operatör uzak ortam hakkında görsel geri beslemenin yanında haptik geri beslemeyle beslenmektedir. Bu haptik geribeslemeye sahip teleoperasyon uygulamalarına haptik teleoperasyon sistemleri denir.

Bu çalışmada,  bir lokal manipülatör ve çok uzak manipülatörle işbirlikçi  haptik teleoperasyon için kontrol şemaları önerilmektedir. İşbirilikçi teloperasyon için konum kontrollü, kuvvet/konum kontrollü  paylaşımlı kontrollü kontrol şemalar sunulmaktadır. Uzak manipülatörler için önerilen kontrol yapılarının asimptotik kararlılıkları iletişim kanallarında sınırlı zaman gecikmelerinin olduğu durum için Lyapunov analizi ile gösterilmektedir. Önerilen kontrol şemaları endüstriye yönelik basit manipülasyon ve pim yerleştirme  uygulamaları ile deneysel olarak gerçekleştirilmektedir. Deneysel uygulamalarda önerilen kontrolcüler, kararlı teleoperasyonlarda konum takip performansları, eğitimli ve eğitimsiz kullanıcılar ile uygulanabilirlik performansı, engelli ortamda manipülasyon ile sağlamlık analizleri ve sistemin şeffaflığı incelenmektedir. 

Önerilen kontrol şemalarının uygulanması ve deneysel doğrulaması, modüler bir sanal modelleme ortamı yardımıyla elde edilir. Çift yönlü  haptik teleoperasyon için önerilen kontrol şeması, birden fazla uzaktan manipülatörle işbirliği görevlerine sanal modelleme ortamı yardımıyla kolayca genişletilebilmektedir. Manipülatörün geometrik parametrelerini içeren kinematik zincirin sanal ortamda tanımlanması ile önerilen iki yönlü ve işbirlikçi kontrolörlerin uygulanması için yeterlidir. Başka bir matematiksel modele ihtiyaç duyulmadığından, geometrik parametreleri mevcut olan herhangi bir rastgele robot kolu haptik teleoperasyon şemasına kolayca entegre edilebilir.

Endüstriyel manipülatörlerin teleoperasyon ile kontrolünde, kullanıcı tarafından uygulanan yüksek frekanslı hareket referansları, sistemin kararsız davranışına neden olabilir. Ayrıca manipülatör kontrolünde aktüatör akımları genellikle güvenlik nedenleriyle sınırlandırılır bu da takip problemine neden olmaktadır. Herhangi bir haptik arayüzle kuvvet geri beslemesi olmadığında, insan operatör tarafındaki hareket referansları izin verilen giriş frekanslarını aşabilir. Bu durumlar uzak ortamda istenmeyen hareketlere ve hasar vb.  neden olur ve ayrıca deneyimsiz kullanıcıların eğitim sürelerini uzatır. Bu çalışmada önerilen sanal haptik etkileşim kuvveti kullanıcıya teleoperasyon sırasında görünür bir atalet hissi sağlar. Yapay olarak oluşturulan bu ikinci dereceden dinamikler, kullanıcıdan gelen yüksek frekanslı hareket referanslarını filtreler ve bu nedenle deneyimsiz kullanıcıların kolayca teleoperasyon gerçekleştirmesine olanak tanır. Önerilen haptik etkileşim sistemi, manipüle edilen yük ile birlikte çalışan manipülatörler arasında sanal yay sönümleyicilerin kullanılmasına dayanmaktadır. Bu çalışmada sunulan deneysel sonuçlar, önerilen haptik etkileşim sisteminin tahmin edilen katkılarını doğrulamaktadır.

Abstract:
Robots are increasingly used in our daily life and in production. In the future, they  will become even smarter, working with humans and working in collaboration with other robots to create smart factories. In some applications, robots are controlled by the human operator and by teleoperation in remote environments. Teleoperation is used to enable a stable dexterous interaction in remote tasks e.g. telesurgery, rehabilitation, mining, space or underwater exploration from safe and reachable (local) environments. 

When the teleoperated robot is working in a remote environment, it interacts with an unknown and dynamic environment. In some applications, the operator is given visual feedback about the remote environment as well as some haptic feedback in order to perform more precise and simple teleoperation. These applications are haptic teleoperation systems. In haptic teleoperation systems incorporating haptic devices as local robots, the human operator is provided with force feedback taking place in real physical interactions in the remote environment. The haptic information in the bilateral/multilateral systems enhance the performances of human operators, improving their manipulative skills of the task performed in the remote environment.

Improvement of the stability and transparency are the main challenges in haptic teleoperation systems. Time delays on the communication channels have a disturbing impact on tracking performance and stability of haptic teleoperation. High velocity inputs by the user can cause instability for the system. Transparency expresses how well the position and force transmission is between the local and remote robot. Stability and transparency are conflicting performance criteria in haptic teleoperation applications. In controller design, a trade off must be filled between the criteria of stability and transparency. Passivity based, small gain approach, adaptive, predictive and intelligent based controllers are proposed to improve stability and transparency in the literature.   

Robots are expected to be robust against the environment and changes in the environment in a sensitive manipulation. This can be provided by controlling both position and force control. Compliance control is required in assembly, grinding, polishing and human-robot interface applications that require both position control and force control of robots.

This thesis proposed to design of a new haptic teleoperation system for bilateral a local and cooperative remote robots for industrial use. Structures proposed for cooperative teleoperation are first implemented for bilateral single master/ single slave (SM/SS) robots and then expanded to cooperative single master/ multi slave (SM/MS) haptic teleoperation systems.

In this study, novel haptic teleoperation control schemes are proposed within the scope of stability and transparency design criteria. The purpose of the proposed control schemes are to improve teleoperation performances with a simple controller structure and user-friendly virtual physical interfaces. 

A virtual remote environment model layer is added to the control system in order to realize a stable teleoperation. The virtual remote environment model is created with open source software and has a modular structure. The proposed control scheme for bilateral haptic teleoperation can be easily extended to cooperative tasks with multiple remote manipulators with the help of a virtual modeling environment. It is sufficient for the implementation of the proposed bilateral and collaborative controllers by defining the kinematic chain containing the geometric parameters of the manipulator in a virtual environment. Since no other mathematical model is needed, any random robot arm with geometric parameters available can be easily integrated into the haptic teleoperation scheme. 

In the control of industrial manipulators by teleoperation, high frequency motion references applied by the user can cause unstable behavior of the system. In addition, actuator currents in manipulator control are generally limited for safety reasons, which causes position tracking problems. In the absence of force feedback with any haptic interface, motion references on the human operator side may exceed allowable input frequencies. These situations cause unwanted movements and damage in the remote environment and also extends the training time of inexperienced users. The virtual haptic interaction force suggested in this study provides the user with a visible sense of inertia during teleoperation. These artificially created second order dynamics filter the high frequency motion references from the user and therefore allow unexperienced users to easily teleoperation. The proposed haptic coupling system is based on the use of virtual spring dampers between manipulators working together with the manipulated load. The experimental results presented in this study confirm the predicted contributions of the proposed haptic interaction system.

All control schemes proposed in this thesis are empirically verified with basic pick-and-place and peg-in-a-hole applications which are commonly seen in the industry. Experimental results show that the proposed haptic coupling system behaves decisively against high motion inputs and communication delays, and provides improvements in position tracking.

Open source \textit{Blender} software is used to create virtual environment in this thesis. 6-dof Stäubli RX160 and 160L robots are used as remote robots and 6-dof Phantom Premium haptic device is used as a local robot in the experiments. Ati Delta sensor is used for measuring the remote force/torque values at the end-effector. 

A new haptic coupling model is created between the peg and the hole in order to create a haptic guidance force for the user to facilitate manipulation in the peg-in-a-hole application. According to the experimental results, it is observed that with the switched haptic teleoperation control scheme created in the peg-in-a-hole application, the physical coupling between the peg and the hole and thus the haptic feedback forces decrease and a faster operation.


The remote environment may not always be perfectly modeled or unexpected sudden changes may be observed in the remote environment. These situations cause system instability and tracking errors. In cases where robots work in a remote environment with position control, this lack of model or sudden changes in the remote environment can be detected by the user and respond late. This situations can be reason of the irreversible damage to remote environment manipulation and health. In order to realize a stable and robust haptic teleoperation against this disturbances in the remote environment, two different adaptation controllers are proposed for remote robots, with position / force control and haptic shared control in cooperative manipulation. 

In the proposed position / force control scheme, one of the remote robots is directly position controlled while the other robots move in the leader-follower formation with force control. In the position / force control scheme, the follower manipulator works completely with pure force control. This makes the system sensitive to disturbances. 

Shared control is one of the approaches that has started to be used frequently in haptic teleoperation systems in recent years. In this method, while haptic teleoperation is performed, remote robots not only follow the reference position sent by the local robot, but also operate the position and / or force control within itself, providing more precise manipulation without damaging the manipulated object.

By applying position / force control and shared controllers in haptic teleoperation, it is aimed to prevent the damages that will occur in the remote environment against delayed / erroneous manipulations during manipulation of robots working in the remote environment until the user receives a signal. PID controller is used for the position control, PD controller is used for the pure force control and explicit stiffness controller is used for shared control scheme. In the experimental results using unknown obstacles in the remote environment, it is seen that the controllers with position / force control and haptic shared control perform a more robust haptic teleoperation.

Lyapunov and asymptotic stability of the proposed control schemes for position-controlled and force / position-controlled remote manipulators is shown by Lyapunov analysis for the case of bounded time delays in communication channels. In the stability analysis, the system dynamics for SM/SS systems are defined in the joint space. Since it is difficult to explain the joint space dynamics of multiple remote robot and part in collaborative SM/MS haptic teleoperation, the dynamics of the entire remote environment is defined by expressing the task space dynamics of the load.

 The proposed haptic teleoparation system with using the coupling system and compatible controllers has been experimentally observed to be user-friendly, allowing easy adaptation of unexperienced users. Experiments show that unexperienced users can easily achieve stable and transparent haptic teleoperation of cooperative manipulators for simple pick-and-place and peg-in-a-hole applications. In addition, similar performances are observed in terms of location tracking errors of remote manipulators in cases where there are experienced and unexperienced users in the loops.