## 2 Related Work
### Methods for Controlling Robots Using Visual Inputs  
In the following chapter, methods utilizing video cameras or RGBD cameras will be explored, as these techniques are among the most widely adopted. The VGPN method, which integrates voice commands and pointing gestures to guide a robot to a specified location, was proposed in [35]. This method was tested on the TurtleBot 2, a wheeled robot. Voice commands offer the advantage of being well-suited for simple and specific target locations, allowing the operator to maintain focus on other tasks without being constrained. However, voice commands can be less effective for complex navigational instructions. Pointing gestures, on the other hand, provide intuitive and natural control for the operator. Despite this, they can suffer from issues related to the accuracy of the operator's pointing and the complexity of the computational processes involved. The VGPN prototype was implemented using ROS. To determine the operator's pointing direction, two key points on the human body are utilized. These points are used to generate a vector that defines the direction of the point. In their work, the authors specifically employ a vector extending from the operator's eye to their wrist to represent the pointing direction. Static and dynamic gestures, including body poses and hand movements, are utilized to control the robot in paper [30]. Body gestures are recognized using an RGBD camera, while hand gestures are detected with a Leap Motion device. The recognized gesturesare transmitted as messages within the ROS middleware. The defined gestures enable the operator to direct the robot's movement in six directions: \(\pm\)x, \(\pm\)y, and \(\pm\)z relative to a selected frame. For instance, an \"up\" gesture corresponds to movement in the +x direction, a \"down\" gesture for -x movement, and similar gestures are used for the other directions. Additionally, two specific static gestures are employed to stop the robot during operation. These gestures deactivate the robot's motor controls, halting its movement. The Kinect v2 RGBD camera (Microsoft Corporation, Redmond, WA, USA) was utilized in [31] to facilitate gesture recognition. Their application identified gestures based on the positioning of the operator's arms and hands, focusing on static gestures. For voice control, the Kinect v2 was configured as the default microphone on the computer, with voice commands being converted into text and matched against a predefined dictionary. These recognized commands were then mapped to signals that control the ABB IRB 120 robot (ABB Inc. Robotics & Discrete Automation, Auburn Hills, MI, USA). The control program for the robot was developed using the RAPID language within the RobotStudio environment. Additionally, the authors designed a user interface (GUI) to enhance the control of the industrial robot through hand movements in 3D space and voice commands. The GUI features a range of functions, including a help button, program information, a list of control methods, details on the current controlled axis, the robot's position in x, y, and z axes, a list of available robots, a robot accuracy window, a speed selection window, and a start button. The interface also includes an \"operator skeleton\" field, which visually represents the operator's movements and gestures, providing an intuitive control experience. A novel control system for a mobile robot is presented in [32], which leverages dynamic gestures captured by the Microsoft Kinect 2 RGBD camera. The control system is divided into two main subsystems: turning and acceleration/braking. The turning subsystem enables the user to control the robot's direction by holding and manipulating a virtual steering wheel. This interaction translates the rotation gestures into commands that adjust the robot's wheel direction. The acceleration and braking subsystem allows users to simulate vehicle controls such as shifting gears and adjusting speed.