General Hand Guidance Framework using Microsoft HoloLens Hand guidance emerged from the safety requirements for collaborative robots, namely possessing jointtorque sensors. Since then it has proven to be a powerful tool for easy trajectory programming, allowing layusers to reprogram robots intuitively. Going beyond, a robot can learn tasks by user demonstrations through kinesthetic teaching, enabling robots to generalise tasks and further reducing the need for reprogramming. However, hand guidance is still mostly relegated to collaborative robots. Here we propose a method that does not require any sensors on the robot or in the robot cell, by using a Microsoft HoloLens augmented reality head mounted display. We reference the robot using a registration algorithm to match the robot model to the spatial mesh. The inbuilt hand tracking and localisation capabilities are then used to calculate the position of the hands relative to the robot. By decomposing the hand movements into orthogonal rotations and propagating it down through the kinematic chain, we achieve a generalised hand guidance without the need to build a dynamic model of the robot itself. We tested our approach on a commonly used industrial manipulator, the KUKA KR5.

Combining Interactive Spatial Augmented Reality with HeadMounted Display for EndUser Collaborative Robot Programming This paper proposes an intuitive approach for collaborative robot enduser programming using a combination of interactive spatial augmented reality (ISAR) and headmounted display (HMD). It aims to reduce userxe2x80x99s workload and to let the user program the robot faster than in classical approaches (e.g. kinesthetic teaching). The proposed approach, where user is using a mixedreality HMD xe2x80x93 Microsoft HoloLens xe2x80x93 and touchenabled table with SAR projected interface as input devices, is compared to a baseline approach, where robotxe2x80x99s arms and a touchenabled table are used as input devices. Main advantages of the proposed approach are the possibility to program the collaborative workspace without the presence of the robot, its speed in comparison to the kinesthetic teaching and an ability to quickly visualize learned program instructions, in form of virtual objects, to enhance the usersxe2x80x99 orientation within those programs. The approach was evaluated on a set of 20 users using the withinsubject experiment design. Evaluation consisted of two pick and place tasks, where users had to start from the scratch as well as to update the existing program. Based on the experiment results, the proposed approach is better in qualitative measures by 33.84% and by 28.46% in quantitative measures over the baseline approach for both tasks.