# Goal-Conditioned Reinforcement Learning within a Human-Robot Disassembly Environment
## 2 Related Works
### Path Planning and Collision Avoidance  
Collision avoidance of robotic manipulators is a hot topic in robot control research. This technique finds a sequence of joint configurations that allows the robot to move from an initial to a target point while avoiding potential obstacles along the way. Conventional path planning methods like rapid-exploring random trees (RRT) [29] and artificial potential fields (APF) [30] are well-researched techniques that can compute a trajectory with a lower computational cost than RL. However, RRT is inefficient for fast, reactive, and dynamic collision avoidance action [31], and APF tends to fall into local minima when attractive and repulsive forces are similar. Alternative solutions include predicting the motion or recognizing the operator's intentions. For instance, while [PERSON] and [PERSON] [32] used recurrent neural networks to predict the operator's movement based on its position, velocity, and acceleration, [PERSON] et al. [33] used an electroencephalogram to detect the operator's shift intentions before the movement was performed. However, although the results were promising, in practice, it remains challenging due to complex human behavior and background noise levels in many industrial plants. In this sense, RL allows adaptive control in dynamic environments due to its self-learning capability when the working environment varies, making it an appealing solution for unstructured environments such as HRI scenarios. Most research relies on actor-critic algorithms [34]. For instance, [PERSON] et al. [35] proposed a path planning algorithm for a multi-arm manipulator which they considered as a single-arm virtual manipulator steering from an initial to a final configuration. In a similar case study, [PERSON] et al. presented two-path planning methods using either residual RL [36] or curriculum RL [37] to find the axis configuration that allowed the robot to perform a collision-free trajectory. However, all these studies validated their methodology only with static obstacles. In turn, [PERSON] et al. [38] presented a framework that mapped HRI tasks and safety requirements onto RL settings. However, the generated human motion in the collision avoidance simulation was randomized without considering possible hazardous scenarios. The same limitation can be noticed in the work of [PERSON] et al. [39] and [PERSON] et al. [40; 41]. In the former, the obstacles moved in limited trajectories periodically, while in the latter a real-time model-free collision avoidance approach was introduced and applied to robotic tasks where an unpredictable obstacle interfered with the robot's workspace. [PERSON] et al. [42] proposed a real-time robotic manipulator motion planning framework which they tested with both static and dynamic obstacles. Nevertheless, the negative reward given to the agent was only provided after a certain threshold was exceeded, which could increase training time and even affect convergence if the target was not reached during exploration. To avoid a similar situation, [PERSON] et al. [43] included a balance estimator in the reward function that compared worker safety with task performance efficiency. In addition, in this study, the obstacle had a motion acquired from skeleton tracking. Yet, as in the other studies, the methodologies were evaluated in via-point experiments, where the robot only moved from an initial to a target point. This does not detract from the fact that in HRI tasks, despite ensuring co-worker safety, the robot should keep the workflow efficient, and accomplish those tasks assigned to it. In this regard, one of the few papers that addressed contact-rich manipulation tasks while the robot agent avoided collision with its surroundings is [PERSON] et al. [44]. However, although the framework integrated the motion planner into the RL policy, collision avoidance for static obstacles was performed by RRT, and RL completed contact-rich tasks, so the authors switched between the two techniques.