## 2 Background and Literature Review  
RL generally involves an advanced model of the Markov decision process (MDP). In terms of sequential decision making, it is based on the interaction between the current state and the system. In this method, the reward is calculated, and then the action needed to achieve the learning objective is determined [1]. The general learning method of the RL algorithm is the Q-learning method, which calculates and updates the Q-function, the behavior value function of the learning object, at every time \(t\). At this time, the algorithm is designed to calculate the Q-function by maximizing the reward value [4]:  
\[Q_{t+1}(s_{t},a_{t})=(1-\alpha)Q_{t}(s_{t},a_{t})+\alpha(r_{t+1}+\gamma maxQ_{ t}(s_{t+1},a)) \tag{1}\]  
In (1), \(s_{t}\) is the state at time \(t\); \(a_{t}\) is the learning object behavior at time \(t\); \(r_{t+1}\) is the reward value at time \(t+1\); and \(\alpha\) is the learning rate. The closer to \(\alpha\), the greater the value of the situation at the current time \(t\) and the behavior of the learning object. The discount rate \(\gamma\) is used to adjust the reward percentage for future behavior [5]. RL is used in a wide range of fields to solve important issues. This technique is applied according to the situation and environment, and methods of solving the corresponding problems are designed. In robot control, RL is an interesting topic and the most commonly used AI method. Research on various robotics topics, such as the use of robots in intelligence, soft robotics, and robot automation through navigation and autonomous control, has been conducted. Table 1 summarizes recent studies focusing on the relationship between humans and robots in relation to the learning of objects that are considered human interventions. It also lists the applications of these studies, keywords, and design methods of RL [6; 7; 8; 9; 10; 11]. For example, RL has been effectively applied to enable individuals suffering from limb paralysis to drink liquids directly with the help of robotic manipulator arms [6]. In that study, five algorithms were applied and compared. Learning was performed effectively with a software emulator program, and the developed solution gave the user the ability to manipulate the cup. An assistant robot was effectively designed with the focus of supplying liquid from the cup using feedback through sensors that provided direct interaction between the human and robot. Subsequently, RL was applied for stable, dynamic walking of biped robots [7]. This study was performed in the absence of prior knowledge or information on dynamic models, and the robot operation was controlled by mapping the motion space from the discrete to continuous domain. The research objective was to solve complex control problems. Among the components constituting the robot legs, a zero-moment point was selected from the sole and mapped to the movement of the limbs to learn balance. This study proved that a robot can learn how to improve its motion in terms of walking speed. Further, the proposed algorithm was implemented in a physical robot to prove its validity and effectiveness. Another article suggested a framework that includes a learning phase that mimics human behavior and an RL phase that learns robot behavior [8]. The two-stage learning framework, which combines imitation and RL, shows how to work with people to lift tables quickly and successfully. The first stage is for learning the existence and location of the object called a table, and the second stage is the learning stage for performing operations and tasks. The robot operation is controlled by combining two types of controllers. This research demonstrated the successful construction of a collaborative robot designed to predict human movements and take proactive actions. In another study, RL was applied to pancake flipping, energy minimization of bipedal walking robots, and archery-based aiming robots [9].