# Deep Reinforcement Learning with Interactive Feedback in a Human-Robot Environment
## 5 Conclusions  
We have presented an interactive deep reinforcement learning approach to train an agent in a Human-Robot environment. We have also performed a comparison between three different methods \begin{table}
\begin{tabular}{c c c} \hline \hline
**DeepRL vs.** & **DeepRL vs.** & **Agent–IDeePRL vs.** \\
**Agent–IDeePRL** & **Human–IDeePRL** & **Human–IDeePRL** \\ \hline \(t=7.6829\) & \(t=7.0192\) & \(t=0.8461\) \\ \(p=6.3947\times 10^{-14}\) & \(p=6.0755\times 10^{-12}\) & \(p=0.3978\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Student’s \(t\)-test for comparison of autonomous DeepRL, agent–IDeePRL, and human–IDeePRL. Figure 7: [PERSON]’s correlation between the collected rewards for different agents. The shown agents include an autonomous agent (\(A_{Au}\)), an interactive agent trained by an artificial trainer (\(A_{AT}\)), and the interactive agents trained by humans (from \(A_{H1}\) to \(A_{H11}\)). The collected reward for all the interactive approaches, including the one using the artificial trainer, presents a similar behavior showing a high correlation. On the contrary, the collected reward by the autonomous agent shows a lower correlation in comparison to the interactive agents. for learning agents. First, we implemented an autonomous version of DeepRL, which had to interact and learn the environment by itself. Next, we proposed an interactive version called IDeepRL, which used an external trainer to give useful advice during the decision-making process through interactive feedback delivered through early advising. We have implemented two variations of IDeepRL by using previously trained artificial agents and humans as trainers. We called these approaches agent-IDeepRL and human-IDeepRL, respectively. Our proposed IDeepRL methods considerably outperform the autonomous DeepRL version in the implemented robotic scenario. Moreover, in complex tasks, which often require more training time, to have an external trainer giving supportive feedback leads to great benefits in terms of time and collected reward. Overall, the interactive deep reinforcement learning approach introduces an advantage in domestic-like environments. It allows for speeding up the learning process of a robotic agent interacting with the environment and allows people to transfer prior knowledge about a specific task. Furthermore, using a reinforcement learning approach allows the agent to learn the task without the necessity of previously labeled data, such as the case for supervised learning methods. In this regard, the task is learned in such a way that the agent learns to recognize the state of the environment as well as to behave in regards to it, deciding where to place the different objects. Our novel interactive-shaping vision-based approach outperforms the current autonomous DeepRL method, used as a baseline in this work. The introduced approaches demonstrate that the use of external trainers, either artificial or human, lead to more and faster reward in comparison to the traditional DeepRL approach. As future work, we consider the use of different kinds of artificial trainers to select possible advisors better. A bad teacher can negatively influence the learning process and somehow limit the learner by teaching a strategy that is not necessarily optimal. To select a good teacher, it is necessary to take into account that an agent that obtains the best results for the task, in terms of accumulated reward, is not necessarily the best teacher [27]. Instead, a good teacher could be one with a small standard deviation over the visited states. This would allow for advising the learner agent in more specific situations. Additionally, we plan to transfer and test the proposed approach in a real-world Human-Robot interaction scenario. In such a case, it is necessary to deal with additional environmental dynamics and variables that in simulated scenarios may be easier to control. Conceptualization, F.C. ; Funding acquisition, F.C. and B.F.; Investigation, I.M. and J.R.; Methodology, A.A.; Supervision, F.C.