Due to world dynamics and hardware uncertainty, robots inevitably fail in task executions, leading to undesired or even dangerous executions. To avoid failures for improved robot performance, it is critical to identify and correct robot abnormal executions in an early stage. However, limited by reasoning capability and knowledge level , it is challenging for a robot to self diagnose and correct their abnormal behaviors. To solve this problem, a novel method is proposed, human-to-robot attention transfer ( H2R-AT) to seek help from a human . H2R-AT is developed based on \textbf{ \textit{\textbf{  a novel stacked neural networks model, transferring human attention embedded in verbal reminders to robot attention embedded in robot visual perceiving . With the attention transfer from a human , a robot understands what and where \textit{ \textit{ human concerns are to identify and correct its abnormal executions. To validate the effectiveness of H2R-AT, two representative task scenarios , " serve water for a human in a kitchen" and " pick up a defective gear in a factory" with abnormal robot executions, were designed in an open-access simulation platform V-REP; 252 volunteers were recruited to provide about 12000 verbal reminders to learn and test the attention transfer model H2R-AT. With an \textit{\textbf{  accuracy of 73.68\\%DIF < in transferring attention and accuracy of 66.86\\% in avoiding robot execution failures, the effectiveness of H2R-AT was validated.\end{abstract} %DIF > in transferring attention, and the high accuracy of 66.86\\% in avoiding grasping failures.
Due to real-world dynamics and hardware uncertainty, robots inevitably fail in task executions, resulting in undesired or even dangerous executions. In order to avoid failures and improve robot performance, it is critical to identify and correct abnormal robot executions at an early stage. However, due to limited reasoning capability and knowledge storage , it is challenging for robots to self-diagnose and -correct their own abnormality in both planning and executing. To improve robot self diagnosis capability, in this research a novel human-to-robot attention transfer ( \textbf{H2R-AT ) method was developed to identify robot manipulation errors by leveraging human instructions.\textit{\textbf{H2R-AT was developed by fusing attention mapping mechanism into a novel stacked neural networks model, transferring human verbal attention into robot visual attention . With the attention transfer , a robot understands \textit{what and\textit{where human concerns are to identify and correct abnormal manipulations. Two representative task scenarios : `` serve water for a human in a kitchen" and `` pick up a defective gear in a factory" were designed in a simulation framework CRAIhri with abnormal robot manipulations; and 252 volunteers were recruited to provide about 12000 verbal reminders to learn and test \textit{\textbf{H2R-AT . The method effectiveness was validated by the high accuracy of 73.68\\%DIF < in transferring attention and accuracy of 66.86\\% in avoiding robot execution failures, the effectiveness of H2R-AT was validated.\end{abstract} %DIF > in transferring attention, and the high accuracy of 66.86\\% in avoiding grasping failures.
R
world
real-world
R
leading to
resulting in
R
To avoid failures for improved
In order to avoid failures and improve
R
robot abnormal executions in
abnormal robot executions at
R
limited by
due to limited
R
level
storage
R
a robot to self diagnose and correct their abnormal behaviors. To solve this problem, a novel method is proposed,
robots to self-diagnose and -correct their own abnormality in both planning and executing. To improve robot self diagnosis capability, in this research a novel
D
H2R-AT) to seek help from a human . H2R-AT is developed based on
A
H2R-AT
A
) method was developed to identify robot manipulation errors by leveraging human instructions.
A
H2R-AT
A
was developed by fusing attention mapping mechanism into
R
attention embedded in verbal reminders to robot attention embedded in robot visual perceiving
verbal attention into robot visual attention
D
from a human
D
what and where
A
what
A
and
A
where
R
its abnormal executions. To validate the effectiveness of H2R-AT, two
abnormal manipulations. Two
R
, "
: ``
R
"
``
D
with abnormal robot executions,
R
an open-access simulation platform V-REP;
a simulation framework CRAIhri with abnormal robot manipulations; and
D
the attention transfer model H2R-AT. With an
A
H2R-AT
A
. The method effectiveness was validated by the high