MITSUBISHI Project A:
Title

Development of a Mapping Space for Intuitive Teleoperation with Heterogeneous Devicesof Multiple Types
Industrial Partner: Advanced Technology R&D Center of Mitsubishi Electric Corp.
Mitsubishi Electric Corp., founded in 1921, is an electronic and electric equipment
manufacturer developing products and solutions in widely diverse fields, including home
appliances, industrial equipment, and space technologies. The Advanced Technology R&D
Center was established to support the business of Mitsubishi Electric Group through thedevelopment of a broad scope of projects covering both basic and new advancedtechnologies. The main research themes include power electronics, mechatronics, satellitecommunications, next generation key devices, system solutions for electric power,transportation, factory automation, and automobiles.
Industrial Mentors

Tomoki Emmei, Ph.D., Mitsubishi Electric Corp.

Masaki Haruna, Mitsubishi Electric Corp.
Munetaka Kashiwa, Ph.D., Mitsubishi Electric Corp.
Introduction

Tele-operation technologies are attracting attention as solutions to reduce labor

shortages and work tasks that must be conducted in dangerous areas. In addition to direct
applications such as "remote surgical robot that can benefit from the skills of a great doctor
anywhere in the world"[1], other applications are emerging recently through studies such as"learning the operator's actions to promote robot autonomy" [2].
Development of operator-to-edge device mapping represents an important issue forteleoperation technology. Taking robot hand operation as an example, the lengths ofoperators’ fingers and the sizes of their palms present individual differences. It is therefore
necessary to select appropriate mapping of the robot hand in response to changes ofoperators. Mapping for non-anthropomorphic devices such as a magic hand is not readily
apparent compared to the case using anthropomorphic five-fingered robot hands as edge
devices. The goal of this project is to construct an appropriate mapping method that enables
connection between different structures and multiple types of devices by abstracting thecorrespondence between these non-anthropomorphic devices.
Technical BackgroundLet us take the remote control of a robot hand as an example again. Two typical mapping

methods exist for robot hands: joint-2-joint and point-2-point. The joint-2-joint is themethod of projecting the joint angle of the operator's fingers directly onto each joint angleof the robot hand. Although this method is good at "grabbing" objects, it is unsuitable for
precise motions because of the difference of fingertip positions between the operator andthe robot hand. The point-2-point mapping method, by contrast, overcomes this difficulty bymapping the fingertip positions of the operator to those of the robot. Consequently, theselection of mapping method has a considerably strong effect on the operability anddiscomfort of the remote control. An appropriate method must be selected depending on
the task.
As described above, mapping between devices that have identical geometry is easy.
However, proper application of a mapping method between devices with different
geometries is not easy. For example, when a three-fingered hand is used as an edge device,
movements between the operator and the robot hand do not always correspond intuitively.
Santello et al. proposed an interesting and effective approach to resolve these difficulties[3]. First, the operator's motion is extracted through three physical quantities: the degree offinger separation (α), the object size (ε), and the degree of finger bending (σ). Then themotion is projected onto the subspace spanned by these parameters. After this subspace isassociated with the motion of the robot hand, the operator's motion is transmitted to theedge device. Using this process, the correspondence between the robot hand and the edge
devices is made intuitive and easy by passing the essence of the motions to the subspacefrom which they are extracted. However, Santello et al. [3] specified that these bases werechosen "on intuition"; some room remains for improvement in the choice of the subspace.
Expectations

The goal of this project is consideration of a method for selecting a subspace that is intuitiveand capable of handling multiple types of edge devices with different structures, referring toan earlier study [3]. Although an inductive approach based on statistics is also possible, as
shown in [4], we would like to consider a deductive approach based on knowledge ofdynamical systems and nonlinear mapping. Although an earlier report [3] describedmappings between heteromorphic hands, the subject matter of this project is not necessarily
limited to mapping robotic hands. Because we possess several devices, including robot hands,
we intend to conduct experiments with them and to specify the edge devices after selection.

Software Packages and Special Requirements

In addition to an interest in human cognition, movement, and psychology, knowledge in anyof the following areas is desirable.
-

dynamical systems-

nonlinear mapping and geometry

Additionally, we plan to use C++ to implement the robot hand teleoperation. Knowledge andexperience in this area are welcome, but are not necessary.
Recommended Reading and References[1] Enayati, N., De Momi, E., & Ferrigno, G. (2016). Haptics in robot-assisted surgery:
Challenges and benefits. IEEE Reviews in Biomedical Engineering, 9(March), 49–65.
https://doi.org/10.1109/RBME.2016.2538080[2] Adachi, T., Fujimoto, K., Sakaino, S., & Tsuji, T. (2018). Imitation learning for object
manipulation based on position/force information using bilateral control. 2018 IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS), 3648–3653.
[3] Santello, M., Flanders, M., & Soechting, J. F. (1998). Postural hand synergies for tool use.
Journal of Neuroscience, 18(23), 10105–10115. https://doi.org/10.1523/jneurosci.18-2310105.1998[4] Meeker, C., Rasmussen, T., & Ciocarlie, M. (2018). Intuitive Hand Teleoperation by Novice
Operators Using a Continuous Teleoperation Subspace. IEEE International Conference on
Robotics and Automation, 5821–5827. https://doi.org/10.1109/ICRA.2018.8460506