Quickly Inserting Pegs into Uncertain Holes using Multiview Images and Deep Network Trained on Synthetic Data This paper explores the use of robots to autonomously assemble parts with variations in colors and textures. Specifically, we focus on peginhole assembly with some initial position uncertainty and holes located on surfaces of different colors and textures. Two inhand cameras and a forcetorque sensor are used to account for the position uncertainty. A program sequence comprising learningbased visual servoing, spiral search, and impedance control is implemented to perform the peginhole task with feedback from the above sensors. Contributions are mainly made in the learningbased visual servoing component of the sequence, where a deep neural network is trained with various sets of synthetic data generated using the concept of domain randomization to predict where a hole is. In the experiments and analysis section, the network is analyzed and compared, and a realworld robotic system to insert pegs to holes using the proposed method is implemented. The results show that the implemented peginhole assembly system can perform successful peginhole insertions on surfaces with various colors and textures. It can generally speed up the entire peginhole process, especially when the initial position uncertainty is large.

Transferable TrialMinimizing Progressive Peginhole Model Peginhole is a finelevel manipulation task that requires highly precise location and the normal direction of the hole, which is beyond stateoftheart object detectorsxe2x80x99 capability. Therefore, we propose a novel method that trains a robot arm to progressively search for the right inserting pose through trials with both force feedback and visual inputs. Under a reinforcement learning (RL) framework, an agent trying to minimize the number of trials is learned based on the sequentially estimated relative poses with regard to the correct inserting pose. Moreover, our learned dynamics model is transferable. Thanks to our contextindependent force and visual feature design, our pretrained model can be finetuned efficiently for another unseen peginhole case. Extensive experiments show the effectiveness of the proposed framework.