Atomic force microscope tip localization and tracking through deep learning based vision inside an electron microscope Scanning Electron Microscopy (SEM) is an ideal observation tool for small scales robotics. It has the potential to achieve automated nanorobotic tasks such as nanohandling and nanoassembly. Path following control of nanorobot end effectors using SEM vision feedback is a key for an intuitive programming of elementary robotic tasks sequences. It requires the ability to track end effectors under various SEM scan speeds. SEM suffers however from tricky issues that limits robotic tracking capabilities. This paper focuses on one specific issue related to the compromise between the scan speed and the image quality. This restriction seriously limits the performance of conventional vision tracking algorithms when used with electron images. At high scan speed, the image quality is very noisy making very difficult to differentiate the robot end effector from the background, hence limiting the tracking capabilities. The work related in this paper explores for the first time the potential value of Convolutional Neural Networks (ConvNet) in the context of nanorobotic vision tracking inside SEM. The aim is to localize an endeffector, AFM cantilever in the case of the study, from SEM images for any scan speed configuration and despite of low images quality. For that purpose, a data set of AFM tip images is build up from SEM images for the learning algorithm. Network performances are estimated under different SEM scan speeds. Thanks to the learning algorithm, experimental results show robust AFM tip tracking capabilities inside the SEM under various scan speed conditions.

Siamese Convolutional Neural Network for Submillimeteraccurate Camera Pose Estimation and Visual Servoing Visual Servoing (VS), where images taken from a camera typically attached to the robot endeffector are used to guide the robot motions, is an important technique to tackle robotic tasks that require a high level of accuracy. We propose a new neural network, based on a Siamese architecture, for highly accurate camera pose estimation. This, in turn, can be used as a final refinement step following a coarse VS or, if applied in an iterative manner, as a standalone VS on its own. The key feature of our neural network is that it outputs the relative pose between any pair of images, and does so with submillimeter accuracy. We show that our network can reduce pose estimation errors to 0.6 mm in translation and 0.4 degrees in rotation, from initial errors of 10 mm  5 degrees if applied once, or of several cm  tens of degrees if applied iteratively. The network can generalize to similar objects, is robust against changing lighting conditions, and to partial occlusions (when used iteratively). The high accuracy achieved enables tackling lowtolerance assembly tasks downstream: using our network, an industrial robot can achieve 97.5% success rate on a VGAconnector insertion task without any force sensing mechanism.