# A Vision-Based Micro-Manipulation System
## 3 System Overview
### Visual Recognition  
Individual living cell image recognition and localization is determined by finding the outline and coordinates of a cell. A state-of-the-art deep learning algorithm YOLOv7 [6] is used, which performs the real-time process of recognition and localization of cell contours and centers. Its architecture, as illustrated in Figure 4, consists of a backbone network, neck network, and head network. The backbone network extracts features from the input image, while the neck and head networks refine the feature maps to generate object recognition of living cells. The algorithm utilizes anchor-based object detection, where anchors are predefined bounding boxes of various sizes and shapes. This enables the network to detect living cells of different shapes and sizes accurately. YOLOv7 also uses a feature pyramid network (FPN) that extracts multi-scale features from the input image. The FPN combines feature maps of different resolutions to generate a feature pyramid, which enables the network to detect living cells at different scales. The network detects living cells in two stages using anchor boxes and the predicted objectness score. YOLOv7 learns to predict the bounding boxes and class labels of living cells from ground truth annotations during training, making it highly advanced and efficient for detecting living cells.  
Figure 3: Image enhancement process of proposed FACE method.  
Utilizing the Contrast Enhancement (FACE) method for image enhancement notably elevates the quality of microscopic images by focusing on contrast improvement. When employed on microscopic images, these enhancement techniques serve as a pivotal element in augmenting object detection accuracy, especially when utilizing YOLOv7. This results in a more resilient and trustworthy detection process. To enhance the precision and robustness of automated yeast cell detection, the system employs a dual dataset approach. One dataset comprises yeast cell images that have undergone contrast enhancement using the FACE method, while the other dataset consists of the original, unaltered cell images. These two datasets are simultaneously integrated into the YOLOv7 model. The FACE-enhanced dataset is utilized for model training, where these images are fed into the model's architectural components, including the backbone, neck, and head, to generate the necessary output. This process also results in the acquisition of training weights and model information. Conversely, the original dataset is deployed for predictive purposes. Leveraging the unaltered dataset reduces prediction time, as there is no need for contrast enhancement. This approach enables real-time detection of yeast cell contours and precise positioning. The complete process of the FACE-YOLO detection mechanism is illustrated in Figure 5.  
Figure 4: Yolov7 architecture.  
Figure 5: FACE-YOLO hybrid enhancement detection model.  
Initially, the original yeast cell images undergo enhancement using the FACE image enhancement method to improve the clarity of cell contours, making them more suitable for recognition. Subsequently, YOLOv7 is employed for training on these enhanced yeast cell images. Following the training process, the original yeast cell images are fed into the model, which produces images containing YOLO-detected results. The post-processed images obtained from YOLO detection are used to create masks for each cell, and individual cells are then extracted. Through binary processing, OpenCV is employed to identify the contours and centroids of the cells. Finally, these contours and centroids are superimposed onto the original images, displaying the contours and centroids of all yeast cells in the images, thereby facilitating subsequent cell manipulation with a gripper. The complete step-by-step image transformation and cell detection process is illustrated in Figure 6.