# A Vision-Based Micro-Manipulation System
## 4 Tests and Results
### Overall System  
The combined system was tested to prove the feasibility of the concept, so practically no fine-tuning was performed for improved precision. It was set up and tested according to the inference module in the architecture shown in Figure 1. The micro-manipulation system, already described in Section 3.4, was combined with a separate computer running the object detection. It was running on Ubuntu Linux 22.04 with an Intel i7 12 core 2.60 GHz CPU and 16 GB of RAM.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Fold** & **1** & **2** & **3** & **4** & **5** & **6** & **7** & **8** & **9** & **10** & **Avg.** \\ \hline Mask R-CNN & 0.89 & 0.93 & 0.92 & 0.90 & 0.89 & 0.91 & 0.81 & 0.92 & 0.91 & 0.89 & 0.89 \\ \hline YOLOv5 & 0.84 & 0.92 & 0.90 & 0.94 & 0.91 & 0.95 & 0.98 & 0.97 & 0.95 & 0.94 & 0.94 \\ \hline YOLOV7 & 0.86 & 0.92 & 0.95 & 0.94 & 0.97 & 0.98 & 0.98 & 0.96 & 0.99 & 0.97 & 0.95 \\ \hline \hline \end{tabular}
\end{table}
Table 4: 10-fold accuracy comparison.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Type** & **Label** & **Mask R-CNN** & **Yolov5** & **Yolov7** \\ \hline Results & & & & & \\ \hline Number of Detections & 5 & 3 & 2 & 5 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparison of different object detection models.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \multicolumn{5}{c}{**Positioning Error**} \\ \hline \multirow{3}{*}{**Axes**} & **Motor** & **Drive** & **Start** & **Average** & **Standard** \\  & **Temperature**, & **Temperature**, & **Point** & **Error, mm** & **Deviation**, \\  & **°C** & **°C** & & **°C** \\ \hline \(X\) & 42 & 28 & 1.3935 & 0.0001 & 0.000539 \\ \hline \(Y\) & 59 & 34 & 1.2520 & 0.0012 & 0.000872 \\ \hline \(Z\) & 44 & 28 & 1.7297 & 0.0003 & 0.000458 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Micro-manipulator test results.  
To bridge the gap between camera and robot coordinates, a micro-scale grid with known grid size was used, visible in Figure 9[48]. With a known mapping from the camera to the robot, the detected object coordinates could be transformed into movements for the robot. In our approach, the center of the camera's vision was set as the goal for movement. After moving the micro-manipulator, the center of the cell of interest should coincide with the goal. The sample size does not permit a situation where the microscope is placed with the whole sample in the FOV, so a single home pose, from which the whole workspace could be seen, is not viable. For this reason, it was chosen to send relative movement coordinates to the manipulator. Whenever a cell was chosen for inspection, the system would transform the center of the detected cell as an X and Y distance for the micro-manipulator to move, relative to its current position. An example of moving to a detected cell can be seen in Figure 10.  
No specific description for choosing the cell to inspect was posed, which means that they could vary in sizes significantly. Five cell center movements, of which all cells were certainly similar sizes, around 3 \(\upmu\)m, can be seen visualized in Figure 11. These examples show an average offset of 0.5862 \(\upmu\)m, which is deemed precise enough to allow the tool to land in the area of the cell for further inspection or manipulation.  
Figure 10: Example of moving the micro-manipulator to a detected cell. Light blue—center of vision; green and dark blue—center and bounding box of chosen cell.  
Figure 9: Micro-scale grid used for calibrating.