# Vision Systems for a UR5 Cobot on a Quality Control Robotic Station
## 3 Vision-Based Measurement Algorithm
### Camera Calibration and Camera-to-Robot Calibration
#### 3.2.3 Vision-Based Measurement Algorithms  
The primary core of the developed software tool encompasses two image preprocessing and analysis algorithms. The first was implemented to calculate the location of a semi-finished product in a cobot system, and the latter is for geometric measurements.  
While the station is operating (Figures 1 and 2), two vision-based measurement algorithms are run; the first involves determining the position and orientation of semi-finished products (parts) placed in the cobot's picking zone. The latter deals with the detection of parts and basic measurements of quantities such as width, height, diameter (in the case of circular objects), and surface area, considering the internal contours of the parts.  
In the case of both vision algorithms, the image preprocessing stage is similar: image conversion to grayscale, thresholding, and detection of contours in the binary image. The diagram below (Figure 14) shows the subsequent steps of the algorithm implementing the process of determining position and orientation and measuring fundamental geometric quantities. The results of the vision measurements are obtained using subpixel methods. The final stage involves converting the obtained values to SI units in mm.  
For the USB camera, the first vision algorithm responsible for determining the position and orientation of semi-finished products is expressed below:  
Figure 14: Developed algorithm, implementing vision measurements based on OpenCV libraries [63; 64; 65].  
Step 1--image preprocessing before contour detection; it includes conversion to grayscale in the case of a color image, smoothing using a Gaussian filter, thresholding, and bit negation for contour detection;  
Step 2--detection of external contours in a binary image using the cv.findContours() function (topological structural analysis is applied to detect contours);  
Step 3--elimination of contours that do not meet the adopted parameters for the extracted image features;  
Step 4--comparison of extracted contours with a defined pattern using Hu moments (cv.matchShapes());  
Step 5--determination of the centroid and orientation angle based on first and second-order moment methods;  
Average operation time: 349 [ms].  
Below is an example result of the operation of the described method (Figures 15 and 16).  
For the Mako camera, the second vision algorithm responsible for measuring the basic geometric features of semi-finished products is as follows:  
Step 1--image preprocessing (conversion to grayscale, image smoothing--Gaussian filtering, thresholding, bit negation);  
Step 2--detection of edges in an image using the Canny algorithm cv.Canny();  
Step 3--detection of external contours by applying the cv.findContours() function, which retrieves contours from the binary image using topological structural analysis;  
Step 4--determining the area of the contours obtained in the previous step with cv.contourArea(). The area is computed using the 'Green' formula. Rejection from further analysis of regions that do not satisfy the established criteria (given range of areas);  
Step 5--polygonal curves approximating, cv.approxPolyDP() and determining figures (rectangles) inscribed on detected shapes cv.boxPoints();  
Step 6--detection of internal contours in a group of objects separated from the image in the previous operation (Figure 16) and determining the points spanning the rectangle circumscribed on the contour extracted;  
Step 7--applying the mechanism presented in the previous steps, determining the basic geometric sizes of the separated objects.  
Figure 15: The result of the vision algorithm computing the position and orientation of the object, with the centroid and the X (red) and Y (green) axes marked. The white ellipse represents geometric moments. Green border—detected outer contours.