# Vision-Based Robotic Grasping of Reels for Automatic Packaging Machines
## 1 Introduction
### Use Case: Reel-Picking for Automatic Packaging Machines  
In automatic packaging machines, raw material often comes in the form of reels, wrapped around a cardboard core of standardized dimensions (e.g., diameter and thickness). Although the sizes of the core are known, in a real production context the overall diameter of the reel and the raw material texture and color may differ from one batch to the other.  
In the use-case analyzed for this work, three types of reels are employed, whose main features are reported in Table 1. The reels are stocked in four semi-horizontal piles on a mobile deposit, called \"wagon\", as shown in Figure 1. At the beginning of each shift, the operator brings the wagon to the warehouse, loads it with as many reels as it can store, and moves it back to its original position on the shop floor. A mobile robot is then in charge of picking a single reel and loading it onto the automatic machine that requested it. The mobile robot is equipped with a robotic arm, an electrical gripper with custom fingers for inside grasping, and an eye-on-hand industrial camera [20]. An industrial PC is also installed onboard the mobile robotic platform and takes care of reading the camera stream, elaborating the incoming images, and communicating with the robot through REST APIs (web Application Programming Interfaces -APIs- or services conforming to the REpresentational State Transfer -REST- architectural style).  
The objective of this work is the development of a robust vision-based algorithm that, employing standard CV techniques and math libraries, can infer from a single image the 3D pose of a reel core with respect to the camera frame. With this information, and knowing from calibration the position of the camera with respect to the end-effector of the manipulator, it is then possible to express such pose in the robot base frame. Finally, a suitable target configuration for the robotic arm may be calculated to insert the gripper fingers into the core, and perform the grasping of the reel (from inside the core).  
The specifications given for this industrial task are:  
1. finger insertion in reels with ID 1 and 3 (see Table 1) require millimetric accuracy due to the limited gap between the fingers and the inner diameter of the core (e.g., 9 mm);
2. it must work with gray-scale 1544 \(\times\) 2064 p images;
3. it must run on an industrial workstation running Ubuntu Server 18.04 LTS operating system and limited computing power (e.g., Intel(R) Core(TM) i5-6440 EQ CPU @ 2.70 GHz, 8 GB RAM, no GPU, no graphics card); and
4. it must execute within 3.0 s.  
The proposed solution relies on a reel-based selection policy of three different methods to extract the contour of the reel core framed from a single image. Next, by exploiting the knowledge of the core's inner/outer diameter dimension, the algorithm infers its 3D pose with respect to the camera.  
The article is structured as follows. Section 2 describes how to obtain the 3D pose estimate of a reel from a digital ellipse that represents the contour of its core in a picture. In Section 3, three methods to obtain such an ellipse from a single image are proposed, all of  
\begin{table}
\begin{tabular}{l c c c} \hline \hline
**Reel Type** & **ID** & **Core Diameter (Inner/Outer)** & **Thickness** \\ \hline _Paper_ & 1 & 77/85 mm & 100 mm \\ _Tags_ & 2 & 158/166 mm & 30 mm \\ _Outer Envelope_ & 3 & 77/85 mm & 100 mm \\ \hline \hline \end{tabular}
\end{table}
Table 1: Reels features.  
them based on known techniques suitably put together. In Section 4, an effective method-selection policy is described that optimizes the results that are presented in Section 5, obtained from the data collected in an experimental campaign. Lastly, in Section 6, conclusions are drawn.