## 2 Methods
### Slip Detection
#### 2.3.3 Optical Path Occlusion and Remaining Background Noise  
To address potential optical path occlusion, a voting system was implemented (Algorithm 1). Pixels within the ellipse that estimates the object dimensions and have a magnitude of zero were excluded from consideration. This step was necessary to enable the voting system to function effectively on minimally textured objects, where large portions of the objects might not exhibit visual changes, even during slip events. These regions needed to be excluded to prevent false positives. Subsequently, the pixels were sorted based on their optical flow (OF) magnitude, and a threshold was set. The majority of the pixels needed to surpass this threshold for a slip event to be qualified as such. The voting system allowed for optical path occlusion by up to 50% of the grasped object's size. Even with such occlusions, the slip detection algorithm continued to work efficiently and still detected slip events accurately (Figure 5). There was a trade-off in determining the necessary majority of pixels that must vote for slip detection. Setting a lower percentage made the algorithm more sensitive to occlusions, potentially leading to false positives triggered by occlusions themselves. However, this lower percentage also required less of the object to be unoccluded in order to detect slip. ```
\(MAJORITY=0.5\) while True do \(flow=calcOpticalFlowFarneback\) \(non\_zero\_OF=flow>0\) \(inv(sort(non\_zero\_OF))\) \(majority=sorted[0:length*MAJORITY]\) if\(majority[end]>THRESHOLD\)then \(slip=True\) elseif\(majority[end]<THRESHOLD\)then \(slip=False\) endif endwhile

Figure 5: A pencil occluding the optical path of the camera to the object without causing slip to be detected. If an actual slip case would occur, more than 50% of the non-zero-magnitude pixel would show optical flow (OF) above the threshold, triggering slip to be detected even if the obstructing object is still. On the other hand, a higher percentage reduced the possibility of false positives caused by occlusions, as a larger portion of the object needed to be unoccluded for slip to be detected. However, this also made slip detection more challenging in cases where the occlusion in front of the object was relatively small. The approach described has its inherent limitations, which are typical for vision-based methods. One potential solution to address the occlusion problem could involve using multiple cameras from different angles to obtain a more comprehensive view of the scene. However, in the specific use case described, the use of multiple cameras from different angles might be impractical or infeasible. #### 2.3.4 Dealing with Compliant Objects

One of the major challenges of using visual data and optical flow for slip detection lies in dealing with compliant objects. When grasping compliant objects, the grippers deform the object, leading to additional optical flow, which may trigger false positives in the algorithm. The grippers move only in one dimension relative to the camera, causing the optical flow resulting from grasp-induced deformations to be primarily horizontal, while the optical flow caused by slip (translational and rotational) is mostly vertical (Figure 6). The Farneback optical flow (OF) method not only estimates the OF magnitude for each pixel but also the angle of the flow. These flow angles were utilized to calculate the mean direction of the slip that triggered the detection. To achieve this, the angles were first converted from polar coordinates to unit vectors. This conversion was necessary to prevent false calculations of the mean angle around 0 or \(2\pi\). Without converting to unit vectors, the general direction of OF angles pointing to the right could erroneously result in the mean direction pointing to the left. Furthermore, OF angles falling within the horizontal alignment range of \(\pm 25^{\circ}\) (Figure 7) were excluded from being classified as slip.