## 2 Methods
### Slip Detection
#### 2.3.3 Optical Path Occlusion and Remaining Background Noise
This exclusion allowed the algorithm to detect possible slip cases, both translational and rotational, while effectively avoiding the classification of OF caused by the deformation of the object due to gripper action as slip. The conversion from polar coordinates to a unit vector for all pixels was computationally expensive and unnecessary for estimating the general direction of the flow. Instead, a more efficient approach was adopted, wherein only the 10,000 pixels with the highest magnitude of optical flow were used to calculate the flow direction (Algorithm 2). ```
\(MAX\_CALCULATIONS=10,000\) sort_by_magnitude(OF_angles) \(limit\_size(OF\_angles,MAX\_CALCULATIONS)\) \(unit\_vectors=(cos(OF\_angles);sin(OF\_angles))\) mean = mean(unit_vectors) direction = normalize(mean)
```

**Algorithm 2** Flow direction algorithm

Figure 6: The original image with the approximated image boundaries and slip direction (**left**) and the HSV representation of the OF direction (**right**). The hue represents the direction of the flow, and the value represents the magnitude. A typical slip case: the green color indicates an angle of 90\({}^{\circ}\), which in this case shows the object slipping downwards, which is indicated by the arrow on the left image. #### 2.3.5 Dealing with False Positives

An essential aspect of slip detection is the subsequent reaction of the robot to prevent the slip of the object and maintain a stable grasp, thereby preventing the object from falling out of the gripper. Achieving the fastest possible reaction time requires careful consideration of several factors. Minimizing computation delay is crucial, which means avoiding processes that introduce delays, such as temporal filtering. Addressing noise introduced by external influences is also important. To strike a balance, the proposed method applied a threshold that had to be chosen carefully. It needed to be low enough to detect slip cases while remaining high enough to avoid classifying noise as slip, reducing the occurrence of false positives. However, setting such a threshold might prevent the detection of slip cases with low magnitudes, as they could be lower in magnitude than the noise. To address these conflicting requirements, the algorithm utilized multiple temporal layers of slip detection. Instead of solely comparing the most recent frame with the previous one, the most recent frame was also compared with past frames at increasing temporal distances. This approach amplified the magnitude of slow slip cases, allowing them to reach the threshold. However, this amplification also affected noise. To mitigate this effect, the magnitude of the optical flow was filtered with a first-order Butterworth filter. The filter had a sampling frequency of 1000 Hz and a cutoff frequency of 20 Hz. It introduced a delay dependent on the camera's frame rate (Equation (2)). In the case of the RealSense D435 camera operating at 30 fps [15], this resulted in an additional delay of approximately 0.033 s for the extra layers. \[\eta=\frac{temporal\ distance}{frame\ rate} \tag{2}\]

The layering approach introduced an additional 0.03 s of delay per frame in temporal distance. Although the added delay was undesirable, it was primarily introduced for the detection of slip cases with very low magnitudes. In this context, the accuracy of slip detection took precedence over the reaction time. For the proposed method, a total of four layers of optical flow calculation were added to the algorithm, comparing the current frame to frames at 1, 2, 4, and 8 time steps in the past. This temporal layering allowed the algorithm to effectively detect slip events of varying magnitudes while accounting for potential noise and occlusions. The layers are visually represented in Figure 8. The initial comparison of the current frame with the preceding frame introduced the minimally necessary reaction delay, which was dependent on the frame rate and resulted in approximately 0.03 s of delay, along with the time for the algorithm's calculations, which were negligible.