# Design and Validation of a Camera-Based Safety System for Fenceless Robotic Work Cells
## 3 Design of the Image-Based Safety System
### Combining Images from Multiple Cameras  
In safety systems created with a single camera, it is predicted that blind areas may occur in the areas overshadowed by the robot arm. This situation can be seen in Figure 7. Therefore, it is recommended to use at least two cameras symmetrically to each other in all systems installed. Thus, blind areas are prevented. One of the problems encountered in the safety system with multiple cameras is a combination of multiple images. A method of combining two images from two cameras at a distance from each other is stereo imaging, which is a relatively mature technology. Stereo imaging brings the advantage of three-dimensional perception at a significant computational cost, yet blind spots still exist. However, the proposed 2-D image stitching is computationally more economical and does not leave any blind spots. Thus, experiments were conducted to prove the elimination of blind spots in the work cell. Another important criterion in combining multiple camera images is the formation of more than one image of an object entering the safety space when scanning the same regions. This creates complexity in determining the number of pixels to detect. The second criterion is that multiple scans of the same areas with image processing algorithms require an additional processing load for the control unit. Since the fastest possible reaction is  
Figure 7: A blind area that may form when using a single camera. essential for the safety system, the additional load on the controller will be disadvantageous. Considering these criteria, partial masking of the image taken from more than one camera was proposed. For each camera, the region where there is no blind spot was processed, and the masking of the outlying zone where the blind spot could occur was performed, and thus, blind spots were avoided. As a result, processing the same zones more than once was prevented. Compared with the image-combining algorithms in the literature, a unique method that can be integrated into the safety system and work more efficiently was used. Figure 8 shows a schematic of a circular work cell placed on the image pixels of a two-camera system. Partial masking was applied to the upper half of the images taken from Camera 1 and Camera 2. A single image was produced by combining unmasked partial images with no blind spots. Figure 9 shows a more detailed 3-D perspective schematic of the image stitching proposed. The mono image developed from the unmasked portions of multiple images is naturally not photo realistic, but it is sufficient to detect a foreign object penetrating the safety zones of a work cell. When three or more cameras are used, pie slice images at 120\({}^{\circ}\), 90\({}^{\circ}\), 72\({}^{\circ}\), etc. can also be stitched similarly. Figure 10 shows a schematic of the monitoring system. The digital data from the cameras were processed in the image processor. As a result of the processed data, the reaction desired to be applied in the scenario was transmitted to the robot arm via the robot controller. Decelerate or stop command was sent back by the processor to the robot arm within a response time. Figure 8: Combining partially masked images of two symmetrical cameras based on each pixel. As a result of the optimization problem, a solution set or a solution point was obtained. In optimization problems involving multiple constraints, fractional programming is a beneficial method to find a practical solution set so that the decision maker can choose the most suitable one among the alternatives in the solution set. Frame frequency (measured in frames per second, FPS) is the rate at which consecutive images, called frames, appear on display. Capturing and analyzing images from the camera  
Figure 10: The control schematic of the safety system. Figure 9: Combining different partially masked real-time camera images. is directly related to frame frequency because it directly affects the operating time of the system.