# Robot Operating System 2 (ROS2)-Based Frameworks for Increasing Robot Autonomy: A Survey
## 5 Example of Using the Framework to Increase Cobot Autonomy: A Proof of Concept
### Perception Layer Implementation  
The perception module is the starting point of the implementation. It is composed of two nodes: the _Camera Driver_ for the environment perception and the _Cobot Driver_, which, in this layer, is responsible for the cobot state acquisition. The first exploits ROS2 packages provided by camera manufacturers in a GitHub repository to use the Intel RealSense D400 camera series [116], to which the D435i model used here belongs. This driver was used both as an interface for the depth camera, to easily configure it with a list of available parameters, to acquire RGB and depth images, and as an interface for the rest of the application, to publish these data in the standard ROS topics _Image_ and _PointCloud2_. The example in  
Figure 14: Software architecture. Figure 15: Software flowchart. Figure 4 is obtained using this _Camera Driver_ with the Intel RealSense D435i camera. The _Cobot Driver_ node instead implements the interface towards the TM5-900, and also, in this case, it leverages the dedicated ROS2 driver provided by the cobot manufacturer as a GitHub repository [117]. Among the functionalities offered by this driver, there is the publishing of the standard _JointState_ topic, based on joint data acquired from the cobot using the already-mentioned socket TCP server _Ethernet Slave_ with the dedicated application-level protocol _TMSTA_ once the _Data Table_ is appropriately defined from cobot side. Therefore, the implementation of the perception module became quite simple due to the availability of ROS2 Foxy-compatible drivers for both the camera and the robot. Otherwise, it would have been more complex, given the need to develop them from scratch. Of further interest is the development of the recognition module, at least as far as the environment around the robot is concerned. In fact, the state of the cobot is handled in the standard way already discussed by its driver, which also incorporates the URDF model of the robot. Environment reconstruction, on the other hand, is not handled by the MoveIt plugin in _PointCloud Occupancy Map Updater_, but is performed by manually adding obstacles to the planning scene based on their size, orientation, and center position information that can be extracted from camera images once their shape is detected with OpenCV libraries. Indeed, OpenCV is used to process the RGB image acquired when the robot is in _Photo_Position_ firstly to find the obstacle contours, using the already-mentioned _apply_canny_ and _findContours_ functions. Subsequently, both the pixel dimensions and orientation of its smallest bounding rectangle are calculated with the _minAreaRect_ function, while the pixel corresponding to the obstacle centroid is obtained using image moments given by a different OpenCV function (i.e., _moments_). The position of the obstacle's center in the cobot reference frame is calculated from the pixel coordinates of the detected centroid, first determining the spatial position x, y, z on the camera frame of the corresponding pixel in the point cloud, and then using Tf2 to change the reference frame. A more complex task is to find the size and orientation of the obstacle from the bounding rectangle returned by the OpenCV function. Regarding sizes, it is necessary to convert them from pixels to a distance measure (e.g., centimeters) using both the information about the obstacle's distance from the camera (i.e., the z coordinate of its center position in the camera frame) and the size (centimeters) of the single pixel in the image acquired at a predefined distance. Finally, the orientation of the obstacle in the cobot's frame is obtained by the angle of rotation of the bounding rectangle returned by the same _minAreaRect_ function (i.e., the angle between the rectangle base and the horizontal axis).