## 3 Method
### Computer Vision Module
#### 3.3.3 Grasp Detection  
The last part of the computer vision pipeline detects how to grasp the target object, i.e., the object that the user is interested in interacting with. The experimental setup included two objects for the user to interact with: a strawberry and a bottle. A simple rule-based strategy relying on fitting cylinders [23] was hence used for the grasp detection. First, the detection from earlier (Section 3.3.1) was used to mask the depth information from the RGB-D camera such that the result was a point cloud of the target object, as shown in Figure 3. An RANSAC-based algorithm [33] was then used to fit a cylinder to the masked point cloud [34], resulting in both the position and orientation of the object (the central axis of the cylinder) along with its approximate size (the cylinder diameter and height).  
The fitted cylinders were then converted into a grasp pose for the exoskeleton, as illustrated in Figure 5, which depicts how the coordinate frame of end-effector, i.e., the Carbonhand, should be positioned and oriented in order to grasp the two objects on the table. The frame of the end-effector is placed in the palm of the Carbonhand and oriented such that the \(z\)-axis is pointing out from the palm, the \(y\)-axis points upwards, and the \(x\)-axis points towards the thumb of the Carbonhand.  
A rule-based approach is used for grasping the objects such that:  
* **Position--**The coordinate frame of the end-effector should be positioned halfway along the height of the cylinder, such that the object is grasped in the middle for stability. Furthermore, the position for grasping the object should be on the outer perimeter of the cylinder to avoid pushing the object away. An offset, equal to the radius of the detected cylinder, is applied in the direction towards the end-effector to avoid this.
* **Orientation--**The coordinate frame of the end-effector should be oriented such that the \(y\)-axis is parallel with the axis of the cylinder, while also pointing upwards to avoid infeasible grasping orientations (such as trying to grasp the object with the palm of the  
Figure 4: Example of the intent prediction using the Euclidean distance from the detected objects \((A,B,C)\) to a ray \(\vec{R}\) projected from the palm of the user’s hand in the exoskeleton. The example is shown in a plane (2D) for simplification.  
hand facing away from the person in the exoskeleton). Furthermore, the \(z\)-axis should be orthogonal to the axis of the cylinder to avoid grasping the object at a skewed angle.  
Furthermore, smaller objects (height less than \(\approx\)3 cm) are difficult to grasp using an upright orientation (such as the orientation of the Carbonhand depicted in Figure 5). This is partly due to the design of the Carbonhand, where only the thumb, middle finger, and ring finger are actuated. An additional check is therefore implemented in the grasp detection, such that all objects fewer than 3 cm in height will be approached as a cylinder laying flat on the table, as also illustrated for the small strawberry in Figure 5. Finally, it should be noted that a cylinder is a poor fit for a strawberry. Nevertheless, the above approach was found to produce an acceptable grasp for both the bottle and the strawberry.