## 2 Materials and Methods
### Methods
#### 2.2.3 Prediction of Reference Points  
Once the offset of the camera alignment with the robotic arm was calculated, the reference points were predicted. Using the previously created socket, the robotic arm was moved (with a velocity of 2 m/s and an acceleration of 2 m/s2) to the following joint positions using the movej() function [36]: base = 90\({}^{\circ}\); shoulder = 0\({}^{\circ}\); elbow = 0\({}^{\circ}\); wrist 1 = -90\({}^{\circ}\); wrist 2 = 90\({}^{\circ}\); wrist 3:240\({}^{\circ}\);  
This movement of the robotic arm has been performed so that the camera can take a frame of the person's back without the arm being in front of them. The person should be approximately in the center of the image. Taking the frame from the person's back, as illustrated on the left side of Figure 7, it is then resized to 512 \(\times\) 512 \(\times\) 3 pixels, since this is the input of the model. Then, the prediction model for image segmentation in humans, already trained and available on the Internet, was used. This model predicts a mask that corresponds to the outline of the person, and with this, it is possible to perform the extraction of the image background. The mask predicted by this model (contour of the person), presented on the right side of Figure 7, will be used later to calculate the center of this same contour. Next, the keypoints, shown in Figure 8, were predicted using the previously trained model. Figure 6: Calculation of the alignment offset between the robotic arm with the camera. On the left (**A**), photographic image frame, with the middle of the gripper represented by a green dot (center of the green circle). On the right (**B**), depth image frame with the middle of the gripper (green dot) represented by a square, with related spatial coordinates, near bottom right corner vertex. 2.2.4. Calculation of the Points to Be Ausculated  
Once the key points on the person's back were obtained, the points to be auscultated are calculated. For this, we used the previewed image with the key points, the mask with the body contour, and the frame of the person's back. First, the unwanted points that were erroneously predicted in the image with the key points (in other words, the noise) were removed. Morphological transformations were used for this, namely image opening. Image opening is the name given to an erosion followed by dilation transformation. These transformations were performed using the morphologyEx() function. Next, the 2D coordinates of the reference points (key points) of the person's back were obtained using the SimpleBlobDetector() function with the following parameters: filterByColor = True; blobColor = 255; minArea = 0.1; minCircularity = 0.01; minConvexity = 0.01; minThreshold = 1; minInertiaRatio = 0.01. In addition, the center of the largest contour present in the mask with the contour of the person was also obtained. This center should correspond to the center of the person's contour. The key points are stored in different variables, since the SimpleBlobDetector() function stores all these points in one variable. They are also separated into right side or left side, considering the previously calculated center. These points are then also sorted considering the value of the \(y\) coordinate. After the keypoints are all sorted and stored, the following computations are performed, depending on the key points which are found (see Figure 9):  
* In the axilla area (\(S1\) and \(S2\)):
* is computed the midpoint between points \(S1\) and \(S2\) (\(S\_mid\)), and the horizontal length of the back through the \(x\) coordinate of these points (difference between these two points) (\(dist_{x}\));
* In the neck area (\(P1\), \(P2\)):
* is computed the midpoint between points \(P1\) and \(P2\) (\(P\_mid\));  
Figure 8: Keypoints on the person’s back, represented with white circles. Labels attributed according to: P for neck, S for armpit, C for waist, 1 for left side and 2 for right side. Figure 7: Frame of the person’s back with and without background, respectively.