# CFAM: Estimating 3D Hand Poses from a Single RGB Image with Attention
## 3 Method
### 2D Keypoint Calculation  
In this section, we introduce _HandSegNet_ to obtain the bounding box of the hand from the input image and _PoseNet_[16] to get the 2D keypoints.  
We use \(J\) to represent the different hand keypoints; \(J=\{1,21\}\) since we found 21 useful keypoints on one hand. \(W=\left\{w_{I}=(x,y,z),J\in[1,21]\right\}\) represents the 3D position of each hand keypoint. The input \(I\in R^{\texttt{inv}\texttt{Loc}\texttt{Loc}\texttt{3}}\) is the RGB image. The picture of the cropped hand mask is \(I_{\text{mask}}\in\mathbb{R}^{\texttt{inv}\texttt{Loc}\texttt{3}}\), which is smaller than the whole input \(I\) and includes only the hand mask. \(R=\left(R_{x},R_{y},R_{z}\right)\) represents the rotation angle of the camera coordinate system relative to the world coordinate system. We use 2D Gaussian keypoint score maps \(P=p_{I}(u,v),J\in[1,21]\) to present the 2D keypoints, where each score map corresponds to one keypoint. \((u,v)\) is the position of the keypoint where the Gaussian score map \(p_{I}(u,v)\) is centered. It is beneficial for the network to learn possible positions of the keypoints during the training process.  
2D keypoint calculation is key to 3D keypoint estimation. To calculate the 2D keypoints, _HandSegNet_ is first used to estimate the region of the hands, namely, _handmask_\(I_{\text{mask}}\), from the original image \(I\in\mathbb{R}^{\texttt{inv}\texttt{Loc}\texttt{3}}\).  
The first neural network used in this method is _HandSegNet_, whose task is to crop the hand region from the image. Directly cropping the horizontal and vertical coordinates of the hand region to obtain a rectangular block is a regression problem. Neural networks are not as good at regression tasks as they are at classification tasks [2]. Thus, we first get the mask of the hand and regard our goal as a task for computing bool images. Next, we determine whether each pixel in the image belongs to the area of the hand. For each pixel, we calculate the probability (\(P_{i}\)) that it belongs the hand mask. When the probability is greater than a threshold, the point is considered to belong to the hand mask.  
Figure 2: The framework of our method. The process is divided into three steps. First, the RGB image (**a**) is brought into the the network as the input. The hand area (**b**) is cropped from the whole RGB image using _HandSegNet_[16]. Then, the 2D score map (**c**) is estimated according to the hand area. Finally, 3D coordinates (**d**) are estimated from the 2D score map and cropped hand area by _PosePrior_.  
Then, the center of mass of the hand mask is calculated. The hand area is cropped around the center of mass.  
Then, \(I_{mask}\) is fed into _PoseNet_ to estimate score maps of different 2D keypoints. The 2D keypoints are estimated from the cropped hand images. Traditional methods directly predict the \(x\) and \(y\) values of each keypoint, which ignores the connection between the fingers and is a task at which the neural network is not good. We do not just estimate the \(x\) and \(y\) values of the keypoints like traditional methods; instead, we obtain a score map of each keypoint, such as that shown in Figure 2c. The score map also represents the location of the keypoint, and it can be understood better by neural network.