# A Comprehensive Study on Deep Learning-Based 3D Hand Pose Estimation Methods
## 4 Datasets & Metrics
### Datasets
\begin{table}
\begin{tabular}{l l l l l l l l} \hline \hline
**Dataset** & **Modality** & **Year** & **View** & **Type** & **\# of Subjects** & **\# of Joints** & **\# of Frames** \\ \hline ICVL [41] & D & 2014 & 3 rd & Real & 10 & 16 & 332.5 K \\ NYU [42] & D & 2014 & 3 rd & Real & 2 & 36 & 81 K \\ MSRA15 [63] & D & 2015 & 3 rd & Real & 9 & 21 & 76.5 K \\ HandNet [138] & D & 2015 & 3 rd & Real & 10 & 6 & 212 K \\ BigHand2.2M [139] & D & 2017 & ego/3 rd & Real & 10 & 21 & 2.2 M \\ HANDS 2017 [45] & D & 2017 & ego/3 rd & Real & 20 & 21 & 1.2 M \\ SynHandSM [92] & D & 2018 & 3 rd & Synthetic & - & 22 & 5 M \\ FreitAND [140] & RGB & 2019 & 3 rd & Real & 32 & 21 & 134 K \\ Dexter1 [79] & RGB+D & 2013 & 3 rd & Real & 1 & 6 & 2 K \\ Dexter+Object [141] & RGB+D & 2016 & 3 rd & Real & 2 & 5 & 3 K \\ RHD [43] & RGB+D & 2017 & 3 rd & Synthetic & 20 & 21 & 44 K \\ STB [44] & RGB+D & 2017 & 3 rd & Real & 1 & 21 & 18 K \\ EgoDexter [136] & RGB+D & 2017 & ego & Real & 4 & 5 & 3190 \\ SynthHands [136] & RGB+D & 2017 & ego & Synthetic & 2 & 21 & 63.5 K \\ \hline \hline \end{tabular}
\end{table} 
Table 1: A representative catalogue of 3D hand pose estimation datasets. comprises additional 5 unseen subjects. The pose annotations for each image are the positions of 21 hand joints. **SynHand5M**[92] is a million-scale synthetic dataset containing 5 M depth images, 4.5 M for training and 500 K for testing, with a resolution of \(320\times 240\). It provides annotations of 22 3D joint positions defined on 26 DOFs and 1193 3D hand mesh vertices, as well as joint angles and segmentation masks. **FreiHand**[140] is a multi-view RGB dataset containing hand-object interactions. In total, it encompasses 134 K samples at \(224\times 224\) resolution, 130 K for training and 4 K for evaluation. Ground-truth 3D locations for 21 keypoints of the hand, as well as the 3D hand shape, the hand mask, and the intrinsic camera matrix, are provided. **Rendered Handpose Dataset (RHD)**[43] consists of 43,986 rendered hand images, 41,258 for training and 2728 for evaluation. Totally, there exist 39 actions performed by 20 characters, with \(320\times 320\) resolution. For each image depth map, segmentation mask and 3D and 2D keypoint annotations are provided. This dataset is considered to be exceedingly challenging due to the fact that it contains large variations in viewpoints and hand shapes, as well as occluded fingers, large visual diversity, and noise. **Stereo Hand Pose Tracking Benchmark (STB)**[44] comprises one subject performing 12 sequences with 6 different backgrounds. Altogether it includes 18 K frames, 15 K for training and 3 K for testing, with a resolution of \(640\times 480\). Since 3D keypoint annotations are provided, 2D keypoint locations can be easily obtained using the intrinsic parameters of the camera. **EgoDexter**[136] includes RGB-D images recorded with a body-mounted camera from egocentric viewpoints of hand interactions with objects in real cluttered scenes, complex hand-object interactions, and natural lighting. It is composed of 4 test sequences, with 3190 frames at \(640\times 480\) resolution, performed by 4 subjects. EgoDexter is employed for evaluation purposes, as no training set is provided. **Dexter+Object**[141] provides six test sequences performed by two subjects, one male and one female. Similarly to EgoDexter, it is only utilized for evaluation purposes. The recordings were conducted by a static camera, containing interactions of a single hand with a cuboid object. This dataset is composed of about 3K samples, with a resolution of \(640\times 320\) providing RGB images, depth maps, and annotations for fingertips and cuboid corners. **Dexter1**[79] contains 7 sequences of a single subject's right hand performing slow and fast hand motions. Roughly the first 250 frames in each sequence correspond to slow motions, while the rest of them are fast motions. It includes about 2 K frames with a resolution of \(320\times 240\).