## 3 Action Classification Definition and the NPU RGB+D Dataset for Basketball
### NPU RGB+D Dataset for Basketball Players
#### 3.2.3 Data Modalities  
We obtained RGB+D videos in svo file format from a ZED camera, and then we extracted the other three data modalities: the L&R views frame sequence, depth maps, and 3D skeleton joint point coordinates as shown in Figure 1. The data acquisition process includes the acquisition process of L&R view frame sequences, depth maps, and skeleton joint point coordinates. A L&R view frame sequence is an RGB image sequence extracted from RGB+D videos by using the export function of ZED SDK, which represents the frames taken by ZED camera from left and right views. The acquisition process of the L&R view frame sequence is as follows. Firstly, we visited [[https://download.stereolabs.com/zedsk](https://download.stereolabs.com/zedsk)]([https://download.stereolabs.com/zedsk](https://download.stereolabs.com/zedsk)) (Accessed on 26 th April 2021) to get the ZED SDK. This paper used ZED SDK 2.8 of CUDA10.0 in Ubuntu16. Secondly, we entered the export folder in the svo recording directory and executed ZED_SVO_Export \"path/to/file_svo\" \"path/to/output/folder/\" 2, where \"path/to/file_svo\" and \"path/to/output/folder/\" were replaced with the actual location of the svo file and the location of the target folder. Finally, the left and right frame sequences of svo video were automatically generated for the specified directories. Figure 3: The positions of cameras on the basketball court. A depth map is a representation of depth information extracted from a 3D video. The ZED camera captures 3D video with high resolution and frame rate and estimates the pixel depth by comparing the pixel displacement between left and right images to obtain the depth map. The depth map stores the distance value (z) of each pixel (x, y) in the image, representing the distance from the back of the camera's left eye to the object. The acquisition process of the depth map was as follows. Firstly, just like getting the L&R view frame sequence, we went to the export directory of ZED SDK. Secondly, we executed ZED_SVO_ Export \"path/to/file.svo\" \"path/to/output/folder/\" 3 command, where \"path/to/file.svo\" and \"path/to/output/folder/\" were replaced with the actual location of the svo file and the location of the target folder. Finally, we got the depth maps in the specified directory. 3D skeleton joint information consists of the 3D locations of 25 major body joints for detected and tracked human bodies in a scene. We extracted the 3D skeleton coordinates based on OpenPose and the ZED camera. OpenPose [36] is an open-source project enabling the real-time tracking of human 2D key points. ZED camera can realize high-precision position tracking and depth perception and provide 3D point cloud information corresponding to 2D points in the picture frame by frame. The acquisition process of extracting the 3D data using OpenPose from a ZED camera was as follows. Firstly, we visited [[https://github.com/stereolabs/zed-openpose](https://github.com/stereolabs/zed-openpose)]([https://github.com/stereolabs/zed-openpose](https://github.com/stereolabs/zed-openpose)) (Accessed on 26 th April 2021) to obtain the zed-OpenPose provided by stereolabs, the technology development laboratory of ZED camera. Secondly, ran main.cpp in the src directory to start the ZED camera and capture video frame by frame. Thirdly, after receiving the RGB frame data, the program automatically woke up the OpenPose thread and obtained the 2D position of 25 human joint points in each frame recognized by OpenPose. Fourthly, the depth information and confidence of the 2D coordinates of joint points were extracted one by one from the point cloud data automatically generated by ZED camera. If the extraction failed, the depth information of the nearest point within the range of 10 pixels was found as a substitute. Fifthly, the 2D coordinates and depth information of each joint point were integrated to get 3D coordinates, which were added to the skeleton point coordinate array.