## 3 Materials and Methods
### Database Acquisition  
Since the aim was to compare different data acquisition systems (RGB and RGB-D), the first step of the work involved the creation of two datasets tailored for the Italian Sign Language (LIS) recognition. The premise is that the signs for letters \(g\), \(j\), \(s\), and \(z\) were not considered, since they use a dynamic gesture that cannot be captured and identified in a single frame, as required by our real-time application. Regarding the RGB database, a freely accessible online database was used [33]. This dataset is composed of around 250 pictures from 11 different people for each of the 22 letters considered. The images are in JPG format with dimensions 622 \(\times\) 415 pixels, and granted a sufficient diversity in terms of hand shapes and possible orientations, as they were taken from 3 different angles (Figure 1). The dataset for RGB-D images was created from scratch. The existing RGB-D datasets concerning LIS, such as A3 LIS-147 [34] and Montalbano [35], contain the subject's hands and body, offering a comprehensive picture of the motions necessary to represent LIS signs. Both include hand movement, body posture, and gestures. In contrast, the current application concentrates on hand movement to make learning as simple as possible for children. As a result, the information offered by these databases has insufficient resolution because the hand is not completely visible in the foreground. To solve this issue, a tailored database acquired using a state-of-the-art consumer-grade camera, the Kinect Azure, was created. The choice of a depth camera with time-of-flight (ToF) technology for depth acquisition has been performed to deal with the current depth camera market evolution. In fact, the most recent smartphones are equipped with ToF sensors because of their remarkable performance and small size. Moreover, the 4K camera allows for excellent RGB images to be associated with the corresponding depth map (Figure 2). Azure Kinect is a highly efficient device also used in other studies to obtain excellent quality 3D video tracking of different anatomical segments [36]. A 10 s video was recorded by seven people for each sign. The recorded videos from the Azure Kinect are in.mkv format, so a script to extract from the different channels the single frames and depth maps to create the dataset was developed. Since the videos were recorded at 30 fps, a total of 1500 images in PNG format for each hand sign were obtained. During the recording slight rotation and flexion of the wrist were performed to make the classifier more adaptable to different camera orientations and guarantee data generalization. Specifically, to train the algorithm for gesture recognition with the cleanest possible data, especially from the z-coordinate point of view, the videos were acquired by placing a green canvas behind each subject, so the hand had a uniform background. Nonetheless, as will be shown in Section 4, since the classifier is based solely on the landmarks coordinates, the algorithm performance is not negatively affected by a change in the background nor different values of distances between the subject and the camera (in accordance with the Kinect operating range). The acquisition device uses two different sensors to capture the RGB image and the Depth image, as shown in Figure 2a,b. Therefore, the dimension of each RGB image in this dataset is 1080 \(\times\) 1920, while the dimension of the depth maps is 576 \(\times\) 640. Consequently, depth map processing was needed to superimpose the two images and properly associate each x and y coordinate with the RGB color and depth value. To better visualize the 3D nature of the depth map, Figure 2c illustrates a cropped version that considers only the volume of interest, namely  
Figure 1: Three examples of images taken from the LIS-dataset for the letter A in three different orientations. the hand. This cropped version was obtained by thresholding the image on the z-axis and excluding everything behind the threshold.