## 4 Structure of the ViT and CNN Models  
The CNN used in this study retains and learns location information. The ViT learns by adding location information. Based on this, we thought it would be possible to perform \begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **Used Angular Feature** & **Dimension** \\ \hline Dataset 1 & The angles of each finger joint (Feature 1) & 20 \\ Dataset 2 & The overall tilts of the hand (Feature 2) & 20 \\ Dataset 3 & Feature 1 and Feature 2 combined & 40 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Three datasets by angular feature combination. Figure 3: Relationship between each coordinate location and angle location for angle calculation performed for feature extraction. 3D recognition using a 2D recognition model by converting each frame of the original 3D data into a 1D feature vector and then converting this into 2D data on the time axis. The ViT model was investigated by changing the number of Encoder layers, and data with various dimensions were input to the model with the highest accuracy. In contrast, the CNN model's internal structure changes by changing the size of the input data, so ablation studies were conducted with three types of data input. The ViT model, based on the transformer's core architecture [5], uses an encoder featuring a multihead self-attention mechanism and fully connected neural networks. The addition of more encoder layers in the ViT amplifies the model's depth, enabling it to perform more intricate feature extraction and facilitating the learning of hierarchical and abstract representations, although this may lead to a higher chance of overfitting. In this study, we investigated the recognition performance for 46 classes of Japanese finger spelling by varying the number of encoder layers in the ViT model. The ViT model's input comprises the angle features extracted from each frame, fixed in number, and transformed into two-dimensional matrix vectors. Unlike conventional ViT models that involved dividing an image into patches, in our study, as depicted in Figure 4a, we treat each frame as a single vector. Then, a linear projection of the angular features is performed upon data input, and position information is assigned to each vector using embedding by [PERSON]. Once the data are input, they are processed with a transformer encoder for training. The transformer encoder comprises four layers: a multihead attention layer, an MLP layer, and a layer normalization layer in front of each layer. The encoder iterates the process 4 to 9 times, and finally, its output is passed to the MLP head for classification into 46 classes. This approach facilitates the input of sequential frames into the transformer's encoder with added positional encoding. Our method employs a multilayer perceptron (MLP) with two fully connected layers, two dropout layers, and the Gaussian error linear unit (GELU) function, combined with residual connections. The encoder structure, comprising a normalization layer, a multihead attention layer, and an MLP layer, is repeated for the specified number of encoders, ending with an additional fully connected layer for classification into 46 classes. The learning rate was initially set at 0.001 and was halved every five epochs. For this implementation of the ViT, we adapted the model to our unique dataset, which comprises either 20 or 40 feature data \(\times\) 20 frames, by omitting the original ViT's patch-splitting component. This modification ensured the effective use of the dataset's unique character  
Figure 4: The structure of the ViT model and CNN model: (**a**) example of our ViT model with 3 encoders; (**b**) example of our CNN model with 8 layers. istics in our experiment. We employed the ViT model to assess the impact of varying the number of encoders on accuracy. The performance of Japanese sign language recognition was verified by varying the encoder layer by one layer, from 4 to 9 layers. Additionally, four multi-head self-attention mechanisms were included in one encoder.