## 3 Proposed Method  
In this section, we describe a method for reducing interaction latency when performing human-VO interactions. In general, in remote MR collaborations, including human-virtual object interactions, remote users cannot know the other user's intentions until an interaction is performed. Since users' intentions include interaction targets, interaction types, etc., if users' intentions are known previously, it is possible to predict changes in the virtual object and environment (e.g., the object's color, sound effect, etc.). If the changes of the virtual object or environment are predictable, the'saved time', highlighted in blue in Figure 1, can be shortened until the change caused by the interaction is revealed to a user. To reduce interaction latency, this study proposes a method to find the interaction information in advance in remote MR collaborations. The proposed method is a method to reduce interaction latency by predicting the occurrence of the interaction before the human-VO interaction is performed. There are many types of human-VO interactions used in the MR environment [24], and one of those types, a representative type that does not require tools and devices other than  
Figure 1: Example of saving time based on predictions. a see-through type displays or HMDs, is the gesture-based interaction. This study mainly targets gesture-based interactions that are not biased toward a specific device and can be widely applied to remote MR collaborations. Meanwhile, the types of gestures that can be used for interaction, such as manipulating virtual objects in a remote MR collaboration, are various, and in many cases, those gestures even differ for each application. This study focused on gestures that were mainly used to transform a target object in a human-VO interaction, such as grab and pinch [21]. For the interaction prediction, it is required to classify the studied target gestures correctly. In the proposed method, k-nearest neighbor (k-NN) [25], which executes quickly on a relatively small dataset without significantly compromising accuracy, was used as an algorithm for gesture classification. The purpose of this study was to investigate the feasibility of the proposed method, so a relatively small and simple k-NN algorithm was adopted. A k-NN algorithm can cause problems of classifying an undefined hand gesture (hereafter none) into a specifically defined hand gesture (grab, pinch). Therefore, the existing k-NN was partially adjusted to further classify undefined hand gestures. Three numbers (3, 5, and 7) were also selected as the value \(k\), since the performance of k-NN may vary depending on the value \(k\). A dataset is essential for classifying gestures with k-NN. In MR, the hand is usually expressed as a 20-keypoint model representing a joint. If all of this information is used, the size of the dataset becomes very large as the number of gesture samples in the dataset increases. Therefore, in this study, we tried to derive a representative value that can represent the hand using joint information before creating a dataset for gesture classification. Since the representative value is used to classify the gesture, it should express whether the gesture is performed or not. In other words, the value that changed the most according to the gesture that was performed should be selected as the representative value. To find this, grab and pinch gestures, the targets of this study, were performed; recorded joint trajectories data are shown in Figure 2.  
Figure 2 shows the joint trajectories with respect to grab and pinch gestures. Each joint's information was recorded by dividing it to confirm the degree of change as follows: gesture at starting moment (Figure 2a,d), gesture at ending moment (Figure 2b,e).