# Enhancing Recognition of Human-Object Interaction from Visual Data Using Egocentric Wearable Camera
## 1 Introduction  
Computer vision is an imperative field of research with diverse applications that can enable communications through wireless networks through pervasive cameras for improving human-computer interaction. One of the most crucial areas in this regard is the detection/recognition of human-object interactions from video cameras (particularly wearable cameras). Understanding human-object interaction (HOI) aims to interpret how human beings interact with objects around them by processing images, videos, or any other data [1]. In this regard, previous studies identified humans and objects within a scene using object detection algorithms. Researchers also analyze the spatial relationships and bodyposes to interpret interactions, like holding or using an object. For videos, they examine how these interactions change over time to capture the full dynamics of human actions. HOI recognition (HOIR) is marked by its significant applications in robotics and assistive technologies. However, it faces numerous challenges in its implementation. Most of the previous studies proposed HOI methods that rely on fixed/static cameras. These studies focused on extracting the texture information, color, and shape of the objects [1; 2; 3] for HOIR. Such methods may fail to perform efficiently in scenarios where objects can exhibit variability in texture, color, or shape due to lighting changes, occlusions, and variations in object appearance. Thus, this variability can lead to ambiguity in recognition results and decrease the robustness of the model. Furthermore, deep learning-based HOIR approaches rely heavily on extensive datasets to model the complex variety of interactions, which is a task that is both cost intensive and laborious [4]. The ambiguity in interpreting interaction, such as distinguishing whether a person is simply holding or using an object, adds a layer of complexity that requires nuanced understanding [5]. The models often struggle to generalize across different contexts, which decreases their effectiveness in unfamiliar scenarios. Also, the use of deep learning for object detection and action recognition has a high computational cost [6; 7]. These challenges emphasize the need to develop alternative approaches for the HOIR system, keeping in mind the computational cost and the efficacy of the output results. A few researchers utilized the egocentric camera view for HOIR as well [8]. The egocentric view offers unique advantages over traditional exocentric (i.e., third-person) camera views by capturing interactions from the user's perspective, which is essential for understanding natural interactions. This perspective emphasizes only the relevant objects and areas, offering clear visibility and direct insight into user attention and intent. Keeping in view the above discussion, hand pose information can be extracted from a wearable egocentric camera to implement HOIR. By utilizing detailed information from the hand pose, we can interpret interactions between humans and objects. Advances in computer vision and machine learning have enabled the precise tracking and analysis of hand joints [9], allowing for the identification of objects based on hand movements. Incorporating hand pose information into HOIR can enhance the detection of subtle differences in how objects are used, making it more robust against common real-world issues, like obscured object views, occlusion, and background clutter. The existing studies show promise in estimating hand pose data from images and videos, which can be crucial for HOIR. In a previous study, the authors proposed a method for estimating hand pose information from images/videos and they highlighted the significant improvements in the results [10]. Furthermore, in another study, the authors demonstrated how deep learning techniques are applied to extract hand pose information from images/videos [11]. Following this, we intended to use the hand pose information to make an HOIR model that emphasized the importance of hand movements for this task.