# Enhancing Recognition of Human-Object Interaction from Visual Data Using Egocentric Wearable Camera
## 3 Proposed System
### Data Acquisition  
The proposed method was developed to work on an egocentric dataset, i.e., first-person view. Although with conventional cameras, i.e., third-person views, there are many datasets available for the same kind of tasks, there is lack of comprehensive egocentric datasets that may be relevant to this study. Most of the relevant datasets (one significant example is [33]) are based on synthetic data, where 2D images are used to generate 3D synthetic images of the hand, and then further processing is performed on synthetic hands. The HOI4D dataset, which was developed by [PERSON] and their team [34], is a good choice in this regard, as it processes the hand annotations against the real images. Furthermore, the HOI4D contains a lot of variations, as it consists of videos from four different cameras that are independent of each other, with different indoor conditions. The dataset includes 2.4 million RGB egocentric video frames from 4000 video sequences. Nine individuals performed interactions with 800 object instances in the dataset across 16 categories in different indoor environments. Each single HOI was captured in a range of settings (differences in rooms,  
Figure 1: Object-dependent interaction recognition (ODIR).  
Figure 2: Object-independent interaction recognition (OIIR).  
room layouts, users, cameras, backgrounds, simplicity and complexity of interactions, etc.). The dataset divides interactions into simple and complex scenarios, with the former offering clear, consistent views and the latter introducing more variability. For more specifics, Table 1 details the object categories and their interactions. To ensure the robustness of our proposed algorithm, with respect to in-the-wild conditions, we performed extensive testing on the HOI4D dataset using class-wise cross-validation. This ensured that the output results were free of any biases, as each data sample (whether simple or complex, from different persons, in varying room settings, and using different cameras) was used for training and testing, but in different iterations.  
\begin{table}
\begin{tabular}{l l} \hline
**Objects** & **Interactions** \\ \hline Bottle & Pick and place \\  & Pick and place (with water) \\  & Pour all the water into a mug \\  & Put it in the drawer \\  & Reposition the bottle \\  & Take it out of the drawer \\ \hline Bowl & Pick and place \\  & Pick and place (with ball) \\  & Put it in the drawer \\  & Put the ball in the bowl \\  & Take it out of the drawer \\  & Take the ball out of the bowl \\ \hline Bucket & Pick and place \\  & Pour water into another bucket \\ \hline Chair & Pick and place to a new position \\  & Pick and place to the original position \\ \hline Kettle & Pick and place \\  & Pour water into a mug \\ \hline Knife & Cut apple \\  & Pick and place \\  & Put it in the drawer \\  & Take it out of the drawer \\ \hline Lamp & Pick and place \\  & Turn and fold \\  & Turn on and turn off \\ \hline Laptop & Open and close the display \\  & Pick and place \\ \hline Mug & Fill with water by a kettle \\  & Pick and place (with water) \\  & Pour water into another mug \\  & Put it in the drawer \\  & Take it out of the drawer \\ \hline Pliers & Clamp something \\  & Pick and place \\  & Put it in the drawer \\  & Take it out of the drawer \\ \hline Safe & Open and close the door \\  & Put something in it \\  & Take something out of it \\ \hline Scissors & Cut something \\  & Pick and place \\ \hline \end{tabular}
\end{table}
Table 1: List of objects and associated interactions in HOI4D dataset.