# Enhanced Pet Behavior Prediction via S2 GAN-Based Heterogeneous Data Synthesis
## 2 Related Works
### Behavior Prediction  
In behavior prediction research, studies have typically been conducted with humans as the primary subjects. Human behavior is predicted using homogeneous data, where similar types of data are combined, and heterogeneous data, where different types of data are combined [17]. Video and sensor data are integrated and used as heterogeneous data. For example, [PERSON] et al. used heterogeneous data from in-car cameras and driver's wearable devices and used machine learning algorithms such as random forest to recognize driver activities such as using the phone, eating, crossing intersections, and highway driving [18]. [PERSON] et al. proposed an indoor action recognition method for the elderly using the RGB-D camera images captured by a humanoid robot, wearable inertial sensors, and surrounding sensors attached to items inside a home [19]. [PERSON] et al. [20] proposed a system that effectively detects human falls and daily activities using wearable accelerometers and video data collected by cameras directly worn on the body. These studies based on heterogeneous data addressed quality issues inherent in homogeneous data and demonstrated excellent prediction performance. Research on behavior prediction of pets predominantly utilizes homogeneous data. Video data used for predicting the behavior of pets are obtained from cameras attached to specific locations. [PERSON] et al. mounted cameras on the ceiling of their experimental space to automatically identify dogs exhibiting attention-deficit hyperactivity disorder behavior [6]. [PERSON] et al. predicted six behavior patterns, including walking, sitting, sleeping, and eating, using Yolov3-based extraction [5]. Moreover, research on predicting behavior through pose estimation is being conducted. Methods have been proposed that use depth cameras or deep-learning techniques to estimate poses from regular RGB images [21; 22]. Sensor-based research typically involves collecting behavioral data using wearable devices. Accelerometers and gyroscope sensors are commonly used for activity measurement. Multisensor data, collected through wearable devices or sensors installed in the environment, offer the advantage of being able to monitor pets' physical activities, location, and even physiological states in real time. [PERSON] et al. proposed a method for recognizing the behavior and emotions of dogs by attaching sensors to their neck and tail to collect data [23]. [PERSON] et al. attached sensors to different parts--one on the neck and the other on the harness--utilizing accelerometers and gyroscope sensors to predict behavior [24]. Research on pet behavior prediction has mostly focused on multisensor data rather than video data [4]. This approach focuses on analyzing and predicting pets' movements or activity patterns based on data collected from sensors. However, relying solely on a multisensor-based approach may face certain limitations in comprehensively understanding and predicting pets' behavior and emotional states. Although sensor data can provide information about pets' physical activities, they may lack the ability to capture the visual aspects of behavior or situational context. For example, the emotional states of pets, such as stress or joy, can be more accurately analyzed through video data. Therefore, for effective pet behavior prediction, it is necessary to utilize a comprehensive approach that integrates various types of data. Unlike human behavior prediction, the behavior patterns of pets are irregular and anomalous. Particularly, the poses of small animals involve short distances between joints, and sensor values and joint coordinates change rapidly over brief periods. These characteristics imply that the same behavior can appear differently in humans. Although research on behavior recognition using heterogeneous data is actively being conducted with a focus on humans, applying methodologies developed for humans to pets may be inappropriate.