# Enhanced Pet Behavior Prediction via S2 GAN-Based Heterogeneous Data Synthesis
## 2 Related Works  
### Data Synthesis for Behavior Prediction  
Data synthesis is the creation of artificial data with statistical features similar to the original data. It is a valuable method for improving the performance of deep-learning-based models by addressing the limitations and biases in the original data [10].  
The role of data synthesis in the field of behavior prediction is very important. By artificially generating data that is difficult to obtain in real situations, data synthesis provides various scenarios necessary for model training. Especially, when collecting behavior data is difficult due to ethical or practical constraints, data synthesis helps overcome these limitations.  
Therefore, research on data synthesis is currently being conducted for various targets of behavior monitoring and under different environments. [PERSON] et al. introduced a SensoryGAN to generate training data and address the challenge of collecting real data for human behavior monitoring based on wearable sensors [11]. They explored a neural network structure optimized for each behavior for the generator and discriminator using a 1D-CNN (Convolution Neural Network) layer and bidirectional LSTM (Long Short TermMemory) layer for actions. [PERSON] et al. proposed a method for generating sequences of joint movements from a single static image [12].  
Research on the synthesis of data for pets is significantly less prevalent than that for humans. Several existing studies have aimed to overcome the issue of insufficient data, particularly for pose estimation. [PERSON] et al. proposed a novel consistency-constrained semi-supervised learning method that uses both spatial and temporal consistencies, enabling accurate keypoint predictions for real images without image labeling [13]. [PERSON] et al. also addressed the challenge of missing data by proposing a parametric canine model, DynaDog+T, for binary segmentation, which allows the creation of synthetic images for 2D/3D joint key points, silhouette maps, and part segmentation [14]. Moreover, there are studies on synthesizing data based on sensor data. [PERSON] et al. proposed a TN (Text to Numerical)-GAN-based multidimensional time-series augmentation method to minimize data bias [15].  
Research in behavior prediction primarily uses GANs or diffusion models to synthesize video-based data, enhancing model accuracy and generalizability by diversifying the training datasets, thereby yielding reliable outcomes. However, most existing studies have focused on homogeneous data synthesis, such as modifying the appearance of images or generating data for specific actions using sensors [16]. While useful for data augmentation, these approaches face limitations in multimodal behavior prediction, which requires the integration of various data sources.  
The synthesis of heterogeneous data offers the potential to maximize the advantages of multimodal learning and use complementary information provided by different data types. However, applications in the behavior prediction or recognition field are still rare, particularly studies targeting pets. Research in this area is still in its early stages, and further studies are warranted to develop specific methodologies and models that can effectively combine heterogeneous data to improve prediction accuracy.