# Enhanced Pet Behavior Prediction via S2 GAN-Based Heterogeneous Data Synthesis
## 3 Pet Behavior Prediction via S2 GAN-based Data Synthesis
### Data Preprocessing  
#### 3.2.1 Video Preprocessing  
The collected videos underwent preprocessing for data synthesis and behavior prediction. The video data were preprocessed in the following order: data augmentation, skeleton extraction, and sequence generation.  
First, data augmentation was performed to enable the model to learn various behavior patterns. Owing to the varied installation locations of CCTVs by different users, augmentation processes that consider these differences and deal with motion blur from fast-moving pets were required. Therefore, augmentations through horizontal flipping, rotation, and blurring were performed to ensure consistent performance across various situations.  
Subsequently, pose estimation was performed using a you only look once version 8 (YOLOv8)-based pose estimator. YOLOv8 is an AI framework that supports various computer vision tasks, such as detection, segmentation, and classification [30]. While there are options such as DeepLabCut [21] and DeepPoseKit [22] for pet-pose estimation, they generally exhibit slower training speeds and are slightly less robust to variations in different environments. YOLOv8, with improved accuracy and speed compared with previous versions of the YOLO series, has gained prominence in recent studies [31]. Therefore, we performed pose estimation using the YOLOv8 model. The extracted joint coordinates were categorized into major sections--head, torso, front legs, and hind legs--constituting a total of 24 points (Figure 3).  
\begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt}} \hline \hline
**Behavior** & **Description** \\ \hline Standing on Two Legs & Standing on hind legs with the body upright, whereas front legs are not touching the ground. \\ Standing on All Fours & All four legs touching the ground, joints fully extended in a stationary position. \\ Sitting on Two Legs & Sitting on hind legs with the buttocks and hind legs touching the ground, whereas the front legs are not touching the ground. \\ Sitting on All Fours & Sitting with the buttocks, hind legs, and belly all touching the ground, whereas the head is not touching the ground. \\ Lying Down & Entire body, including buttocks, legs, belly, and head, touching the ground, with the head positioned between the front legs. \\ Lying & Back touching the ground, covering both the full back-on-ground pose and partially lying on one side with the belly visible. \\ Walking & All four legs not simultaneously touching the ground, alternating steps while moving forward. Encompass all walking actions. \\ Sniffing & Light movement of the head with the nose close to the ground. \\  & Encompasses slow walking while sniffing. \\ Eating & Having something in the mouth, including actions such as chewing and drinking. \\ \hline \hline \end{tabular}
\end{table}
Table 2: Predicted pet behaviors and descriptions.  
Finally, data sequences were generated using time windows set to 3 s intervals. Each time window was designed to overlap 50% of the next, ensuring continuity in the time-series data and minimizing information loss. Each sequence includes 20 frames, each containing the x and y coordinates of the joints. Consequently, the dimensions of the data sequence are structured as (sequence length, number of frames, number of joints, 2), where 2 denotes the x and y coordinates for each joint; this structure was specifically designed to effectively capture the visual characteristics of behavioral patterns.