## 3 Methodology and Algorithms
### Proposed Augmentation Framework via Kinematic Gesture Simulation
#### 3.2.3 Sign Language Diversity Enhancement Strategy Driven by Multimodal Perturbation  
Following the accurate reconstruction of hand gestures through inverse kinematics (IK), we introduce a multimodal perturbation strategy to systematically enhance the diversity and robustness of synthesized sign language sequences within a virtual environment. As illustrated in Figure 2, this stage applies structure-preserving perturbations across spatial, kinematic, and visual dimensions to emulate the natural variability and observational noise encountered in real-world gesture execution.  
The process begins with IK-based joint angle estimation from video-derived keypoints, which are mapped onto a 3D digital hand model to reconstruct precise finger trajectories. To further enhance fidelity beyond local articulation, a body animation module generates synchronized upper-body motion, including arms and torso, ensuring that the resulting digital gestures reflect both fine-grained control and holistic motion patterns consistent with natural sign communication.  
To introduce controllable variation into the synthesized gestures, we design three complementary perturbation mechanisms:  
* Depth Offset: Simulates variations in camera-to-subject distance by perturbing the depth plane of the virtual viewpoint.
* 2D Spatial Perturbation: Introduces localized shifts in the image plane (X-Y), mimicking tracking noise, occlusions, or signer-dependent expressive differences.
* IK Sensitivity Control: Adjusts solver hyperparameters such as target weight and damping, enabling variability in articulation flexibility and gesture expressiveness.  
All perturbations are governed by a sensitivity parameter \(\varepsilon\), which ensures semantic consistency while introducing naturalistic motion deviation. From a single original gesture instance \(D_{1}\), this strategy generates a diverse sequence set \(D_{1}\to D_{n}\), collectively forming the Augmented Gesture Sequences shown in Figure 3.  
Figure 3: Augmented gesture sequence generated via virtual perturbation pipeline. Structure-preserving variations are applied in depth, spatial position, and inverse kinematic sensitivity, resulting in diverse but semantically consistent gesture variants governed by a sensitivity parameter \(\varepsilon\). This enhances generalization and simulates real-world gesture variability.  
Figure 2: Real video sequence extracted from raw gesture recording. The frames illustrate the original signer’s motion trajectory captured under constrained conditions. These unaugmented sequences serve as the source input for inverse kinematics-based hand pose reconstruction and virtual simulation.  
To further illustrate the effect of the augmentation pipeline, a supplementary video is provided showing the real gesture sequence and its IK-AUG-generated variants in motion. The video demonstrates how the perturbation components preserve semantic integrity while introducing realistic variability. It is available at: [[https://youtu.be/hawctx3](https://youtu.be/hawctx3)]([https://youtu.be/hawctx3](https://youtu.be/hawctx3)) g2 FE (accessed on 8 July 2025).