Title: DepthSense+DP: Adaptive Learning for Robust and Differential Private Silent Speech Recognition
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Adaptive Learning, Silent Speech Recognition, Privacy Aware, Depth Sensing, Biometric securiny, Zero Shot Generalization, Multimodal Fusion

Abstract: Silent speech recognition (SSR) enables privacy preserving human computer interaction but existing methods remain fragile across devices, users, and environments, while incurring high computational costs. We introduce DepthSense+DP, the first systematic solution for privacy aware cross device SSR that jointly achieves real time performance, robustness, and privacy. Our framework tackles the unique challenge of applying Differential Privacy (DP) to 3D depth point clouds, where biometric identifiers must be perturbed without degrading articulatory features. By combining adaptive neural modules with multimodal fusion, the system learns user specific lip and tongue dynamics while maintaining generalization. Input is transformed into anonymized depth point clouds, enabling zero shot transfer across unseen users and devices. The training strategy supports edge deployment under noisy and varied orientations, while privacy preserving data augmentation ensures biometric protection. Extensive evaluation demonstrates reduced error rates and improved robustness, establishing DepthSense+DP as an efficient and secure foundation for next generation SSR.

TL;DR: DepthSense+DP bridges depth-sensed lip movements and ultrasound tongue dynamics through adaptive neural learning, enabling private, robust silent speech recognition across wearables and ambient devices.