Omnidirectional and blindspot-free pre-touch sensing systems
Creators: name: Rupavatharam, Siddharth
Publisher: name: No Publisher Supplied
Year: 2023

description: Interactions with cyber-physical systems are dynamic three-dimensional actions that are a function of context and physical proximity to the system. These interactions occur just above the surface of the system extending up to 10cm away in the pre-touch range. Existing sensing modalities find it challenging to perform pre-touch sensing due to fixed fields of views and bounded upper and lower sensing distances, limiting sensing areas and interactions. This motivates the need for omnidirectional pre-touch sensing systems to unlock novel immersive interactions. This thesis investigates leveraging capacitive and acoustic sensing modalities to enable omnidirectional and blindspot-free pre-touch sensing. This is accomplished by developing novel sensing systems for distinct interactions. Chapter 2 investigates pre-touch sensing for human-computer interactions. State-of-the-art hand gesture recognition techniques, used in conjunction with augmented and virtual reality headsets, require hands to be present in the field of view of externally mounted cameras. This requirement hinders the useability of systems as interactions are constrained to fixed areas. To overcome this, a hand gesture recognition glove is designed using electrodes placed on fingertips to sense and recognize dynamic, micro finger gestures. The glove detects hand gestures by measuring pair-wise distance between fingertips as a function of the capacitance between them. This chapter presents, the capacitance sensing circuit, a synchronous undersampling signal demodulation scheme, and neural network based gesture recognition algorithms designed towards realizing the system. The novel capacitance-based pre-touch sensing system recognizes 14 unique hand gestures with an accuracy of 97%. Chapter 3 investigates pre-touch sensing for safer human-robot interactions. Existing proximity detection and collision avoidance techniques create a distributed sensor by deploying numerous sensors with small individual sensing ranges over the entire surface of a robot. Sensing modalities used by these sensors find it challenging to detect objects near the surface, while large scale sensor deployments require robot surface modifications. To address both these issues, a novel acoustic field based pre-touch sensing system that performs close-proximity detection and bearing estimation is proposed. A single piezoelectric transducer is used to vibrate an entire robot surface to transmit inaudible ultrasonic waves and create an omnidirectional, blindspot-free acoustic field engulfing the robot. Objects and humans that enter this field reflect acoustic signals back allowing them to be detected. This chapter presents, transducer deployment, acoustic signal generation and reception toolchain, and spectral power based thresholding algorithms towards realizing a real-world deployable system. The novel acoustic-field based pre-touch sensing system detects proximity with 93.8% sensitivity and 96.6% specificity for a robot in motion. It also estimates object bearing and maps it to three zones on the robot with 100% sensitivity and specificity. Chapter 4 investigates pre-touch sensing for robot-environment interactions. Robot arm manipulators use end-effectors or grippers to interact with objects in their vicinity using pre-touch sensing modalities to inform gripper position to ensure successful grasps. However, self-occlusions make it challenging to re-shape grasps when the gripper is near an object. To combat this, a novel acoustic field based pre-touch and contact detection tactile sensor with reactive pregrasping abilities is proposed. A piezoelectric transducer is embedded in each finger of a parallel jaw gripper and is excited to create an acoustic field engulfing the entire gripper. Objects entering an open gripper reflect signals back to both fingers providing information to adjust gripper pose pre-grasp. While acoustic field reflections inform pre-grasp object detection and gripper alignment, post-grasp contact is monitored based on changes in finger-transducer acoustic coupling. This chapter presents, a novel 3D printed finger for embedding a piezoelectric transducer, a narrow band coherent detection-based signal processing pipeline to extract weak signals, and algorithms to detect proximity and contact. The sensing system aligns objects at the center of a parallel jaw gripper with a mean error of $leq$ 5mm. It detects objects in close-proximity with a sensitivity and specificity of 92.4% and 95.3% respectively and detects contact with a sensitivity and specificity of 93.7% and 98.7% respectively. In conclusion, this dissertation leverages acoustic field and capacitance based sensing modalities to perform omnidirectional and blindspot-free pre-touch sensing. It demonstrates that the proposed sensing methods extend interaction areas outwards into the immediate vicinity of devices, making interactions occur in three dimensions while being direct, dynamic, immersive, and intuitive. Effectively unlocking another dimension of interaction between man and machine.