Sensor Enhanced Communication and Computer Interaction
Creators: name: University Of California Office Of The President
Publisher: name: University of California Office of the President
publisherIdentifier: https://ror.org/00dmfq477
publisherIdentifierScheme: ROR
Year: 2024

description: Computation is undergoing a transformation away from “desktop computing” for the global business professional and towards “computational devices” for everyone. These new markets will be driven both by new uses such as “computational video communication” and “media stations,” and by new users – the next one billion customers – who have not been reached by today’s computers. It is unlikely that keyboards and mice will be the dominant interaction tool in these domains. Computers will need to get smarter about sensing their current context, as well as smarter about sensing the needs of their users. Is there a person standing anywhere nearby? Are the lights on in the room? Did someone point at the screen? Does the user standing here actually look like one of the authorized users of this device? Today’s computers have no way to answer these questions. Tomorrow’s computers will likely have new sensors, cameras, lights and widgets to allow these new forms of interaction. For example, HP’s touch screen computers can sense finger position. The proposed project investigates three domains in which the introduction of new sensors based on lights and cameras could have a large impact on the way in which computer interaction occurs: 1) Face relighting and matting for video communication – Video communication in the home and hotel room is plagued by difficulties with poor lighting, and the need to remove background scene clutter. Neither of these problems can be robustly resolved using only video imagery. We propose active sensing and illumination to overcome these difficulties. 2) 3D structured light sensing – 3D sensing is a critical enabler to a wide variety of new interactions including: simple presence awareness, gesture interfaces, and reliable face recognition for security. We propose combining elements of time-of-flight and triangulation structured light sensing as a path towards better sensors. 3) Tracking people for interaction and gesture recognition – As computers move into living rooms and 3rd world communities, the need to accurately detect and track people’s gestures will increase. We propose using a high quality human shape prior, together with 3D sensors as a way to improve the quality of human tracking. The world’s workforce and markets are now global. California and Silicon Valley have maintained their position of leadership by attracting talent to live and work here, and by innovating here. This project will fund training for two Ph.D. students towards degrees finished in California. Recent Ph.D. students are among those most likely to start new companies, thus creating jobs and economic development in the state. The project focuses on basic and proof of concept research in video-conferencing, 3D imaging and computer interaction. If successful, this research will lead to improved video conferencing products, new products for 3D imaging, and computer systems that sense hand gestures in addition to mouse and keyboard input.