[General Topics] JeffKang2:
Depth sensors

Kinect, Touch+, Leap Motion, Nimble Sense, Project Tango, SoftKinetic, Intel RealSense, etc. are all various combinations of hardware and software to detect depth and gestures.

There was also a MIT and Microsoft article about the ability to use infrared LED lights and machine learning techniques to turn a simple single-lens camera into a 3-D one.

technologyreview/com/news/529986/turning-a-regular-smartphone-camera-into-a-3-d-one/

However, at this point, there seems to be a shift to depth cameras, such as the Intel RealSense that is already integrated into laptops.

Time-of-flight (TOF) cameras for point cloud data have more potential,.

It would need to be built into your phone to be cost-effective, and even then, look how pricey Project Tango is right now.

Intel RealSense is in the HP Sprout (has a projector), but it can’t do surface touch yet.

3divi has a "turn a surface into touch surface" prototype Youtube video (youtube/com/watch?v=upGTLrSUa5c ) that uses Kinect, and a Pico projector.

Touch+ was the biggest help last year, as it was only $75 for surface touch, but the people behind it can’t even get the drivers out yet.

Check out Wired’s article on the UI of Magic Leap:

>Magic Leap UI: Totems: cameras on headset could track any piece of material that’s been defined as “mouse.”

>Shows keyboard made from soft rubber that would deform and provide haptic feedback while HMD overlaid images.

>AR system may render virtual computer keyboard on surface of rectangular aluminum

wired/com/2015/01/magic-leaps-vision-for-virtual-reality/

It's like the Optimus Maximum keyboard of 2008, where every keyboard key has an OLED display so that you can remap any key to do whatever you want.

It's like that, except it doesn't cost $2000.

Eye-tracking

An Android SDK for eye-tracking was revealed at CES.

dailydot/com/technology/eye-tribe-eye-tracking-android-sdk-ces-2015/

I think that modifying the camera for eye-tracking is cheap, and most likely much cheaper than changing the camera to a depth sensor like in Project Tango.

With surface touch, you still might be looking at the surface some of the time, like in the dual screen HP Sprout.

With eye-tracking, you could have an eye-tracking “select-what-am-looking-at” button.

Look, touch an easy-to-reach “tap-where-I’m-looking” button, look, and then touch the same button again.

You don’t have to keep changing your hand and finger positions between each tap.

For typing:

>Microsoft patents eye-tracking keyboard software

>The idea’s just like swipe-based keyboard software, but instead of tracking the motion of your fingertip, the system tracks eye movement.

pocketnow/com/2014/12/24/eye-tracking-keyboard

Automated lip reading

It’s a tough problem, but people are working on it:

>“The first problem for automated lip reading is face and lip recognition.

This has improved in leaps and bounds in recent years.

A more difficult challenge is in recognizing, extracting and categorizing the geometric features of the lips during speech.

This is done by measuring the height and width of the lips as well as other features such as the shape of the ellipse bounding the lips, the amount of teeth on view and the redness of the image, which determines the amount of tongue that is visible.

Hassanat’s own visual speech recognition system is remarkably good.

His experiments achieve an average success rate of 76 percent, albeit in carefully controlled conditions.

The success rate is even higher for women because of the absence of beards and mustaches.

All this suggests that there is significant potential for visual speech recognition systems in the future, particularly as an aid to other forms of speech recognition.”

– Ahmad Hassanat at Mu’tah University in Jordan

technologyreview/com/view/530641/the-challenges-and-threats-of-automated-lip-reading/

EEG

I’ve heard about the Emotiv EEG.

Just a few basic commands that are paired up with other inputs, like eye-tracking, would be useful.

E.g. of some adequate, basic commands:

* 1) open menu of on-screen shortcuts

* 2) select-what-I’m-looking-at (using eye tracker)

* 3) no/back/cancel

What Emotiv offers:

> “In order to provide consistency and a simple range of possible actions, each user profile will contain space for training data for up to 15 different commands, which are internally labelled COMMAND1 to COMMAND15.

With an eye tracker, you probably don’t need that many brain commands if you’re just going to be using the mind for a “Yes,-select-what-I'm-looking-at” or “back” for on-screen, virtual buttons.

Thought functions merely as the switch.