Source: https://patents.google.com/patent/US9092093
Timestamp: 2018-02-21 09:39:57
Document Index: 97833706

Matched Legal Cases: ['Application No. 61', 'Application No. 61', 'Application No. 61', 'Application No. 61', 'Application No. 61', 'Application No. 61']

US9092093B2 - Steering wheel user interface - Google Patents
US9092093B2
US9092093B2 US14590010 US201514590010A US9092093B2 US 9092093 B2 US9092093 B2 US 9092093B2 US 14590010 US14590010 US 14590010 US 201514590010 A US201514590010 A US 201514590010A US 9092093 B2 US9092093 B2 US 9092093B2
US14590010
US20150123947A1 (en )
A steering wheel that identifies gestures performed on its surface, including a circular gripping element including a thumb-receiving notch disposed along its circumference, an array of light-based proximity sensors, mounted in the gripping element, that projects light beams through the notch radially outward from the gripping element, and detects light beams reflected back into the gripping element by a moving object at or near the notch, and a processor, coupled with the proximity sensor array, for determining polar angles along the circumference of the gripping element occupied by the object, responsive to light beams projected by the proximity sensor array and reflected back by the object being detected by the proximity sensor array.
This application is a continuation-in-part of U.S. application Ser. No. 14/551,096, entitled LIGHT-BASED CONTROLS IN A TOROIDAL STEERING WHEEL and filed on Nov. 24, 2014, by inventors Gunnar Martin Fröjdh, Simon Fellin, Thomas Eriksson, John Karlsson, Maria Hedin and Richard Berglind, the contents of which are hereby incorporated herein in their entirety.
This application is also a continuation-in-part of U.S. application Ser. No. 14/555,731, entitled DOOR HANDLE WITH OPTICAL PROXIMITY SENSORS and filed on Nov. 28, 2014, by inventors Sairam Iyer, Stefan Holmgren and Per Rosengren, the contents of which are hereby incorporated herein in their entirety.
U.S. application Ser. No. 14/555,731 is a continuation-in-part of U.S. application Ser. No. 14/312,787 entitled OPTICAL PROXIMITY SENSORS and filed on Jun. 24, 2014 by inventors Stefan Holmgren, Sairam Iyer, Richard Berglind, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, John Karlsson, Thomas Eriksson, Alexander Jubner, Remo Behdasht, Simon Fellin, Robin Åman and Joseph Shain, the contents of which are hereby incorporated herein in their entirety.
U.S. application Ser. No. 14/555,731 is a continuation-in-part of U.S. application Ser. No. 14/311,366 entitled OPTICAL TOUCH SCREENS and filed on Jun. 23, 2014 by inventors Robert Pettersson, Per Rosengren, Erik Rosengren, Stefan Holmgren, Lars Sparf, Richard Berglind, Thomas Eriksson, Karl Erik Patrik Nordström, Gunnar Martin Fröjdh, Xiatao Wang and Remo Behdasht, the contents of which are hereby incorporated herein in their entirety.
U.S. application Ser. No. 14/555,731 is a continuation-in-part of U.S. application Ser. No. 14/140,635 entitled LIGHT-BASED PROXIMITY DETECTION SYSTEM AND USER INTERFACE and filed on Dec. 26, 2013 by inventors Thomas Eriksson and Stefan Holmgren, the contents of which are hereby incorporated herein in their entirety.
International Application No. PCT/US14/40112 is claims priority benefit of U.S. Provisional Application No. 61/828,713 entitled OPTICAL TOUCH SCREEN SYSTEMS USING REFLECTED LIGHT and filed on May 30, 2013 by inventors Per Rosengren, Lars Sparf, Erik Rosengren and Thomas Eriksson; of U.S. Provisional Application No. 61/838,296 entitled OPTICAL GAME ACCESSORIES USING REFLECTED LIGHT and filed on Jun. 23, 2013 by inventors Per Rosengren, Lars Sparf, Erik Rosengren, Thomas Eriksson, Joseph Shain, Stefan Holmgren, John Karlsson and Remo Behdasht; of U.S. Provisional Application No. 61/846,089 entitled PROXIMITY SENSOR FOR LAPTOP COMPUTER AND ASSOCIATED USER INTERFACE and filed on Jul. 15, 2013 by inventors Richard Berglind, Thomas Eriksson, Simon Fellin, Per Rosengren, Lars Sparf, Erik Rosengren, Joseph Shain, Stefan Holmgren, John Karlsson and Remo Behdasht; of U.S. Provisional Application No. 61/929,992 entitled CLOUD GAMING USER INTERFACE filed on Jan. 22, 2014 by inventors Thomas Eriksson, Stefan Holmgren, John Karlsson, Remo Behdasht, Erik Rosengren, Lars Sparf and Alexander Jubner; of U.S. Provisional Application No. 61/972,435 entitled OPTICAL TOUCH SCREEN SYSTEMS and filed on Mar. 31, 2014 by inventors Sairam Iyer, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, Thomas Eriksson, Alexander Jubner and Joseph Shain; and of U.S. Provisional Application No. 61/986,341 entitled OPTICAL TOUCH SCREEN SYSTEMS and filed on Apr. 30, 2014 by inventors Sairam Iyer, Karl Erik Patrik Nordström, Lars Sparf, Per Rosengren, Erik Rosengren, Thomas Eriksson, Alexander Jubner and Joseph Shain, the contents of which are hereby incorporated herein in their entirety.
U.S. application Ser. No. 14/311,366 is a continuation of International Application No. PCT/US14/40579 entitled OPTICAL TOUCH SCREENS and filed on Jun. 3, 2014 by inventors Robert Pettersson, Per Rosengren, Erik Rosengren, Stefan Holmgren, Lars Sparf, Richard Berglind, Thomas Eriksson, Karl Erik Patrik Nordstrom, Gunnar Martin Fröjdh, Xiatao Wang and Remo Behdasht, the contents of which are hereby incorporated herein in their entirety.
Element FIGS.
100 4, 5
106 6, 7
107 6, 7
108 6, 7
109 6, 7
201 6, 8, 9
202 6, 8, 9
203 8, 9
205 8, 9
300 2-7
301 8, 9
302 8, 9
303 8, 9
304 8, 9
305 8-10
400 1 steering wheel
401 1 grip
402 1 right spoke
403 1 left spoke
404 1 bottom spoke
405 1 answer button
406 1 reject button
410 12, 14-19 steering wheel
411 2-5 steering wheel frame
412 2-5 top cover
413 2-5 thumb notch
414 2-7 PCB
415 2-5, 7 light baffle
416 3, 5 transparent cover section
417 3, 5 transparent cover section
418 12, 14, 15, 17, 18 finger
419 12, 14, 16-19 hand
420 12, 15 steering wheel surface
421-424, 428, 12, 15 hand/finger
431-434 movement directions
425 12 clock icon
426 12 finger
430 14 double-tap gesture
436 14-16 Illumination
437 14 movement of illumination
438 14, 17 tap gesture
440 6, 11 Processor
441-443 11 network client
444 11 message bus
601 10 light beam
602 3, 8-10 light beam
603 8, 9 light beam
604 8, 9 light beam
1001-1005 13 vehicle application state
1010-1019 13 vehicle application action
The outward facing light emitters are used to provide visual indications to the user by illuminating light transmissive portion 416 of the steering wheel cover, and emit light in the visible range. Lenses 300 are described in assignee's co-pending application U.S. Ser. No. 14/555,731, entitled DOOR HANDLE WITH OPTICAL PROXIMITY SENSORS, the contents of which are incorporated herein in their entirety by reference.
Methods for determining two-dimensional coordinates of an object detected by the disclosed proximity sensor array are described in assignee's co-pending application Ser. No. 14/312,787, entitled OPTICAL PROXIMITY SENSORS, and US Ser. No. 14/555,731, entitled DOOR HANDLE WITH OPTICAL PROXIMITY SENSORS, both incorporated herein in their entireties by reference. Because the present application is for a steering wheel and the proximity sensor array is arranged along an arc-shaped grip of the steering wheel, the determined coordinates are polar coordinates, including a polar angle and a radial coordinate. The polar angle corresponds to a coordinate along the proximity sensor array, which in application Ser. Nos. 14/312,787 and 14/555,731 is described as an x-axis coordinate. The radial coordinate corresponds to a distance from the proximity sensor array, which in application Ser. Nos. 14/312,787 and 14/555,731 is described as a y-axis coordinate.
Thumb-Tap Tap thumb on steering wheel rim Small Discrete
Thumb-Glide Glide thumb along steering Small Continuous
Thumb-Long- Hold thumb on steering wheel rim Small Continuous
Grab Grab hold of steering wheel rim Large Continuous
Rim-Tap Tap hand on steering wheel rim Large Discrete
Thumb-Tap small-object tap
Thumb-Glide small-object glide
Thumb-Long-Press small-object touch-and-hold
Grab large-object grab
Rim-Tap large-object tap
Gesture Additional Example Example
Gesture Components Parameters Context Command
Tap inside Thumb-Tap Thumb-tap Cruise Increase or
notch performed at top or control is decrease cruise
bottom of illuminated active control speed in
portion of illuminated 5 mph
segment of steering increments
Tap on Rim-Tap During Reactivate phone
steering phone call interaction, e.g.,
wheel outer when phone call
rim is active for set
Enables hanging
up the phone call
with a clockwise
Single Two Thumb- Thumb-taps have Vehicle is in Activate cruise
object Taps different time motion control and
double-tap stamps, similar illuminate
inside notch center angles location of
Single Two Rim-Taps Side of steering Car is not Activate Park
object wheel rim (left or moving, and Assist to park on
double-tap right) at which Park Assist left or right side
on steering double-tap is icon is of car, based on
wheel outer performed displayed on tapped side of
rim HUD rim
Multi-touch Two Thumb- Thumb-taps have Autonomous Activate
double-tap Taps similar time stamps, drive is not autonomous
inside notch different center active drive
Extended Thumb-Long- Thumb-long-press Cruise Increase or
touch inside Press performed at top or control is decrease cruise
notch bottom of illuminated active control speed in
portion of illuminated 1 mph
Grab Grab Autonomous Deactivate
drive is autonomous
active drive, and enter
Swipe Thumb-Glide clockwise/counter- Cruise Increase or
clockwise control is decrease distance
active from forward car
Radial swipe Thumb-Glide Thumb-glide data Cruise Open cruise
structures have control is control menu on
similar center angles active HUD
and different radial
Slide Grab Grab data structures Portion of Move illumination
have different time steering to new hand
stamps and different wheel is location (follow
center angles selectively slide movement)
Switch Grab Grab data structures Portion of Move illumination
hands have different time steering to new hand
stamps and different wheel is location (jump to
center angles selectively other side of
illuminated wheel)
1. A steering wheel that identifies gestures performed on its surface, comprising:
a circular gripping element comprising a thumb-receiving notch disposed along its circumference;
an array of light-based proximity sensors, mounted in said gripping element, that projects light beams through said notch radially outward from said gripping element, and detects light beams reflected back into said gripping element by a moving object at or near said notch; and
a processor, coupled with said proximity sensor array, for determining polar angles along the circumference of said gripping element occupied by the object, responsive to light beams projected by said proximity sensor array and reflected back by the object being detected by said proximity sensor array.
2. The steering wheel of claim 1 wherein said processor generates one or more data structures for the moving object, each data structure comprising:
a) a time stamp,
b) a polar angle at which the moving object starts,
c) a polar angle at which the moving object ends,
d) a middle polar angle of the moving object, and
e) an assigned state being one of the group RECOGNIZED, UPDATED and ENDED,
wherein said processor assigns the RECOGNIZED state to the data structure when the moving object is initially detected by said proximity sensor array,
wherein said processor assigns the UPDATED state to the data structure when the moving object is further detected by said proximity sensor array after the initial detection, and
wherein said processor assigns the ENDED state to the data structure when the moving object ceases to be detected by said proximity sensor array.
3. The steering wheel of claim 2 wherein said processor discriminates between large and small objects based on the polar angles in said data structure.
4. The steering wheel of claim 3 wherein said processor identifies a component gesture of the moving object as consisting of one of the types: (i) a small-object tap gesture, (ii) a small-object glide gesture, (iii) a small-object touch-and-hold gesture, (iv) a large-object tap gesture, and (v) a large-object grab gesture.
5. The steering wheel of claim 4 wherein said processor combines at least two component gestures of the same type, having assigned states ENDED, to further identify a compound gesture.
6. The steering wheel of claim 4 wherein said processor combines at least two component gestures of the same type, having assigned states RECOGNIZED and UPDATED, respectively, to further identify a compound gesture.
7. The steering wheel of claim 6 wherein the compound gesture is a double-tap gesture, combining two small-object tap gestures.
8. The steering wheel of claim 7 wherein said processor activates a cruise control application in response to identifying the double-tap gesture.
9. The steering wheel of claim 8 further comprising visible light illumination means, connected to said processor, for selectively illuminating portions of said notch, wherein the activated cruise control application illuminates a portion of said notch at which the double-tap gesture was performed, and wherein in response to a small-object tap gesture at one end of the illuminated portion, the cruise control application increases a cruise control speed by a first increment, and in response to a small-object tap gesture at the other end of the illuminated portion, the cruise control application decreases the cruise control speed by the first increment.
10. The steering wheel of claim 9 wherein in response to a small-object touch-and-hold gesture at one end of the illuminated portion, the cruise control application increases the cruise control speed by a second increment, different than the first increment, and in response to a small-object touch-and-hold gesture at the other end of the illuminated portion, the cruise control application decreases the cruise control speed by the second increment.
11. The steering wheel of claim 6 wherein the compound gesture is a swipe gesture, combining two small-object glide gestures having different middle polar angles.
12. The steering wheel of claim 11 wherein said processor is in communication with a telephone, and in response to a first swipe gesture said processor issues a command to the telephone to answer an incoming call.
13. The steering wheel of claim 12 wherein in response to a second swipe gesture said processor issues a command to the telephone to hang up the answered call.
14. The steering wheel of claim 13 wherein when the answered call continues for a time interval, said processor only issues the hang up command when the second swipe gesture is preceded by a large-object tap gesture.
15. The steering wheel of claim 12 further comprising visible light illumination means, connected to said processor, for selectively illuminating portions of said notch, wherein the processor illuminates at least a portion of said notch to indicate the incoming call.
16. The steering wheel of claim 6 wherein the compound gesture is a slide gesture, combining two consecutive large-object grab gestures having different middle polar angles.
17. The steering wheel of claim 16, further comprising visible light illumination means, connected to said processor, for selectively illuminating portions of said notch, wherein in response to a detected slide gesture the illuminated portions track the middle polar angles of the slide gesture.
18. The steering wheel of claim 6 wherein the compound gesture is an extended touch gesture, combining two consecutive small-object touch-and-hold gestures.
19. The steering wheel of claim 1 wherein said processor further determines a radial coordinate of the object, responsive to light beams projected by said proximity sensor array and reflected back by the object being detected by said proximity sensor array.
20. The steering wheel of claim 19 wherein said processor combines at least two component gestures of the same type, having similar middle polar angles and different radial coordinates, respectively, to further identify a radial swipe gesture.
21. The steering wheel of claim 19 wherein in response to the radial swipe gesture, the processor opens a menu of options for an active application on a display connected to the steering wheel.
US14590010 2009-02-15 2015-01-06 Steering wheel user interface Active US9092093B2 (en)
US201261730139 true 2012-11-27 2012-11-27
US14590010 US9092093B2 (en) 2012-11-27 2015-01-06 Steering wheel user interface
EP20150160235 EP2937263B1 (en) 2014-03-31 2015-03-23 Steering wheel user interface
US14805445 US9710144B2 (en) 2012-11-27 2015-07-21 User interface for curved input device
US15647693 US20170308281A1 (en) 2012-11-27 2017-07-12 Autonomous drive user interface
US14551096 Continuation-In-Part US9389710B2 (en) 2001-11-02 2014-11-24 Light-based controls on a toroidal steering wheel
US14805445 Continuation US9710144B2 (en) 2009-02-15 2015-07-21 User interface for curved input device
US20150123947A1 true US20150123947A1 (en) 2015-05-07
US9092093B2 true US9092093B2 (en) 2015-07-28
ID=53006692
US14590010 Active US9092093B2 (en) 2009-02-15 2015-01-06 Steering wheel user interface
US14805445 Active US9710144B2 (en) 2009-02-15 2015-07-21 User interface for curved input device
US15647693 Pending US20170308281A1 (en) 2009-02-15 2017-07-12 Autonomous drive user interface
US (3) US9092093B2 (en)
US20150324111A1 (en) * 2012-11-27 2015-11-12 Neonode Inc. Steering wheel user interface
EP2853470B1 (en) * 2013-09-26 2017-03-15 FERRARI S.p.A. Motor vehicle steering wheel provided with controls with rear projection
JP2016016767A (en) * 2014-07-09 2016-02-01 ユニバーシティ・オブ・タンペレＵｎｉｖｅｒｓｉｔｙ ｏｆ Ｔａｍｐｅｒｅ Tactile type device
JPH03216719A (en) 1990-01-22 1991-09-24 Fujitsu Ltd Position instruction device
EP1059603A2 (en) 1999-06-07 2000-12-13 Kabushiki Kaisha Tokai-Rika-Denki-Seisakusho Touch position determining method
US20020158453A1 (en) 1998-09-23 2002-10-31 Richard Levine Vehicle air-bag minimum distance enforcement apparatus, method and system
US20040199309A1 (en) 2003-04-03 2004-10-07 Coato Workshop, Inc. Navigational method and device for user interface for motor vehicles
US6944557B2 (en) 2002-01-31 2005-09-13 Fujitsu Limited Ultrasonic length measuring apparatus and method for coordinate input
WO2010093570A1 (en) 2009-02-15 2010-08-19 Neonode Inc. Light-based touch screen
WO2010121031A1 (en) 2009-04-16 2010-10-21 Neonode Inc. Optical touch screen systems using reflected light
US20120179328A1 (en) 2011-01-12 2012-07-12 GM Global Technology Operations LLC Steering wheel system
US20120283894A1 (en) 2001-10-24 2012-11-08 Mouhamad Ahmad Naboulsi Hands on steering wheel vehicle safety control system
US20130024071A1 (en) 2011-07-22 2013-01-24 Clas Sivertsen Steering Wheel Input Device Having Gesture Recognition and Angle Compensation Capabilities
GB1107666A (en) 1966-03-30 1968-03-27 Cosmic Car Accessories Ltd Steering wheels
GB9625262D0 (en) 1996-12-04 1997-01-22 Autoliv Dev Improvements in or relating to an air-bag arrangement
US9092093B2 (en) * 2012-11-27 2015-07-28 Neonode Inc. Steering wheel user interface
JP5375747B2 (en) 2010-06-08 2013-12-25 株式会社デンソー Biometric information detection system
JP2012181639A (en) 2011-03-01 2012-09-20 Stanley Electric Co Ltd Operation input device
JP6095478B2 (en) 2013-05-16 2017-03-15 スタンレー電気株式会社 The input device
JP6229419B2 (en) 2013-10-03 2017-11-15 スズキ株式会社 Handle switch device
Moeller et al., IntangibleCanvas: Free-Air Finger Painting on a Projected Canvas, CHI EA '11 Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems, pp. 1615-1620, ACM, New York, NY, May 2011.
Moeller et al., ZeroTouch: A Zero-Thickness Optical Multi-Touch Force Field, CHI EA '11 Proceedings of the 2011 Annual Conference Extended Abstracts on Human Factors in Computing Systems, pp. 1165-1170, ACM, New York, NY, May 2011.
Moeller et al., ZeroTouch: An Optical Multi-Touch and Free-Air Interaction Architecture, Proc. CHI 2012 Proceedings of the 2012 Annual Conference Extended Abstracts on Human Factors in Computing Systems, pp. 2165-2174, ACM, New York, NY, May 5, 2012.
Myers et al., Two-Handed Input Using a PDA and a Mouse, CHI Letters, vol. 2, Issue 1, CHI 2000, Apr. 1-6, 2000.
Myers, Mobile Devices for Control, Mobile HCI 2002, LNCS 2411, pp. 1-8, Springer-Verlag, Berlin, Heidelberg, 2002.
Myers, Using Handhelds and PCs Together, Communications of the ACM, vol. 44, No. 11, ACM, New York, NY, Nov. 2001.
US9710144B2 (en) * 2012-11-27 2017-07-18 Neonode Inc. User interface for curved input device
US20150123947A1 (en) 2015-05-07 application
US20170308281A1 (en) 2017-10-26 application
US20150324111A1 (en) 2015-11-12 application
US9710144B2 (en) 2017-07-18 grant
US20130024071A1 (en) 2013-01-24 Steering Wheel Input Device Having Gesture Recognition and Angle Compensation Capabilities
JP2001282457A (en) 2001-10-12 Coordinate input system, control method of coordinate input system and computer readable recording medium in which a program to make computer execute the method is recorded
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:JUBNER, ALEXANDER;ERIKSSON, THOMAS;FROJDH, GUNNAR MARTIN;AND OTHERS;SIGNING DATES FROM 20150106 TO 20150108;REEL/FRAME:034693/0431