Source: https://patents.google.com/patent/US9746923
Timestamp: 2018-04-22 09:20:11
Document Index: 561083541

Matched Legal Cases: ['Application No. 10712198', 'Application No. 2015', 'Application No. 2015', 'Application No. 2015', 'Application No. 10', 'Application No. 10']

US9746923B2 - Systems and methods for providing features in a friction display wherein a haptic effect is configured to vary the coefficient of friction - Google Patents
Systems and methods for providing features in a friction display wherein a haptic effect is configured to vary the coefficient of friction Download PDF
US9746923B2
US9746923B2 US12696893 US69689310A US9746923B2 US 9746923 B2 US9746923 B2 US 9746923B2 US 12696893 US12696893 US 12696893 US 69689310 A US69689310 A US 69689310A US 9746923 B2 US9746923 B2 US 9746923B2
US20100231367A1 (en )
This patent application is related to U.S. patent application Ser. No. 12/697,010, filed the same day as the present application and entitled “Systems and Methods for a Texture Engine,” which is incorporated by reference herein in its entirety.
This patent application is related to U.S. patent application Ser. No. 12/697,042, filed the same day as the present application and entitled “Systems and Methods for Using Multiple Actuators to Realize Textures,” which is incorporated by reference herein in its entirety.
This patent application is related to U.S. patent application Ser. No. 12/697,037, filed the same day as the present application and entitled “Systems and Methods for Using Textures in Graphical User Interface Widgets,” which is incorporated by reference herein in its entirety.
This patent application is related to U.S. patent application Ser. No. 12/696,900, filed the same day as the present application and entitled “Systems and Methods for Providing Features in a Friction Display,” which is incorporated by reference herein in its entirety.
This patent application is related to U.S. patent application Ser. No. 12/696,908, filed the same day as the present application and entitled “Systems and Methods for Interfaces Featuring Surface-Based Haptic Effects,” which is incorporated by reference herein in its entirety.
Additional detail regarding generation and use of textures can be found in U.S. patent application Ser. Nos. 12/697,010, 12/697,042, and 12/697,037, referenced above and entitled “Systems and Methods for a Texture Engine,” “Systems and Methods for Using Multiple Actuators to Realize Textures,” and “Systems and Methods for Using Textures in Graphical User Interface Widgets,” respectively. For instance, patterns of differing friction or patterns of vibration may be provided to mimic the feeling of textures such as brick, rocks, sand, grass, fur, various fabric types, water, molasses, and other fluids, leather, wood, ice, lizard skin, metals, and other texture patterns. Other textures not analogous to real-world textures may also be used, such as high-magnitude vibrotactile or other feedback when a “danger” texture is desired.
an actuator coupled to the touch surface, the actuator configured to receive a haptic signal and output a haptic effect; and
receive haptic parameters associated with a location on the touch surface;
map the position of the touch to the location on the touch surface;
determine a haptic effect to simulate a presence of a feature based in part on a color and a degree of shading at the location on the touch surface, wherein the haptic effect is configured to vary a coefficient of friction on the touch surface at the position of the touch, wherein the degree of shading corresponds to a level of the coefficient of friction; and
transmit a haptic signal to generate the haptic effect using the actuator.
2. The system set forth in claim 1, wherein the feature further comprises a texture at or near the determined position of the touch.
3. The system set forth in claim 1, wherein the feature comprises a boundary between a first region and a second region in a graphical user interface displayed in the display.
4. The system set forth in claim 3, wherein the boundary comprises a boundary between two simulated keys of a keyboard.
5. The system set forth in claim 1, wherein determining the haptic effect further comprises:
6. The system set forth in claim 5, wherein the selected position is the position of the touch.
7. The system set forth in claim 5, wherein the selected position is determined based on the position of the touch and a velocity of the touch.
8. The system set forth in claim 1, wherein the actuator is configured to displace the touch surface vertically, laterally, or vertically and laterally at an ultrasonic frequency.
9. The system set forth in claim 8, wherein the actuator comprises at least one of:
10. The system set forth in claim 1, wherein the processor is further configured to determine a pressure of the touch based on data from the sensor, wherein the haptic effect is selected based at least in part on the pressure of the touch.
displaying an image in a display area comprising a plurality of pixels;
detecting a touch in a touch area associated with the display area in response to contact between an object and a touch surface;
receiving haptic parameters associated with a location on the touch surface;
mapping the position of the touch to a location on the touch surface;
determining a haptic effect to simulate a presence of a feature based in part on a color and a degree of shading at the location on the touch surface, wherein the haptic effect is configured to vary a coefficient of friction on the touch surface at the position of the touch, wherein the degree of shading corresponds to a level of the coefficient of friction; and
12. The method set forth in claim 11, wherein the feature further comprises a texture at or near the determined position of the touch.
13. The method set forth in claim 11, wherein the feature comprises a boundary between a first region displayed in a display area and a second region in the display area, the display area mapped to the touch area.
14. The method set forth in claim 13, wherein the boundary comprises a boundary between two keys of a keyboard depicted in the display area.
15. The method set forth in claim 11, wherein selecting a haptic effect comprises:
16. The method set forth in claim 15, wherein the selected position is the position of the touch or is determined based on the position of the touch and a velocity of the touch.
17. The method set forth in claim 11, wherein the actuator is configured to displace the touch surface vertically, laterally, or vertically and laterally at an ultrasonic frequency.
18. The method set forth in claim 17, wherein the actuator comprises at least one of:
19. A non-transitory computer-readable medium embodying program code, which when executed by a processor is configured to cause the processor to:
display an image in a display area comprising a plurality of pixels;
track a position of a touch on a touch surface associated with the display area;
map the position of the touch to a location on the touch surface;
transmit a haptic signal to at least one actuator to generate the feature by varying a coefficient of friction of the touch surface.
20. The system of claim 1, wherein varying the coefficient of friction of the touch surface comprises increasing the coefficient of friction of the touch surface.
21. The method of claim 11, wherein varying the coefficient of friction of the touch surface comprises increasing the coefficient of friction of the touch surface.
22. The non-transitory computer readable medium of claim 19, wherein varying the coefficient of friction of the touch surface comprises increasing the coefficient of friction of the touch surface.
JP2017176473A JP2018026144A (en) 2009-03-12 2017-09-14 System and method for interface characterized by Surface-based haptic effect
JP2017194165A JP2018032417A (en) 2009-03-12 2017-10-04 The system and method using a plurality of actuators in order to realize the texture
US20100231367A1 true US20100231367A1 (en) 2010-09-16
US9746923B2 true US9746923B2 (en) 2017-08-29
JP (20) JP5694204B2 (en)
WO2013032302A3 (en) 2011-09-01 2013-04-25 Lg Innotek Co., Ltd. Touch panel and liquid crystal display comprising the same
US20160018894A1 (en) 2013-03-20 2016-01-21 Nokia Technologies Oy A Touch Display Device with Tactile Feedback
GB201315165D0 (en) * 2013-08-24 2013-10-09 Gandhi Paavan Braille upon the surface of a visual display unit
EP3293616A1 (en) * 2015-05-07 2018-03-14 Fujitsu Limited Electronic device and vibration control program
US5144187A (en) 1990-03-23 1992-09-01 Rockwell International Corporation Piezoelectric motor
US20020017833A1 (en) 2000-06-30 2002-02-14 C.R.F. Societa Consortile Per Azioni Self-compensating piezoelectric actuator for a control valve
US20040233162A1 (en) 2003-05-19 2004-11-25 Alps Electric Co., Ltd. Force-feedback input device
US20060061558A1 (en) 2004-09-20 2006-03-23 Danny Grant Products and processes for providing multimodal feedback in a user interface device
US20060267416A1 (en) 2005-05-31 2006-11-30 Canon Kabushiki Kaisha Vibration wave motor
US20070021961A1 (en) 2005-07-19 2007-01-25 Samsung Electronics Co., Ltd. Audio reproduction method and apparatus supporting audio thumbnail function
US20070036450A1 (en) 2000-05-09 2007-02-15 Tetsujiro Kondo Data processing apparatus and method and recording medium
US20070069611A1 (en) 2005-09-27 2007-03-29 Samsung Techwin Co., Ltd. Piezoelectric actuator, and apparatus and method for actuating the same
US20090134744A1 (en) 2007-11-27 2009-05-28 Korea Institute Of Science And Technology Ring type piezoelectric ultrasonic resonator and piezoelectric ultrasonic rotary motor using the same
US20090227296A1 (en) 2008-03-10 2009-09-10 Lg Electronics Inc. Terminal and method of controlling the same
US20100026976A1 (en) 2008-07-30 2010-02-04 Asml Holding N.V. Actuator System Using Multiple Piezoelectric Actuators
KR20080092482A (en) 2004-11-30 2008-10-15 임머숀 코퍼레이션 Systems and methods for controlling a resonant device for generating vibrotactile haptic effects
US20100315212A1 (en) 2008-02-04 2010-12-16 Nokia Corporation Device and method for providing tactile information
CN106125973A (en) 2009-03-12 2016-11-16 意美森公司 Systems and methods for providing features in friction display
JP2015130183A (en) 2009-03-12 2015-07-16 イマージョン コーポレーションＩｍｍｅｒｓｉｏｎ Ｃｏｒｐｏｒａｔｉｏｎ System and method for providing feature in friction display
ANNA DE WITT: "Designing Sonification of User Data in Affective Interaction", THESIS, 1 January 2007 (2007-01-01), pages 1 - 92, XP002551466, Retrieved from the Internet <URL:https://www.nada.kth.se/utbildning/grukth/exjobb/rapportlistor/2007/rapporter07/de_witt_anna_07142.pdf> [retrieved on 20091021]
Biet, M. et al., "Discrimination of Virtual Square Gratings by Dynamic Touch on Friction Based Tactile Displays," Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2008, Symposium, IEEE, Piscataway, NJ, XP 031339918, pp. 41-48, Mar. 2008.
Chakrabarti et al., "Rendering Color Information using Haptic Feedback", University of British Columbia CS Technical Report, Retrieved from httpsilvvww.c.s.ubc.calcgi-binttri2004flR-2004-, Jul. 22, 2004, 10 pages.
Chowdhury, M.A. et al., "The Effect of Amplitude of Vibration on the Coefficient of Friction for Different Materials," Tribology International, 2008, 41:307-314.
Chowdhury, Mohammad Asaduzzaman, Maksud Helali. "The effect of amplitude of vibration on the coefficient of friction for different materials." Tribology International. vol. 41, issue 4 (Apr. 2008): 307-317. *
Corel Paint Shop Pro Photo X2 Reviewer's Guide, Copyright 2007, web page at http:/ /web.archive.org/web/20071102133023/http://www.corel.com/conte nt!vpk!psppx2/PS P PX2Reviewer-Guide.pdf, as available via the Internet and accessed Apr. 7, 2012.
Corel Paint Shop Pro Photo X2 Reviewer's Guide, Copyright 2007, web page at http:/ /web.archive.org/web/20071102133023/http://www.corel.com/conte nt!vpk!psppx2/PS P PX2Reviewer—Guide.pdf, as available via the Internet and accessed Apr. 7, 2012.
De Witt, A., "Designing Sonification of User Data in Affective Interaction," Master of Science Thesis Stockholm, Sweden, XP 002551466, at hppt://w3.nada.kth.se/utbildning/grukth/exjobb/rapportlistor/2007/de-witt-anna-07142, as available via the Internet and printed Oct. 20, 2009.
European Patent Office Application No. 10712198.0, Office Action dated Feb. 27, 2017, 5 pages.
IAN OAKLEY, SILE O'MODHRAIN: "Contact IM: Exploring Asynchronous Touch Over Distance", PROCEEDINGS OF CSCW 2002, 1 January 2002 (2002-01-01), XP007910188, Retrieved from the Internet <URL:http://www.whereveriam.org/work/palpable/ContactIM.pdf> [retrieved on 20091015]
International Preliminary Report on Patentability mailed Sep. 22, 2011 for corresponding International Application No. PCT/US2010/026894.
International Preliminary Report on Patentability mailed Sep. 22, 2011 for corresponding International Application No. PCT/US2010/026897.
International Preliminary Report on Patentability mailed Sep. 22, 2011 for corresponding International Application No. PCT/US2010/026900.
International Preliminary Report on Patentability mailed Sep. 22, 2011 for corresponding International Application No. PCT/US2010/026905.
International Preliminary Report on Patentability mailed Sep. 22, 2011 for corresponding International Application No. PCT/US2010/026907.
International Preliminary Report on Patentability mailed Sep. 22, 2011 for corresponding International Application No. PCT/US2010/026909.
Japanese Patent Office Application No. 2015-083829, Notice of Decision to Grant dated Feb. 7, 2017, 3 pages.
Japanese Patent Office Application No. 2015-083829, Office Action dated Jun. 21, 2016, 3 pages and translation 3 pages.
Japanese Patent Office Application No. 2015-083829, Office Action dated Nov. 8, 2016, 2 pages.
Japanese Patent Office, Notice of Reasons for Rejection, Application No. JP2011-554172 dated Jan. 28, 2014, 3 pages.
Japanese Patent Office, Notice of Reasons for Rejection, Application No. JP2012-539970 dated Apr. 1, 2014, 3 pages.
Japanese Patent Office, Notice of Reasons for Rejection, Application No. JP2015-019308 dated Mar. 1, 2016, 4 pages.
KAARESOJA T, BROWN L M, LINJAMA J: "Snap-Crackle-Pop: Tactile Feedback for Mobile Touch Screens", PROCEEDINGS OF EUROHAPTICS, PROCEEDINGS OF THE EUROHAPTICS 2006, JULY 2006, 1 July 2006 (2006-07-01), Proceedings of the Eurohaptics 2006, July 2006, pages 1 - 2, XP002551465, Retrieved from the Internet <URL:http://lsc.univ-evry.fr/~eurohaptics/upload/cd/papers/f80.pdf> [retrieved on 20091020]
Kaaresoja, T. et al., "Snap-Crackle-Pop: Tactile Feedback for Mobile Touch Screens," Proceedings of the Eurohaptics 2006, XP 002551465, at http://lsc.univ-evry.fr/(eurohaptics/upload/cd/papers/f80, as available via the Internet and printed Oct. 20, 2009.
Korean Patent Office Application No. 10-2016-7024778, Office Action dated Apr. 28, 2017.
Korean Patent Office, Notice of Last Preliminary Rejection, Application No. 10-2011-7023987 dated Nov. 29, 2016.
Korean Patent Office, Notice of Preliminary Rejection, Application No. KR10-2011-7023987 dated Apr. 27, 2016.
Korean Patent Office, Notice of Preliminary Rejection, Application No. KR10-2011-7023987 dated Aug. 21, 2015.
Korean Patent Office, Notice of Preliminary Rejection, Application No. KR10-2012-7015581 dated Apr. 12, 2016.
Maeno, T. et al., "Tactile Display of Surface Texture by use of Amplitude Modulation of Ultrasonic Vibration," IEEE Ultrasonics Symposium, 2006, pp. 62-65.
Minsky, Margaret Diane Rezvan, "Computational Haptics: The Sandpaper System for Synthesizing Texture for a Force-Feedback Display," Ph.D. Dissertation, Massachusetts Institute of Technology, Jun. 1995, 217 pages.
Office Action mailed Jan. 15, 2013 for corresponding U.S. Appl. No. 12/696,900.
Office Action mailed Jan. 15, 2013 for corresponding U.S. Appl. No. 12/696,908.
Office Action mailed Jan. 15, 2013 for corresponding U.S. Appl. No. 12/697,037.
Office Action mailed Nov. 15, 2012 for corresponding U.S. Appl. No. 12/947,532.
Rovers, A. et al., HIM: A Framework for Haptic Instant Messaging, CHI 2004 (CHI Conference Proceedings, Human Factors in Computing Systems), XP 002464573, Vienna Austria, Apr. 2004, p. 1313-1316.
Search Report and Written Opinion mailed Jun. 8, 2010 for corresponding PCT Application No. PCT/US2010/026894.
Search Report and Written Opinion mailed Jun. 8, 2010 for corresponding PCT Application No. PCT/US2010/026897.
Search Report and Written Opinion mailed Jun. 8, 2010 for corresponding PCT Application No. PCT/US2010/026900.
Search Report and Written Opinion mailed Jun. 8, 2010 for corresponding PCT Application No. PCT/US2010/026905.
Search Report and Written Opinion mailed Jun. 8, 2010 for corresponding PCT Application No. PCT/US2010/026907.
Search Report and Written Opinion mailed Jun. 8, 2010 for corresponding PCT Application No. PCT/US2010/026909.
Sekiguchi, Y. et al., "Haptic Interface using Estimation of Box Contents Metaphor," Proceedings of ICAT 2003, Tokyo, Japan, XP 002551467, at http://www.vrsj.org/ic-at/papers/2003/00947-00000, as available via the Internet and printed Oct. 20, 2009.
Tang, H. et al., "A Microfabricated Electrostatic Haptic Display for Persons with Visual Impairments," IEEE Transactions on Rehabilitation Engineering, Sep. 1998, 6(3):241-314.
The State Intellectual Property Office of the Peoples Republic of China, Notification of First Office Action, Application No. CN201080011744.3 dated Aug. 12, 2015, 6 pages.
The State Intellectual Property Office of the Peoples Republic of China, Notification of First Office Action, Application No. CN201080011744.3 dated Dec. 16, 2014, 7 pages.
The State Intellectual Property Office of the Peoples Republic of China, Notification of First Office Action, Application No. CN201080011744.3 dated Sep. 23, 2013, 8 pages.
The State Intellectual Property Office of the Peoples Republic of China, Notification of First Office Action, Application No. CN201080051254.6 dated Jun. 13, 2014, 11 pages.
The State Intellectual Property Office of the Peoples Republic of China, Notification of Fourth Office Action, Application No. CN201080051254.6 dated Dec. 31, 2015, 3 pages.
The State Intellectual Property Office of the Peoples Republic of China, Notification of Second Office Action, Application No. CN201080011744.3 dated Jun. 24, 2014, 13 pages.
The State Intellectual Property Office of the Peoples Republic of China, Notification of Second Office Action, Application No. CN201080051254.6 dated Nov. 27, 2014, 7 pages.
The State Intellectual Property Office of the Peoples Republic of China, Notification of Third Office Action, Application No. CN201080051254.6 dated Jul. 2, 2015, 12 pages.
U.S. Appl. No. 12/696,908, Non Final Office Action dated Mar. 31, 2017.
U.S. Appl. No. 12/947,532, Non-Final Office Action dated May 23, 2017.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/696,900 dated Oct. 28, 2016, 3 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/696,900, dated Aug. 19, 2016, 11 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/696,900, dated Dec. 12, 2016, 11 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/696,900, dated May 9, 2016, 12 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/696,908, dated Mar. 27, 2012, 24 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/696,908, dated May 26, 2016, 14 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/696,908, dated Nov. 10, 2016, 14 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,010, dated Apr. 11, 2012, 22 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,010, dated Dec. 7, 2016, 3 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,010, dated Jan. 18, 2013, 15 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,010, dated May 20, 2016, 12 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,010, dated Sep. 16, 2016, 18 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,037, dated Apr. 12, 2012, 38 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,037, dated Jun. 6, 2016, 14 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,042, dated Apr. 2, 2012, 21 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,042, dated Feb. 5, 2013, 46 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,042, dated May 17, 2016, 43 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/697,042, dated Nov. 30, 2016, 20 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/947,321, dated Apr. 3, 2014, 29 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/947,321, dated Mar. 18, 2013, 30 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/947,321, dated Oct. 9, 2013, 23 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/947,532, dated Mar. 11, 2014, 17 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/947,532, dated Mar. 12, 2013, 19 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/947,532, dated Sep. 27, 2013, 14 pages.
United States Patent and Trademark Office, Office Action , U.S. Appl. No. 12/947,532, dated Sep. 9, 2016, 26 pages.
Watanabe, T. et al., "A Method for Controlling Tactile Sensation of Surface Roughness Using Ultrasonic Vibration," IEEE International Conference on Robotics and Automation, 0-7803-1965-6/95, 1995, pp. 1134-1139.
WILLIAMSON J, MURRAY-SMITH R, HUGHES S: "Shoogle: Excitatory Multimoda Interaction on Mobile Devices", PROCEEDINGS OF THE SIGCHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS 2007, CHI 2007 2007 ASSOCIATION FOR COMPUTING MACHINERY US, NEW YORK, NY : ACM, 2 PENN PLAZA, SUITE 701 NEW YORK NY 10121-0701 USA, 28 April 2007 (2007-04-28) - 03-05-2007, 2 Penn Plaza, Suite 701 New York NY 10121-0701 USA, pages 121 - 124, XP002549378, ISBN: 978-1-59593-593-9, DOI: 10.1145/1240624.1240642
Williamson, J. et al., "Shoogle: Excitatory Multimodal Interaction on Mobile Devices," CHI 2007 Proceedings-Shake, Rattle and Roll: New Forms of Input and Output, 2007, pp. 121-124, XP002549378.
YUICHIRO SEKIGUCHI, KOICHI HIROTA, MICHITAKA HIROSE: "Haptic Interface Using "Estimation of Box Contents" Metaphor", PROCEEDINGS OF ICAT 2003, DECEMBER 3-5, TOKYO, JAPAN, 3 December 2003 (2003-12-03) - 5 December 2003 (2003-12-05), pages 1 - 6, XP002551467, Retrieved from the Internet <URL:http://www.vrsj.org/ic-at/papers/2003/00947_00000.pdf> [retrieved on 20091022]