User interface device provided with surface haptic sensations

A user interface device includes a housing, a user input element supported by the housing, and a haptic output device supported by the housing. The haptic output device is configured to generate a haptic effect at a surface of the user interface device. The surface is part of the housing and/or the user input element. A processor disposed within the housing. The processor is configured to receive an input command from the user input element, communicate the input command to a host computer, receive an output command from the host computer, and output a haptic signal based on the output command to the haptic output device to generate the haptic effect at the surface.

FIELD

The present invention is related to a user interface device provided with surface haptic sensations.

BACKGROUND

Current user input elements of user interface devices, such as gaming peripherals, typically have a fixed static feel according to their material properties. For example, gaming peripherals, such as gamepads, are used for a wide range of games and are used for control in a wide range of gaming situations. Despite the great variety of situations in which a gamepad is used in a game, the traditional user input elements, such as joysticks, buttons, d-pads, and triggers, all have a fixed mechanical surface feel that is dependent on the materials that are used to manufacture them. In addition, standard gaming peripherals currently only use whole body vibration to provide tactile feedback to the user.

SUMMARY

It is desirable to have the ability to change the feel of the surface features on surfaces of user interface devices, such as the surfaces of user input elements of gaming peripherals, to provide a whole new experience for game players. For example, it is desirable to have the ability to change the surface properties of the device according to the function the user is controlling, or the game situation the user is experiencing, to enhance the immersion feeling in the game. By providing the surfaces of user input elements on gaming peripherals to have, for example, a programmable friction sensation, the user input elements may communicate additional information to the user, such as game state. This may also allow a controller of a gaming system to adapt to the gaming scenario currently engaged in to make the gaming peripheral, such as a gamepad or wand, feel more like the virtual object the user is experiencing.

According to an aspect of the invention, there is provided a user interface device that includes a housing, a user input element supported by the housing, and a haptic output device supported by the housing. The haptic output device is configured to generate a haptic effect at a surface of the user interface device. The surface is part of the housing and/or the user input element. The user interface device includes a processor disposed within the housing. The processor is configured to receive an input command from the user input element, communicate the input command to a host computer, receive an output command from the host computer, and output a haptic signal based on the output command to the haptic output device to generate the haptic effect at the surface.

According to an aspect of the invention, there is provided a system that includes a display configured to display video content, a host computer comprising a first processor configured to output the video content to the display, and a user interface device in communication with the host computer. The user interface device includes a housing, a user input element supported by the housing, and a haptic output device supported by the housing. The haptic output device is configured to generate a haptic effect at a surface of the user interface device. The surface is part of the housing and/or the user input element. The user interface device includes a second processor disposed within the housing. The processor is configured to receive an input command from the user input element, communicate the input command to the host computer, receive an output command from the host computer, and output a haptic signal based on the output command to the haptic output device to generate the haptic effect at the surface.

DETAILED DESCRIPTION

FIG. 1schematically illustrates an embodiment of a system100in accordance with embodiments of the invention. As illustrated, the system100includes a user interface device110, a host computer140, and a display160. In an embodiment, the system100is a gaming system, the user input device110is a gaming peripheral or controller, and the host computer140is a gaming console, and as discussed in further detail below.

The user interface device110, which is illustrated in further detail inFIG. 2, includes a housing112, user input elements114supported by the housing112, a haptic output device116(FIG. 2) supported by the housing112, and a processor118(FIG. 2) disposed within the housing112. In an embodiment, the user interface device110may be a computer mouse. In an embodiment, the user interface device110may be a gaming peripheral, such as a gamepad, a wand, a remote control, a force sensitive plate, a steering wheel, a computer mouse, or any other device that is configured to allow a user to provide input to the host computer140and receive haptic effects from the haptic output device116, as discussed in further detail below. For example, the user interface110may be a wearable device, such as a pair of glasses configured to allow a user to provide input to the host computer140and/or receive haptic effects from the haptic output device116.

The housing112may be formed from one or more materials that provide different functionalities to the user interface device110. For example, a portion of the housing112may include a stiff polymeric material that is suitable for supporting the user input elements114, the haptic output device116, and the processor118. In an embodiment, a portion of the housing112may include a conductive material so that the haptic output device116may provide a particular type of haptic effect to the user, as discussed in further detail below.

The user input elements114may include one or more of a button114a, a d-pad114b, an analog joystick114c, a trigger114d, a touchscreen114e, and a touch sensitive surface114f, as illustrated inFIG. 2. Each of the user input elements114is configured to allow the user to provide an input command to the host computer140via the processor118, as discussed in further detail below.

The haptic output device116is configured to generate a haptic effect or sensation to the user of the user interface device110. The haptic effects or sensations that are generated by the haptic output device116may be created with any method for creating haptics, such as vibration, deformation, kinesthetic sensations, electrostatic or ultrasonic friction, etc. In an embodiment, the haptic output device116may include an actuator, for example, an electromagnetic actuator such as an Eccentric Rotating Mass (“ERM”) in which an eccentric mass is moved by a motor, a Linear Resonant Actuator (“LRA”) in which a mass attached to a spring is driven back and forth, or a “smart material” such as piezoelectric materials, electro-active polymers or shape memory alloys, a macro fiber composite actuator, an electro-static actuator, an electro-tactile actuator, and/or another type of actuator that provides a physical feedback such as a haptic (e.g., vibrotactile) feedback. The haptic output device116may include non-mechanical or non-vibratory devices such as those that use electrostatic friction (ESF), ultrasonic surface friction (USF), or those that induce acoustic radiation pressure with an ultrasonic haptic transducer, or those that use a haptic substrate and a flexible or deformable surface, or those that provide projected haptic output such as a puff of air using an air jet, or thermal devices that provide temperature effects, and so on. In an embodiment, multiple haptic output devices may be used in the user interface device110to generate different haptic effects.

Returning toFIG. 1, the host computer140includes at least one processor150configured to execute one or more computer program modules, as described below with respect toFIG. 3. The host computer140may be any type of computing device, such as a personal computer or as mentioned above, a gaming console when the system100is a gaming system, or a handheld device, such as a touch screen handheld device, which may be a mobile device. As illustrated inFIG. 1, the host computer140is in communication with the user interface device110though a wired or wireless communication channel170established between a communication port141of the host computer140and a communication port111of the user interface device110.

As illustrated inFIG. 3, the processor150is configured to execute one or more computer program modules. The one or more computer program modules may include one or more of a content provision module152, a haptic effect determination module154, a haptic output device control module156, and/or other modules. The processor150may be configured to execute the modules152,154, and/or156by software, hardware, firmware, some combination of software, hardware, and/or firmware, and/or other mechanisms for configuring processing capabilities on processor150.

It should be appreciated that although modules152,154, and156are illustrated inFIG. 3as being co-located within a single processing unit, in embodiments in which the host computer140includes multiple processors, one or more of modules152,154, and/or156may be located remotely from the other modules. For example, as discussed in further detail below, the processor118of the user interface device110may include one or more of the modules152,154, and/or156. The illustrated embodiment is not intended to be limiting in any way.

The description of the functionality provided by the different modules152,154, and/or156described below is for illustrative purposes, and is not intended to be limiting, as any of the modules152,154, and/or156may provide more or less functionality than is described. For example, one or more of the modules152,154, and/or156may be eliminated, and some or all of its functionality may be provided by other ones of the modules152,154, and/or156. As another example, the processor150may be configured to execute one or more additional modules that may perform some or all of the functionality attributed below to one of the modules152,154, and/or156.

The content provision module152is configured to control the provision of content to the user of the system100via the display160. If the content includes computer generated images (e.g., in a videogame, virtual world, augmented reality virtual world, simulation, etc.), the content provision module152is configured to generate the images and/or views for display to the user through the display160. If the content includes video and/or still images, the content provision module152is configured to access the video and/or still images and to generate views of the video and/or still images for display on the display160. If the content includes audio content, the content provision module152is configured to generate the electronic signals that will drive a speaker, which may be part of the display160, to output corresponding sounds. The content, or information from which the content is derived, may be obtained by the content provision module152from an electronic storage158, which may be part of the processor150, as illustrated inFIG. 3, or may be separate from the processor150.

The haptic effect determination module154is configured to determine the haptic effect or sensation to be generated by the haptic output device116of the user interface device110for the user, based on information received from the content provision module152or the user input element114or any other input that is configured to determine the user's intent. Determining the haptic effect may include determining one or more parameters that include an amplitude, frequency, duration, etc., of the haptic sensation. The haptic effect is determined by the haptic effect determination module156to enhance one or more aspects of the experience provided by the content to the user. For example, the haptic effect may be determined to enhance one or more of the realism of the content, the enjoyability of content, perception of the content by the user, and/or other aspects of the experience provided by the content being conveyed to the user via the display160.

The haptic output device control module158is configured to control the haptic output device116to generate the haptic effect determined by haptic effect determination module156. This includes communicating the haptic output signal to be generated by the processor150to the haptic output device116. The haptic effect to be generated may be communicated over wired communication links, wireless communication links, and/or other communication links between the processor150and the haptic output device116, such as through the wired or wireless communication channel170established between a communication port141of the host computer140and a communication port111of the user interface device110. In an embodiment, at least a portion of the functionality attributed to the haptic output device control module158may be disposed in the processor118carried by the user interface device110and data from the haptic effect determination module156located in the processor150may be communicated to the haptic output device control module158in the processor118through the wired or wireless communication channel170.

The haptic output device116is configured to receive the haptic signal from the haptic output device control module158, and output a haptic effect to the user through the user interface device110. The haptic sensations the user feels may depend on the information being communicated to the user by the host computer140. For example, the haptic sensation may be used to convey game information, such as object interaction, character movement, character status, and/or character type. The haptic sensations can also be used to indicate the current mode of interaction, such as moving a character, or holding a specific weapon. The haptic sensations may also be queried in the sense that the user may explore a surface of the housing112and/or user input elements114to determine or check for specific information.

The surface of the housing112and/or the user input elements114such as the buttons114a, d-pad114b, and joystick114cof the user interface device110may have programmable friction, texture, such as a feel of grooves and bumps, or other haptic sensations. The haptic sensations may be created by electrostatic friction, ultrasonic friction, local surface deformation, electrotactile stimulation, such as electrocutaneous stimulation, thermal effects and/or vibration.

In an embodiment, surface haptic sensations may be in the form of programmable textures or varying friction sensations. For example, the surface of the d-pad114bmay have electrodes120embedded in one or more of the four sections of the d-pad114b. When the user moves his/her finger or thumb over the surface of the d-pad114b, the electrodes120may be charged in a time varying fashion in order to create an electrostatic force on the user's finger to create the sensation of detents, textures or a programmable friction sensation. D-pads are often used to enter directional information into the game. The user typically slides his/her thumb along this type of user input element114, which is particularly suited to provide electrostatic haptic sensations.

According to embodiments of the invention, surface haptic sensations may be created on at least one of the following user input element114surfaces: buttons114a, d-pad114b, analog joystick114c, triggers114d, touchscreen114e, and touch sensitive surfaces114f, and/or on at least one of the surfaces, i.e. front, back, or side surfaces, of the housing112.

In an embodiment, the friction of the user input elements114may be programmably adjusted. For example, buttons114athat are active in a gaming application may have one friction sensation and those that are not used in the gaming application may have another level of friction sensation, so the user may feel the active button through haptic feedback.

In an embodiment, a user may move his/her finger along the back of the housing112in order to query for information, such as boss strength or character status.

In an embodiment, programmable surface detents may be placed on the d-pad114bwhen the user's character is interacting with different surfaces in the game. For example, a different sensation may be felt when the user's character is running through mud as compared to when running on hard ground.

In an embodiment, the top of the analog joystick114cmay have a programmable texture to indicate game environment or interaction mode. For example, a different texture may display when a user is driving a different vehicle or is using a different weapon.

In an embodiment, the triggers114dmay have a different texture or friction depending on weapon type, firing rate, etc. In an embodiment, the user may set the feel of the haptic effect based on the user's preference.

In an embodiment, the user interface device110may be a wand or wand-like motion controller that has programmable textures or friction on the housing112to indicate weapon type or specific ways objects are being interacting with. For example, an ice sword may have a low simulated friction coefficient compared to a regular sword.

In an embodiment, one of the user input elements114described above, such as one of the buttons114a, or another user input element on the user interface device110may be used to adjust the level of friction on, for example, the d-pad114b.

In an embodiment, electronic circuitry may be included in the user interface device110to drive the haptic output device116based on the output of the haptic output device control module. For example, an amplifier may be used to reach the voltages need for certain types of actuators to generate the desired haptic effect.

According to embodiments of the invention, the haptic output device116is coupled directly to, or integrated with, the surface to be augmented with surface haptics, depending on the desired haptic effect.

In embodiments in which at least part of the haptic effect includes electrostatic friction, the outer surface of the housing112or the surface of the input element114providing the surface of the user interface device110to be augmented is made of a conductive material covered with a thin insulator. For example, one of the buttons114amay be made of metal and covered with a thin layer of plastic. The conductive layer is connected to the driving electrical signal provided by the haptic output device116. In this embodiment, the user's body must be connected to a shared ground, either through another surface of the device (e.g., back of the housing112) or by some other method, such as a bracelet worn by the user or a grounded chair on which the user is sitting. In an embodiment, the roles of the ground and surface may be reversed, as is done with reverse electrovibration.

In an embodiment in which at least part of the haptic effect includes ultrasonic friction, the surface to be augmented vibrates at ultrasonic frequencies. Such an effect may be done using actuators, such as piezoelectric actuators. In this embodiment, the surface is typically suspended at vibration nodes so that vibrations are not dampened. For example, a piezoelectric actuator may be attached to a d-pad that is suspended on springs.

In an embodiment in which at least part of the haptic effect includes local surface deformation, deformable actuators, such as macro fiber composite (MFC) actuators may be used at the surface to be actuated to either deform directly or move an intermediate object to deform the surface. For example, a deformable actuator may bend at the surface of a d-pad to create a concave shape. Surface feel may also be adjusted by having small protrusions at the surface.

In an embodiment in which the haptic effect is electrotactile stimulation, electrodes may be placed at the surface that is to produce haptic feedback. For example, the housing112may be covered with a film containing an array of electrodes, or select surfaces of the user input element114, such as the d-pad114b, may be covered with a film containing an array of electrodes120, as described above. In an embodiment, the electrodes may alternatively be embedded in the material of the surface.

In an embodiment in which the haptic effect is a thermal effect, a thermal device, such as a Peltier cooler, may be used at the surface to control its thermal properties, either directly or through an intermediary surface with appropriate thermal conductance.

In an embodiment in which the haptic effect is vibration, similar to ultrasonic friction described above, vibrations are applied to the surface through an actuator connected to the surface. The actuator may be an eccentric rotating mass (ERM) actuator, a linear resonant actuator (LRA), a piezoelectric actuator, or any other vibration actuator.

In an embodiment, the user interface device110may include a touch sensor122at the surface to be augmented that is configured to sense a position of contact with the surface by the user and communicate the position to the processor118so that the haptic effect may be modulated. For example, electrostatic friction is typically applied to an entire surface but can be modulated as a finger moves against it to create the illusion of local features. Such a haptic effect requires sensing of the finger position, and communicating the sensor readings to the haptic output device control module156or the content provision module152in order to adjust the haptic rendering to the surface.

Exemplary implementations of embodiments of the invention in a gaming environment will now be described. In an implementation, the texture of one of the buttons114amay be repetitively felt by the user while waiting for the right timing for an action to take during a game. For example, a gamer could be waiting for a laser gun to recharge or for a cue from the game for the performance of a timing-critical operation. The texture of the button114amay be configured to change gradually to indicate progress, abruptly to indicate a sudden change, or very briefly to provide a cue that can easily be missed.

In an implementation, a game may vary the friction at the surface of a control such as the d-pad114bin order to disturb the player and make the game more difficult. For example, a car that has been damaged may be controlled with a d-pad having unpredictable friction.

Games often involve repetitive gestures such as repeatedly pressing left-right on a d-pad or shaking a gamepad. In an implementation, the texture of the d-pad114bor a surface of the housing112may change to provide feedback about the performance of an action, such as progress being made in the breaking of a door, or some other action requiring a repetitive gesture.

Games often have a tutorial at the beginning in which the controls are gradually explained. In an implementation, surface effects may be used to highlight the buttons114aand other user input elements114that are being shown in the tutorial. For example, a tutorial may explain that a particular button needs to be pressed and add a texture to that button to make it easier to find. This may also be used to help players in “easy” mode, for example, to indicate what button needs to be pressed in certain situations. For example, a haptic effect may be provided to the user input element114that is predicted to be the most likely element to be used next.

In an implementation using the thermal device as the haptic output device116, the surface of the housing may get hot to match events in a game, such as an explosion or shooting a gun.

In an implementation, surface haptics may also be used to indicate the presence of virtual controls such as buttons and sliders, for example on the touch sensitive surface114f, and simulate physical interactions while the virtual controls are used. For example, a game may use different virtual controls on the backside of the housing112, depending on the current context, such as different mini-games with distinct input requirements. Touch sensitive surfaces may also be used for gestural input with haptic textures and effects providing confirmation and additional feedback. This may be particularly relevant if additional sensors are used on the surface to enable spatial tactile effects. In an implementation, the touch sensitive surfaces may also be used for free space gestural input. For example, the surface feel of the device may change according to the position, orientation, and/or velocity of the device. In an implementation, surface haptics may be used to create a virtual button that is felt by the user to be raised or depressed. The stiffness of the virtual button may be modified with surface haptics to make the button seem easier or harder to press.

In an implementation, the haptic state of surface of the housing112or at least one of the user input elements114may communicate ambient information about the current state of the user interface device110. For example, in embodiments in which the user interface device is battery powered, different textures of at least a portion of the surface may indicate different battery levels in a subtle, non-obtrusive manner.

The illustrated and above-described embodiments are not considered to be limiting in any way, and embodiments of the present invention and may be used to enable haptic feedback in various electronic devices, such as touch screen handheld devices (mobile devices, PDA, and navigation systems), automotive applications, etc.

Although many of the examples described herein relate to gaming devices, it should be understood that the present disclosure also encompasses other types of human-computer interfaces involving touch sensitive structures. In addition, other features and advantages will be apparent to one of ordinary skill in the art upon reading and understanding the general principles of the present disclosure. These other features and advantages are intended to be included in the present disclosure as well.

The embodiments described herein represent a number of possible implementations and examples and are not intended to necessarily limit the present disclosure to any specific embodiments. Instead, various modifications can be made to these embodiments as would be understood by one of ordinary skill in the art. Any such modifications are intended to be included within the spirit and scope of the present disclosure and protected by the following claims.