Haptic effect generation system

A haptic effect generation system includes a computing platform including a hardware processor, an analog-to-digital converter (ADC), a digital-to-analog converter (DAC), and a memory storing a haptic software code, as well as a haptic transformer coupled to the computing platform. The haptic transformer receives an input signal, and transforms the input signal to a first audio signal corresponding to the input signal. The ADC converts the first audio signal to a first audio data. The hardware processor executes the haptic software code to receive the first audio data from the ADC, and to generate a second audio data using the first audio data, the second audio data corresponding to a desired haptic effect. The DAC converts the second audio data to a second audio signal. The haptic transformer then transforms the second audio signal to a haptic actuator signal for producing the desired haptic effect.

BACKGROUND

Modern electronic devices are increasingly being designed to engage users via multiple sensory modes. For example, personal communication devices may utilize a combination of visual, auditory, and haptic modes to interact with a user. With respect to the visual and auditory effects produced by modern electronic devices, the conventional art includes a wide variety of tools, application programming interfaces (APIs), and editing software for working with audio-visual content. In addition, there presently exists considerable expertise in producing audio-visual experiences providing educational, therapeutic, social, and entertainment focused interactions. However, the conventional art lacks a comparable richness in hardware technologies, software tools, and technical expertise for the development of haptic based interactions.

SUMMARY

There are provided haptic effect generation systems and methods, substantially as shown in and/or described in connection with at least one of the figures, and as set forth more completely in the claims.

DETAILED DESCRIPTION

The following description contains specific information pertaining to implementations in the present disclosure. One skilled in the art will recognize that the present disclosure may be implemented in a manner different from that specifically discussed herein. The drawings in the present application and their accompanying detailed description are directed to merely exemplary implementations. Unless noted otherwise, like or corresponding elements among the figures may be indicated by like or corresponding reference numerals.

As noted above, modern electronic devices are increasingly being designed to engage users via multiple sensory modes, including visual, auditory, and haptic modes. As also noted above, the conventional art includes a wide variety of tools, application programming interfaces (APIs), and editing software for working with audio-visual content, as well as considerable expertise in producing audio-visual experiences. However, the conventional art lacks a comparable richness in hardware technologies, software tools, and technical expertise for the development of haptic based interactions.

The present application is directed to haptic effect generation systems and methods. The haptic effect generation systems and methods disclosed in the present application enable the creation, editing, storing, sharing, and broadcasting of haptic data files corresponding respectively to a broad range of haptic effects. Such a haptic effect generation system can be implemented through use of a computing platform coupled to a haptic transformer and including a hardware processor for executing a haptic software code. Moreover, the haptic transformer can advantageously be implemented using readily available audio based hardware components. As a result, the haptic effect generation systems and methods disclosed in the present application are advantageously easy to use, simple to adopt, and can be implemented to produce a wide variety of haptic user interactions.

FIG. 1shows a diagram of an exemplary haptic effect generation system for implementing the inventive concepts described above. Haptic effect generation system100includes haptic transformer130coupled to computing platform102, which itself includes hardware processor104, memory106, analog-to-digital converter (ADC)108, and digital-to-analog converter (DAC)110. As shown inFIG. 1, ADC108and DAC110are controlled by hardware processor104. As further shown inFIG. 1, memory106stores haptic software code120, and may include haptic data file126produced by haptic software code120when executed by hardware processor104. Also shown inFIG. 1are one or more input sources140and haptic actuators150.

It is noted that, in some implementations, haptic effect generation system100may include one or more input sources140and/or haptic actuators150. However, in other implementations, haptic effect generation system100may receive input signal142from one or more input sources140external to haptic effect generation system100. Moreover, in some implementations, haptic effect generation system100may send haptic actuator signal134to haptic actuators150external to haptic effect generation system100. It is further noted that although computing platform102is shown as a personal computer (PC) inFIG. 1, that representation is provided merely as an example. In other implementations, computing platform102may be implemented as a network server, or may take the form of another type of personal communication device, such as a smartphone or tablet computer, for example.

According to the exemplary implementation shown inFIG. 1, haptic transformer130of haptic effect generation system100receives input signal142from one or more input sources140, and transforms input signal142to first audio signal132corresponding to input signal142. ADC108of haptic effect generation system100converts first audio signal132to first audio data122. Hardware processor104of computing platform102executes haptic software code120to receive first audio data122from ADC108, and to generate second audio data124using first audio data122.

Second audio data124corresponds to a desired haptic effect, and may be utilized in a process to produce the desired haptic effect, or may be stored in haptic data file126, by hardware processor104, for later use. It is noted that, when stored by hardware processor104in haptic data file126, second audio data124may be subsequently edited, shared, such as through being copied, and/or may be broadcast, such as by being transmitted to another computing platform (other computing platform not shown inFIG. 1). It is further noted that haptic data file126may be an audio data file, and may be stored as an MPEG-1 and/or MPEG-2 Audio Layer III (MP3) file, or as a Waveform Audio File Format (WAV) file, for example.

When used to produce the desired haptic effect, second audio data124is converted to second audio signal112by DAC110. Haptic transformer130receives second audio signal112from DAC110and transforms second audio signal112to haptic actuator signal134, which is produced as an output to haptic actuators150. Haptic actuators150may then instantiate the desired haptic effect based on haptic actuator signal134.

FIG. 2shows a more detailed diagram of haptic transformer230suitable for use in haptic effect generation system100, inFIG. 1, according to one implementation. As shown inFIG. 2, haptic transformer230includes filtering and modulation circuit236, and frequency converter, driver, and amplifier circuit238. As further shown inFIG. 2, haptic transformer230is configured to receive input signal242from one or more input sources140, inFIG. 1, and to transform input signal242to first audio signal232using filtering and modulation circuit236. In addition, haptic transformer230is configured to receive second audio signal212from DAC110, inFIG. 1, and to transform second audio signal212to haptic actuator signal234using frequency converter, driver, and amplifier circuit238.

Haptic transformer230including filtering and modulation circuit236, and frequency converter, driver, and amplifier circuit238, corresponds in general to haptic transformer130, inFIG. 1, and those two corresponding features may share any of the characteristics attributed to either feature in the present application. Moreover, input signal242, first audio signal232, second audio signal212, and haptic actuator signal234, inFIG. 2, correspond in general to input signal142, first audio signal132, second audio signal112, and haptic actuator signal134, inFIG. 1, and may share any of the characteristics attributed to those corresponding features in the present application.

Filtering and modulation circuit236and frequency converter, driver, and amplifier circuit238may each include an analog circuit. For example, each of filtering and modulation circuit236and frequency converter, driver, and amplifier circuit238may be implemented using readily available audio circuit components, such as audio mixers, filters, drivers, and amplifiers, for example. Haptic transformer130/230uses filtering and modulation circuit236to transform input signal142/242, which has a non-zero frequency, to first audio signal132/232having a frequency in the audio band, i.e., up to approximately twenty kilohertz (20 kHz). Haptic transformer130/230uses frequency converter, driver, and amplifier circuit238to transform second audio signal112/212, which may be an up to 20 kHz signal, to lower frequency haptic actuator signal134/234. In some implementations, for example, haptic actuator signal134/234may have a frequency of less than or approximately equal to 300 Hz.

FIG. 3shows a more detailed diagram of input sources340suitable for use with haptic effect generation system100, inFIG. 1, according to one implementation. As shown inFIG. 3, input sources340may include one or more alternating signal sources344, and/or one or more non-alternating signal sources346for producing input signal342. Input sources340including one or more alternating signal sources344, and/or one or more non-alternating signal sources346correspond in general to one or more input sources140, inFIG. 1. Thus, one or more input sources140may share any of the characteristics attributed to corresponding input sources340in the present application. In addition, input signal342corresponds in general to input signal142/242inFIG. 1/2, and may share any of the characteristics attributed to those corresponding features in the present application.

Alternating signal sources344may include one or more of a microphone, accelerometer, and pulse-sensor, for example. Examples of non-alternating signal sources346include stretch sensors, potentiometers, switches and dials, and force or pressure sensors. In implementations in which one or more non-alternating signal sources346are utilized, haptic transformer130/230or input sources140/340may include circuitry for passing an oscillating signal through each non-alternating signal source to produce one or more input signals corresponding to input signal142/242. By contrast, in implementations in which one or more alternating signal sources344are utilized, the alternating signal source or sources344may produce input signal142/242directly.

FIGS. 4A, 4B, and 4Cdepict exemplary haptic effects produced by a haptic effect generation system, according to one implementation. Each ofFIGS. 4A, 4B, and4C shows an exemplary hand460and forearm462of a human user experiencing the haptic effect depicted in the respective figure. In addition, each ofFIGS. 4A, 4B, and 4Cshows haptic actuators450including first haptic actuator451and second haptic actuator452. Haptic actuators450including first and second haptic actuators451and452correspond in general to haptic actuators150, inFIG. 1. Thus, haptic actuators150may share any of the characteristics attributed to corresponding haptic actuators450in the present application.

First and second haptic actuators451and452may take the form of vibratory elements, and may be implemented using one or more of speakers, subwoofers, buzzers, bone conductors, and piezo elements, for example. Use of at least two haptic actuators, such as first and second haptic actuators451and452, advantageously enables generation of haptic stereo effects.

It is noted that althoughFIGS. 4A, 4B, and 4Cdepict the generation of haptic effects through use of two haptic actuators shown as first and second haptic actuators451and452, in other implementations, haptic actuators150/450may include more than two haptic actuators. It is further noted that althoughFIGS. 4A, 4B, and 4Cdepict the generation of haptic effects through contact of first and second haptic actuators451and452with forearm462for conceptual clarity, more generally, haptic actuators150/450may be situated so as to contact both hands, the torso, a leg or legs, or the head of the user experiencing the haptic effect.

InFIG. 4A, the haptic effect produced using first and second haptic actuators451and452is apparent tactile motion sensation454. Apparent tactile motion sensation454can be produced by applying consecutive stimulations at two different locations using first and second haptic actuators451and452. For example, a first stimulation at location 1 using first haptic actuator451is followed by a second stimulation at location 2 using second haptic actuator452. The result of those two consecutive stimulations is apparent tactile motion sensation454, in which the user experiencing the haptic effect perceives an illusory stimulation traveling from location 1 to location 2.

InFIG. 4B, the haptic effect produced using first and second haptic actuators451and452is phantom tactile sensation456. Phantom tactile sensation456can be produced by applying substantially simultaneous stimulations at two different locations using first and second haptic actuators451and452. For example, a first stimulation at location 1A using first haptic actuator451is applied substantially simultaneously with a second stimulation at location 1B using second haptic actuator452. The result of those two substantially simultaneous stimulations is phantom tactile sensation456, in which the user experiencing the haptic effect perceives an illusory stimulation applied at a location between locations 1A and 1B. The intensity and perceived location of phantom tactile sensation456is determined by the intensities of the real stimulations applied using first and second haptic actuators451and452.

InFIG. 4C, the haptic effect produced using first and second haptic actuators451and452is sensory saltation effect458, also known as the “cutaneous rabbit” illusion. Sensory saltation effect458can be produced by applying consecutive sets of stimulations at two different locations using first and second haptic actuators451and452. For example, three brief stimulations at location 1-3 using first haptic actuator451may be followed by three brief stimulations at location 2-3 using second haptic actuator452. The result of those two consecutive sets of stimulations is sensory saltation effect458, in which the user experiencing the haptic effect perceives a sensation of jumping on and between locations 1-3 and 2-3.

The features shown inFIGS. 1, 2, 3, 4A, 4B, and 4Cof the present application will be further described by reference toFIG. 5, which presents flowchart500outlining an exemplary method for generating haptic effects, according to one implementation. With respect to the method outlined inFIG. 5, it is noted that certain details and features have been left out of flowchart500in order not to obscure the discussion of the inventive features in the present application.

Referring toFIG. 5in combination withFIGS. 1, 2, and 3, flowchart500begins with receiving input signal142/242/342(action510). As shown inFIG. 1/2, input signal142/242/342may be received by haptic transformer130/230of haptic effect generation system100from input source or sources140/340. As noted above, in some implementations, haptic effect generation system100may include one or more input sources140/340. As further noted above, in some implementations, one or more input sources140/340may be alternating signal sources capable of producing input signal142/242/342directly.

However, and as also noted above, in some implementations, one or more input sources140/340may be non-alternating signal sources346, such as resistance sensors, for example, incapable of producing input signal142/242/342having a signal frequency other than zero. In those implementations, haptic transformer130/230or input sources140/340may include circuitry for generating an oscillating signal for passing through each of one or more non-alternating signal sources346to produce input signal142/242/342having a non-zero frequency.

Flowchart500continues with transforming input signal142/242/342to first audio signal132/232corresponding to input signal142/242/342(action520). As shown inFIG. 1/2, input signal142/242/342is transformed to first audio signal132/232by haptic transformer130/230, using filtering and modulation circuit236. As discussed above, filtering and modulation circuit236may include an analog circuit. For example, filtering and modulation circuit236may be implemented using readily available, and even off-the-shelf, audio circuit components, such as audio mixers, filters, drivers, and amplifiers, for example. Haptic transformer130/230uses filtering and modulation circuit236to transform input signal142/242having a non-zero frequency to first audio signal132/232having a frequency in the audio band, i.e., a frequency of up to approximately 20 kHz.

Flowchart500continues with converting first audio signal132/232to first audio data122(action530). Conversion of first audio signal132/232to first audio data122may be performed by ADC108of computing platform102, under the control of hardware processor104, for example.

Flowchart500continues with receiving first audio data122from ADC108(action540). First audio data122may be received from ADC108by hardware processor104executing haptic software code120.

Flowchart500continues with generating second audio data124corresponding to a desired haptic effect, using first audio data122(action550). Generation of second audio data124corresponding to a desired haptic effect, using first audio data122, may be performed by hardware processor104executing haptic software code120.

Haptic software code120includes audio processing software for performing audio mixing and audio production. Haptic software code120, when executed by hardware processor104, may generate second audio data124corresponding to the haptic effects illustrated inFIGS. 4A, 4B, and 4C, for example. That is to say, haptic software code120, when executed by hardware processor104, may generate second audio data124corresponding to a desired haptic effect such as apparent tactile motion sensation454, phantom tactile sensation456, or sensory saltation effect458, as well as others.

Flowchart500continues with converting second audio data124to second audio signal112/212(action560). Conversion of second audio data124to second audio signal112/212may be performed by DAC110of computing platform102, under the control of hardware processor104, for example.

Flowchart500continues with transforming second audio signal112/212to haptic actuator signal134/234for producing the desired haptic effect (action570). As shown inFIG. 1/2, second audio signal112/212is transformed to haptic actuator signal134/234by haptic transformer130/230, using frequency converter, driver, and amplifier circuit238. As discussed above, frequency converter, driver, and amplifier circuit238may include an analog circuit. For example, frequency converter, driver, and amplifier circuit238may be implemented using readily available, and even off-the-shelf, audio circuit components, such as audio mixers, filters, drivers, and amplifiers, for example. Haptic transformer130/230uses frequency converter, driver, and amplifier circuit238to transform second audio signal112/212, which may be an up to 20 kHz audio band signal, to lower frequency haptic actuator signal134/234. As noted above, in some implementations, for example, haptic actuator signal134/234may have a frequency of less than or approximately equal to 300 Hz.

Flowchart500can conclude with producing haptic actuator signal134/234as an output (action580). As shown inFIG. 1/2, haptic transformer130/230produces haptic actuator signal134/234as an output for use by haptic actuators150/450to produce the desired haptic effect. In some implementations, as discussed above by reference toFIGS. 4A, 4B, and 4C, the desired haptic effect may be one of apparent tactile motion sensation454, phantom tactile sensation456, and sensory saltation effect458.

The audio processing capabilities provided by haptic software code120, combined with the functionality provided by haptic transformer130/230, advantageously enable the use of high-speed, high-bandwidth audio channels for the generation of desired haptic effects using haptic actuator signal134/234. According to the implementations disclosed in the present application, input signal142/242/342can be received and processed so as to generate haptic actuator signal134/234corresponding to a desired haptic effect in real time with respect to input signal142/242/342received from input sources140/340.

The haptic effect generation systems and methods disclosed in the present application can be implemented in any of a wide variety of use cases in which coherent, real time, haptic feedback complements a user experience. Examples of such use cases include video games, movies, sporting events, and theme park attractions such as rides and virtual reality interactive experiences. In addition, the haptic effect generation systems and methods disclosed herein can be utilized to enable couches, beds, tables, walls, and other architecture and furniture to react to user actions.

Moreover, in some implementations, the present haptic effect generation systems and methods may be employed to provide therapeutic and/or assistive services. For example, haptic feedback can be used to complement relaxation and meditation, monitor and guide breathing, and provide therapeutic massage. In yet another use case, the present haptic effect generation systems and methods may be incorporated into smart vests, smart belts, or headgear configured to provide directional and awareness cues to motor cycle riders, athletes, and construction workers, for example.

Thus, the present application discloses haptic effect generation systems and methods enabling the creation, editing, storing, sharing, and broadcasting of haptic data files corresponding respectively to a broad range of haptic effects. The haptic effect generation systems disclosed in the present application can be implemented through use of a computing platform coupled to a haptic transformer and including a hardware processor for executing a haptic software code. In addition, the haptic transformer can advantageously be implemented using readily available audio based hardware components. As a result, the haptic effect generation systems and methods disclosed in the present application are advantageously easy to use, simple to adopt, and can be implemented to produce a wide variety of haptic user interactions.