VIRTUAL REALITY SYSTEMS WITH SYNCHRONOUS HAPTIC USER FEEDBACK

Methods and systems for providing synchronous haptic feedback to one or more users during a virtual reality experience include a virtual reality device, a gaming computer that generates the virtual reality gameplay to be displayed to a user via the virtual reality device, and a haptic computing device that receives, from the gaming computing device, virtual reality application data indicative of the virtual reality gameplay displayed to the user. The haptic computing device converts the virtual reality application data received from the gaming computing device into haptic output data and sends a control signal including the haptic output data to a controller that in turn transmit an activation signal to one or more haptic devices positioned proximate to user. In response to receipt of the activation signal from the controller, one or more haptic device generates a haptic effect palpable by the user and synchronized with the virtual reality gameplay.

TECHNICAL FIELD

This disclosure relates generally to virtual reality (“VR”) systems and, in particular, to VR systems configured to provide synchronous haptic feedback to one or more users.

BACKGROUND

VR experiences allow users to view and interact with a virtual world as if it were the real world. VR experiences can be used in a variety of applications including, but not limited to, gaming, simulations, presentations, and movies. Conventionally, a user immersed in the VR experience in the visual sense, visually observing a VR gameplay via a virtual reality device (“VRD”), which is typically placed on the head and over the eyes of the user and is coupled to a VR gaming computer.

Virtual characters/avatars controllers by the users within the VR gameplay undergo a variety of actions (e.g., walking (e.g., through a cold or hot environment), running, falling, driving, flying, swimming, etc.)) that translate into in-game haptic sensations attributable to the user-controlled virtual character/avatar. In order to make the VR experience more immersive for the user and as realistic as possible, it is desirable to reproduce, synchronously, and in a proportion complementary to the VR gameplay, one or more of such haptic effects for the user engaged in the VR gameplay via the virtual reality device without significantly increasing the computing power required for the VR gaming computer to render such haptic effects.

DETAILED DESCRIPTION

The following description is not to be taken in a limiting sense, but is made merely for the purpose of describing the general principles of exemplary embodiments. Reference throughout this specification to “one embodiment,” “an embodiment,” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases “in one embodiment,” “in an embodiment,” and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

Generally speaking, this application describes systems and methods for providing, via one or more haptic devices, synchronous haptic feedback to one or more users immersed in a virtual reality experience, thereby providing the one or more users with a more realistic VR experience. It should be noted that the term realistic does not necessarily imply that the virtual world must look like the “real world” (i.e., the world as it currently exists). Rather, the term realistic is used to describe the degree to which the virtual world appears to be real, as opposed to virtual, to the user engaged in the VR gameplay. For example, a clearly fictional environment (e.g., an environment including humans hunting dinosaurs with guns) may not be realistic in the sense that it is not conceivable in the real world. Rather, the fictional environment would be considered realistic by a user in that it visually appears to the user to be real, as opposed to virtual, especially if the user were to noticeably feel, in the real world, palpable haptic effects (e.g., wind, heat, cold, vibration, etc.) corresponding to the VR action the user is immersed in by controlling a virtual character/avatar within the VR gameplay.

Typically, virtual worlds are rendered in real time (i.e., real, or substantially real, time as a user experiences the virtual world). The virtual worlds are rendered in real time because the world must be rendered based on the user's interaction with the virtual world. For example, if the user looks to his or her left, the objects of the virtual world that are positioned to his or her left must be rendered. As another example, if the user is interacting with an object or is engaged in a physical action in the virtual world, the virtual world must be rendered based on this interaction or action. Real time rendering of VR gameplay, even without the additional processing that would be required to analyze VR gameplay and generate suitable haptic effects in the real world, requires very quick rendering and thus significant computing power.

In some embodiments, a system for providing synchronous haptic feedback to at least one user during a VR experience includes: at least one VR device configured to be mounted to a head of the at least one user and to display VR gameplay to the at least one user; at least one gaming computing device operatively coupled to the at least one VRD and configured to generate the VR gameplay to be displayed to the at least one user via the at least one VRD; a haptic computing device including a programmable processor and being in communication with the at least one gaming computing device and configured to receive, from the at least one gaming computing device, VR application data indicative of the VR gameplay displayed to the at least one user. The processor of the haptic computing device is programmed to convert the VR application data received from the at least one gaming computing device into haptic output data. The system further includes at least one controller configured to receive, from the haptic computing device, a control signal including the haptic output data; and at least one haptic device positioned proximate to the at least one user and being operatively coupled to the at least one controller. In response to receipt by the at least one controller of the control signal including the haptic output data from the haptic computing device, the at least one controller is configured to transmit an activation signal to the at least one haptic device. In response to receipt by the at least one haptic device of the activation signal from the at least one controller, the at least one haptic device is configured to generate a haptic effect palpable by the at least one user and synchronized with the VR gameplay.

In other embodiments, a method for providing synchronous haptic feedback to at least one user during a VR experience includes: providing at least one VRD configured to be mounted to a head of the at least one user and to display VR gameplay to the at least one user; providing at least one gaming computing device operatively coupled to the at least one VRD and configured to generate the VR gameplay to be displayed to the at least one user via the at least one VRD; providing a haptic computing device including a programmable processor and being in communication with the at least one gaming computing device and configured to receive, from the at least one gaming computing device, VR application data indicative of the VR gameplay displayed to the at least one user; converting, via the processor of the haptic computing device the VR application data received from the at least one gaming computing device into haptic output data; providing at least one controller configured to receive, from the haptic computing device, a control signal including the haptic output data; providing at least one haptic device positioned proximate to the at least one user and being operatively coupled to the at least one controller; in response to receipt by the at least one controller of the control signal including the haptic output data from the haptic computing device, transmitting, via the at least one controller, an activation signal to the at least one haptic device; and in response to receipt by the at least one haptic device of the activation signal from the at least one controller, generating, via the at least one haptic device, a haptic effect palpable by the at least one user and synchronized with the VR gameplay.

FIG. 1shows one exemplary embodiment of a system100for providing synchronous haptic feedback to a user110by way of two haptic devices120a,120bduring a VR experience. WhileFIG. 1shows only one user110immersed in the VR experience, it will be appreciated that the system100may be configured for multiple (e.g., 2, 3, 4, 5, 10, or more) users. For example, an exemplary system300shown inFIG. 3is configured for two users210a,210bimmersed in the VR experience. In the exemplary embodiment illustrated inFIG. 1, the user110is shown sitting in a chair190, which could be a conventional office-type chair, or a more specialized gaming chair that may be configured to be controlled by the Haptic CPU150(described in more detail below) to tilt (e.g., forward, backward, and side-to-side) synchronously to the movement being experienced within the VR gameplay environment by the user-controlled virtual character/avatar.

While the exemplary chairs and haptic devices shown inFIGS. 1 and 3as separate devices that are spaced from one another, it will be appreciated that, in some embodiments, any of the chairs190,390a,390bmay be gaming chairs that are physically coupled to and/or physically incorporate one or more of the haptic devices referred to herein. For example, in some aspects, a gaming chair usable with any of systems100,200,300may be configured to physically incorporate a fan/air blower, a heater, a cooler, a rumbling/shaking device, or the like to provide multiple different haptic effects to the user110during VR gameplay. It will also be appreciated that the VR systems described herein may be experienced by the user110without using the chair190and while standing upright. Similarly, while the exemplary haptic controllers160a,160b,360a-dand haptic devices170a,170b,370a-dare shown inFIGS. 1 and 3as separate devices that are spaced from one another, it will be appreciated that, in some embodiments, the haptic controllers160a,160b,360a-dmay be physically incorporated and/or directly physically coupled to their respective haptic devices170a,170b,370a-d,as shown, by way of example, inFIG. 2.

In the embodiment shown inFIG. 1, the system100includes a VRD120, which is, in effect, the user interface that visually displays the VR gameplay to the user110. The exemplary VRD120may include a display (e.g. LCD, LED, or the like) for visually displaying the VR gameplay to the user110when the VRD120is worn on the head of the user110and over the eyes of the user110. In addition, in the embodiment illustrated inFIG. 1, the VRD120is operatively coupled (e.g., via communication channel135, which may be wired or wireless) to a pair of VR gaming controllers130a,130b,which, when moved and/or otherwise interacted with by the user110, control the movement of a virtual character or avatar controlled by the user110within the VR gameplay. It will be appreciated that the systems100and300are illustrated with hand-held VR gaming controllers130a,130b,330a,330b,330c,330dby way of example only, and that the systems100and300may be implemented such that the users110,310a,310bexperience the same VR gameplay without the use of hand-held VR gaming controllers130a,130b.In other words, the VR gaming controllers130a,130b,330a,330b,330c,330dare optional in some implementations of the systems and methods described herein.

In some embodiments, the VR gaming controllers130a,130binclude includes one or more sensors capable of detecting movement of the VR gaming controllers130a,130b.For example, the sensors included in the VR gaming controllers130a,130bmay include, but are not limited to, a gyroscope to detect an orientation of each of the VR gaming controllers130a,130b,light sensors to detect movement of the VR gaming controllers130a,130brelative to external marking lights, position sensors to detect the relative position and/or distance of the VR gaming controllers130a,130b,velocity sensors to detect the speed of movement of the VR gaming controllers130a,130bby the user110, or the like. In some embodiments, the VR gaming controllers130a,130bare configured (e.g., by including buttons, switches, pads, etc.) to permit the user110to provide input affecting action of the virtual character/avatar within the VR gameplay being displayed to the user110via the VRD120without physically moving the VR gaming controllers130a,130brelative to each other.

In the embodiment illustrated inFIG. 1, the system100includes a gaming computing device (“Game CPU”)140operatively coupled to the VRD120. While the term “Game CPU” is used, it will be appreciated that the Game CPU may be used to generated VR game play including, but not limited to games, real life activity simulations, various presentations, movies, or the like. Generally, the Game CPU140performs the computing operations necessary to generate the VR gameplay to be displayed to the user110via the VRD120. In some aspects, the movement and/or orientation and/or position and/or velocity data collected by the sensor(s) of the VR gaming controllers130a,130bis transmitted from the VR gaming controllers130a,130b(e.g., via a communication channel145, which may be wired or wireless) to the Game CPU140. The Game CPU140then updates the VR gameplay and renders the objects and/or scenes updated within the VR gameplay based on the movement of the hands of the user110and the associated movement of the VR gaming controllers130a,130b.The updated VR gameplay data is then transmitted from the Game CPU140(e.g., via a communication channel125, which may be wired or wireless) to the VRD120to be observed and interacted with by the user110.

In some embodiments, the Game CPU140renders the VR gameplay on the display of the VRD120by rendering, in real time, or near real time, the graphical objects representing the virtual character/avatar and the background (e.g., landscape, buildings, roads, ground/aerial/sea vehicles, weapons, other virtual characters/avatars, etc.) that the virtual character/avatar controlled by the user110interacts with as part of the VR gameplay. In one aspect, during the VR experience, the Game CPU140renders the VR gameplay and receives data representative of movement and/or relative position/orientation of the VR gaming controllers130a,130bphysically manipulated by the user110, and generates VR gameplay that is synchronized to the movement of the VR gaming controllers130a,130b,and transmits the synchronized VR gameplay data to the VRD120for display to the user110.

In the embodiment illustrated inFIG. 1, the system100includes a haptic computing device (“Haptic CPU”)150that is configured to receive VR application data from the Game CPU140, and to translate (as described in more detail below) the VR gameplay data received from the Game CPU140into a real world haptic effect that is palpable to the user110synchronously to the perceived haptic effect being experienced within the VR gameplay by the virtual character/avatar being controlled by the user110. In particular, as will be discussed in more detail below, when the VR gameplay being generated by the Game CPU140is such that the user-controlled virtual character/avatar is experiencing one or more haptic effects (e.g., wind, heat, cold, pressure, shaking, physical damage, etc.) within the VR gameplay environment, the Haptic CPU150is configured to synchronously generate a corresponding haptic effect via one or more controllers160a,160band haptic devices170a,170b.As such, the user110, while controlling the virtual character/avatar within the VR gameplay environment via the VR gaming controllers130a,130b,can also experience (i.e., physically feel) the real world haptic effect corresponding to the haptic effect being experienced by the user-controller virtual character/avatar, thereby advantageously providing the user110with a more realistic and immersive VR experience.

In the embodiments illustrated inFIGS. 1-3, exemplary systems100,200,300include haptic devices in the form of fans170a,170b,270a-d,370a,370bthat generate the haptic effect of air flow representing wind that is physically felt by the user110while immersed in the VR gameplay being generated by the Game CPU140and displayed to the user110via the VRD120. WhileFIGS. 1 and 3shows systems100and300that deploy two haptic devices such as fans per user, it will be appreciated that, depending on the desired system configuration suitable for a given VR game, the number of haptic devices within a given VR system per user may be increased (e.g., from 2 to 3, from 2 to 4, from 2 to 6, from 2 to 8, or from 2 to 10, or more), or decreased (e.g., from 2 to 1. For example, the exemplary system200shown inFIG. 2deploys four haptic devices270a,270b,270c,270dpositioned around the user110. All fans described herein with reference to systems100,200,300may be configured to tilt and pan to vary the angles and orientation of the fans relative to the user110, thereby providing a wind-like effect palpable to the user110that is more realistically matched to the direction of the wind affecting the virtual character/avatar being controlled by the user110within the VR gameplay.

While the exemplary systems100,200,300illustrated inFIGS. 1-3illustrate the haptic devices as fans configured to produce air flow representative of in-game wind affecting the user-controller virtual character/avatar, it will be appreciated that one or more of the illustrated fans may be replaced with other haptic devices (e.g., heaters, coolers, vibration vests, rumble floors, and scent generators, or the like). For example, in some aspects, while the virtual character/avatar being controlled by the user110is riding an all-terrain vehicle (“ATV”) through a desert (e.g., in a driving game), a system including at least one haptic device in the form of a fan and at least one haptic device in the form of a heater would generate, synchronously to the movement of the virtual character/avatar within the VR gameplay, both air flow representative of the fast in-game wind being experienced by the virtual ATV driver and heat representative of the hot in-game desert temperature being experienced by the virtual ATV driver. In other aspects, while the virtual character/avatar controlled by the user110is skiing down a steep slope of a snowy mountain, a system including at least one haptic device in the form of a fan, a haptic device in the form of a vibration vest and/or a rumbling floor, and at least one haptic device in the form of an air cooler would generate, synchronously to the in-game movement of the virtual character/avatar, air flow representative of the fast in-game wind being experienced by the skier and cold air representative of the cold in-game high altitude temperature being experienced by the skier, and vibration/rumbling representative of a scenario, where the virtual skier loses footing and falls (e.g., crashing into a mesh fence defining the boundaries of the virtual ski course.

In the embodiments illustrated inFIGS. 1-3, the Haptic CPUs150,250,350of systems100,200,300are configured to selectively control (i.e., activate/deactivate) each of the haptic devices170a,170b,270a-d,430a,370b,over communication channels165,265,365(which may be wired or wireless) and the network180,280,380, via controllers160a,160b,260a-d,360a,360bthat are each operatively coupled to a respective one of the haptic devices. In some embodiments, as will be described in more detail below, the Haptic CPU150illustrated inFIG. 1is configured to convert VR application data (received from the Game CPU140and representative of the VR gameplay experienced by the user-controlled virtual character/avatar) into a haptic output data representative of a haptic effect to be selectively generated relative to the user110by the haptic devices170aand/or170b,and to then transmit a control signal including the generated haptic output data via communication channel155(which may be wired or wireless) and over the network180via a data distribution protocol to the controllers160a,160bthat control the haptic devices170a,170b.

In some aspects, the controller (e.g.,160b) that receives a control signal including the haptic output data from the Haptic CPU150over the network180generates and transmits an activation signal to its respective haptic device170b.In some aspects, the activation signal generated and transmitted by the controller160bdoes not cause the haptic device170bto simply generate air flow synchronously with the wind being experienced at a given time by the virtual character/avatar being controlled by the user110within the VR gameplay environment, but causes the haptic device170bto generate air flow proportionally to (i.e., at a speed complementary to) the speed of the wind determined by the Haptic CPU150to correlate to the VR gameplay activity (e.g., running, falling, surfing, driving, skiing, flying, sailing, etc.) that the user-controlled virtual character/avatar is engaged in at that time. In other words, in some aspects, the haptic effect that is generated by the haptic device170bthat received the activation signal from the controller160b(which received a control signal including haptic output data from the Haptic CPU150) is proportional in magnitude to the haptic effects being experienced by a virtual character/avatar controlled by the user110within the VR gameplay.

In the exemplary embodiments illustrated inFIGS. 1-3, the components of the systems100,200,300communicate with one another via a network180. The network180may be a wide-area network (WAN), a local area network (LAN), a personal area network (PAN), a wireless local area network (WLAN), Wi-Fi, Zigbee, Bluetooth (e.g., Bluetooth Low Energy (BLE) network), or any other internet or intranet network, or combinations of such networks. Generally, communication between various electronic devices of systems100,200,300may take place over hard-wired, radio frequency-based, cellular, Wi-Fi, or Bluetooth networked components, or the like. In some embodiments, one or more electronic devices of systems100,200,300may include cloud-based features, such as cloud-based memory storage. In some embodiments, the controllers160a,160bare configured to receive the control signal including the haptic output data from the Haptic CPU150via communication protocols including, but not limited to, Art-Net protocol, DMX (digital multiplex protocol), or the like. By the same token, the controllers160a,160bmay be DMX controllers configured to receive and/or transmit DMX data.

In some embodiments, the system100includes one or more localized Internet-of-Things (IoT) devices and controllers in communication with the Haptic CPU150. As a result, in some embodiments, the localized IoT devices and controllers can perform most, if not all, of the computational load and associated monitoring that would otherwise be performed by the Haptic CPU150, and then later asynchronous uploading of summary data can be performed by a designated one of the IoT devices to the Haptic CPU150, or a server remote to the Haptic CPU150. In this manner, the computational effort of the overall system100may be reduced significantly. For example, whenever a localized monitoring allows remote transmission, secondary utilization of controllers keeps securing data for other IoT devices and permits periodic asynchronous uploading of the summary data to the Haptic CPU150or a server remote to the Haptic CPU150. In addition, in an exemplary embodiment, the periodic asynchronous uploading of summary data may include a key kernel index summary of the data as created under nominal conditions. In an exemplary embodiment, the kernel encodes relatively recently acquired intermittent data (“KRI”). As a result, in an exemplary embodiment, KRI includes a continuously utilized near term source of data, but KRI may be discarded depending upon the degree to which such KRI has any value based on local processing and evaluation of such KRI. In an exemplary embodiment, KRI may not even be utilized in any form if it is determined that KRI is transient and may be considered as signal noise. Furthermore, in an exemplary embodiment, the kernel rejects generic data (“KRG”) by filtering incoming raw data using a stochastic filter that provides a predictive model of one or more future states of the system and can thereby filter out data that is not consistent with the modeled future states which may, for example, reflect generic background data. In an exemplary embodiment, KRG incrementally sequences all future undefined cached kernels of data in order to filter out data that may reflect generic background data. In an exemplary embodiment, KRG incrementally sequences all future undefined cached kernels having encoded asynchronous data in order to filter out data that may reflect generic background data.

With reference toFIG. 4, the exemplary Haptic CPU450is a computer-based device and includes a processor-based control circuit/processor410(for example, a microprocessor or a microcontroller) electrically coupled via a connection415to a memory420and via a connection425to a power supply430. The control circuit410of the Haptic CPU450can comprise a fixed-purpose hard-wired platform or can comprise a partially or wholly programmable platform, such as a microcontroller, an application specification integrated circuit, a field programmable gate array, and so on. These architectural options are well known and understood in the art and require no further description.

The control circuit410of the Haptic CPU450can be configured (for example, by using corresponding programming stored in the memory420as will be well understood by those skilled in the art) to carry out one or more of the steps, actions, and/or functions described herein. In some embodiments, the memory420may be integral to the control circuit410of the Haptic CPU450, or can be physically discrete (in whole or in part) from the control circuit410and is configured non-transitorily store the computer instructions that, when executed by the control circuit410, cause the control circuit410to behave as described herein.

As used herein, this reference to “non-transitorily” will be understood to refer to a non-ephemeral state for the stored contents (and hence excludes when the stored contents merely constitute signals or waves) rather than volatility of the storage media itself and hence includes both non-volatile memory (such as read-only memory (ROM)) as well as volatile memory (such as an erasable programmable read-only memory (EPROM)). Accordingly, the memory420and/or the control circuit410of the Haptic CPU450may be referred to as a non-transitory medium or non-transitory computer readable medium. The control circuit410of the Haptic CPU450is also electrically coupled via a connection435to an input/output440that can, for example, receive (e.g., via the wireless or wired communication channel145ofFIG. 1) VR application data from the Game CPU140, as well as to send (e.g., via the wireless or wired communication channel155ofFIG. 1) control signals including haptic output data to the controllers160a,160b.

In the embodiment shown inFIG. 4, the processor-based control circuit410of the Haptic CPU450is electrically coupled via a connection445to a user interface455, which may include a visual display or display screen460(e.g., LED screen) and/or button inputs470that provide the user interface455with the ability to permit an operator (e.g., VR game master) to manually control the Haptic CPU450by inputting commands, for example, via touch-screen and/or button operation or voice commands. In some aspects, the display screen460permits the operator to see various menus, options, and/or alerts displayed by the Haptic CPU450. The user interface455of the Haptic CPU450may also include a speaker480that may provide audible feedback (e.g., alerts) to the operator.

In some embodiments, as described above, the Game CPU140is configured to derive/generate VR application data based on the movement/actions/interactions of the virtual character/avatar (which may affect any of the senses (e.g., physical touch, temperature, pressure, vibrations, etc.) of the virtual character/avatar) controlled by the user110(e.g., via the VR gaming controllers130a,130b) within the VR gameplay environment. In other embodiments, the Game CPU140is configured to derive/generate VR application data based on preprogrammed/pre-stored hand-tailored (i.e., custom) values associated with the movement/actions/interactions of the virtual character/avatar controlled by the user110within the VR gameplay environment. In some aspects, the Game CPU140is configured to transmit (e.g., via the communication channel145and over the network180) this VR application data to the Haptic CPU150.

In some embodiments, the control circuit410of the Haptic CPU450ofFIG. 2is configured (e.g., via the input/output440) to receive, from one or more gaming CPUs (e.g.,140or340aand340b) and over the network180, VR application data indicative of the VR gameplay displayed via one or more VRDs (e.g.,120or320a,320b) to one (e.g.,110) or more (e.g.,310a,310b) users. In certain aspects, the processor410is programmed to convert the VR application data received from the Game CPUs into haptic output data. In some embodiments, the haptic output data is based on a correlation, by the processor410of the Haptic CPU450, of the speed of movement of the user-controlled virtual character/avatar and/or the speed of one or more environmental objects within the VR gameplay, to a projected/predicted magnitude (i.e., speed) and direction of in-game wind that would affect the virtual character/avatar within the VR gameplay. In one embodiment, the processor410of the Haptic CPU450is programmed to extract, from the VR application data received from the Game CPU140, first haptic data indicative of haptic effects being experienced by a virtual character/avatar within the VR gameplay, and to incorporate, into the haptic output data, second haptic data that is proportional in magnitude to the haptic effects being experienced by the virtual character/avatar within the VR gameplay.

In certain multi-player implementations akin to the exemplary system300shown inFIG. 3, the Haptic CPU350is configured to receive multiple streams of VR application data from two (or more) Game CPUs340a,340bover the network180and to detect the VR application data (if any) that is indicative of VR gameplay in which the virtual character/avatar is presently experiencing haptic effects (e.g., wind, high heat, freezing cold, physical damage, etc.). In certain aspects, the generated haptic output data is based on a correlation, by the processor410of the Haptic CPU450, of the speed of movement of the user-controlled virtual character/avatar and/or the speed of one or more environmental or other objects within the VR gameplay, to a calculated magnitude (i.e., speed) and direction of in-game wind that would affect the virtual character/avatar within the VR gameplay.

In other words, when a first stream of VR application data received from Game CPU340aover the network180is indicative, for example, of VR gameplay, where the virtual character/avatar is riding a skateboard down a ramp, and a second stream of VR application data received from Game CPU340bis indicative, for example, of VR gameplay, where the virtual character/avatar is driving a racecar along a racetrack, the correlation by the processor of the Haptic CPU350of the received VR application data to the calculated wind speed would result in haptic output data that will cause significantly faster movement of the blades of the fan370bassociated with the Game CPU340bas compared to the movement of the blades of the fan370aassociated with the Game CPU340. In some aspects, the haptic output data generated by the Haptic CPU350to be transmitted (via the controller360cand/or360d) to haptic device370cand/or370dassociated with the user310bwill include second haptic data that is proportional to the speed of a racecar and will cause significantly faster movement of the blades of the fan370b,while the haptic output data generated by the Haptic CPU350to be transmitted (via the controller360aand/or360b) to haptic device370aand/or370bassociated with the user310awill include second haptic data that is proportional to the speed of a skateboard and will cause significantly slower movement of the blades of the fan370a.

In certain multiplayer implementations akin to the exemplary system300shown inFIG. 3, VR gameplay being experienced by the virtual characters/avatars controlled by the users310a,310bmay be different in that, at a given time, the virtual character/avatar controlled by user310amay be experiencing an in-game haptic effect such as wind or heat or cold (e.g., due to the virtual character/avatar driving a car or being in adverse in-game weather conditions), while the virtual character/avatar controlled by user310bmay not be experiencing an in-game haptic effect such as wind or heat or cold (e.g., due to user hiding within a house by standing still in basement). In such multi-player situations where some user-controlled virtual characters/avatars are experiencing in-game haptic effects and some are not, the processor of the Haptic CPU350is programmed to selectively control (e.g., activate/not activate/deactivate) the haptic devices370a,370b,370c,370dof the system300such that one (or two or three) of the haptic devices (e.g.,370b) is activated by the Haptic CPU350via the controller360aand the remaining haptic devices (e.g.,370aand/or370cand/or370d) are not.

In some aspects, if the processor of the Haptic CPU350detects, in the VR application data stream received from the Game CPU340a(but not in the VR application data stream received from the Game CPU340b), first haptic data indicative of the virtual character/avatar experiencing haptic effects, the processor of the Haptic CPU350is programmed to selectively convert the VR application data received from Game CPU340ainto haptic output data that will ultimately cause the haptic device370bto be activated via the controller360bto produce wind-like air flow without converting the VR application data received from Game CPU340binto haptic output data and allowing the haptic devices370a,370c,and370dto remain inactivated (or to be deactivated via their respective controllers). As discussed above, the haptic output data generated by the Haptic CPU350to be transmitted (via the controller360b) to haptic device370bassociated with the user310awill include second haptic data that is proportional to the speed of a racecar and will cause faster movement of the blades of the fan370bwhen the virtual car is moving forward faster within the VR gameplay and slower movement of the blades of the fan370bwhen the virtual car is moving forward slower within the VR gameplay.

With reference toFIGS. 3 and 4, in some aspects, after generating the haptic output data based on an correlation analysis and conversion of the VR application data into haptic output data, the processor410of the Haptic CPU450is programmed to generate and transmit (e.g., in DMX format as discussed above) a control signal including the generated haptic output data (which may, as discussed above, include second haptic data indicating magnitude of the haptic effect to be produced in proportion to the in-game haptic effects) over the network180to a selected one or more of the controllers360a,360b,360c,360dthat control their respective haptic devices370a,370b.In the racecar example above, the controller360bthat receives a control signal including the haptic output data (which may include the second haptic data) from the Haptic CPU350, generates and transmits an activation signal to its respective haptic device370b.As mentioned above, the activation signal transmitted by the controller360bmay turn on the haptic device370bfrom an OFF state and causes the haptic device370bto synchronously generate air flow at a speed complementary (i.e., fast speed) to the speed of the wind determined by the Haptic CPU350to correspond to the VR gameplay activity (i.e., driving a race car) that the virtual character/avatar being controlled by the user310ais engaged in at the time.

With reference toFIGS. 1-5, one method500of operation of the system100will now be described. For exemplary purposes, the method500depicted by way of a block diagram inFIG. 5is described in the context of the systems100and300ofFIGS. 1 and 3, but it is understood that embodiments of the method500may be implemented in other VR systems.

The exemplary method500for providing synchronous haptic feedback to one or more users during a VR experience includes providing one or more virtual reality devices (e.g.,120) configured to be mounted to a head of one or more users (e.g.,110) and to display VR gameplay to one or more users (block510). In addition to providing one or more virtual reality devices (120,320a,and/or320b) that are configured to display the VR gameplay to one or more users (110,310a,and/or310b), the method500further includes providing one or more gaming computing devices (140,340a,and/or340b) operatively coupled to the virtual reality devices (120,320a,and/or320b) and configured to generate the VR gameplay to be displayed to the user(s) (110,310a,and/or310b) via the VRD(s) (120,320a,and/or320b) (block520).

In the embodiment illustrated inFIG. 5, the method500further includes providing a haptic computing device (150,350) including a programmable control circuit/processor (410) and being in communication with one or more gaming computing devices (140,340a,and/or340b) and configured to receive, from the gaming computing device(s) (140,340a,and/or340b), VR application data indicative of the VR gameplay displayed to the user(s) (110,310a,and/or310b) (block530). The method500illustrated inFIG. 5additionally includes converting, via the processor (410) of the haptic computing device (150,350), the VR application data received from the gaming computing device(s) (140,340a,and/or340b) into haptic output data (block540).

The method500further includes providing one or more haptic device controllers (160,360a,360b,360c,and/or360d) configured to receive, from the haptic computing device (150,350), a control signal including the haptic output data (block550), and well as providing one or more haptic devices (170,370a,370b,370c,and/or370d) positioned proximate to the user(s) (110,310a,310b) and being operatively coupled to haptic device controller(s) (160,360a,360b,360c,and/or360d) (block560). During operation of the method500illustrated inFIG. 5, in response to receipt by one or more haptic device controller(s) (160,360a,360b,360c,and/or360d) of the control signal including the haptic output data from the haptic computing device (150,350), transmitting, via one or more haptic device controller(s) (160,360a,360b,360c,and/or360d), an activation signal to one or more haptic device(s) (170,370a,370b,370c,and/or370d) (block570). In addition, in response to receipt of the activation signal including the haptic output data from a haptic device controller (160,360a,360b,360c,and/or360d), generating, via one or more respective haptic device(s) (170,370a,370b,370c,and/or370d), a haptic effect palpable by the user(s) (110,310a,310b) and synchronized with the VR gameplay (block580).

As mentioned above, in some aspects, the method500further includes providing one or more gaming chairs (190,390a,and/or390b) configured to support the user(s) (110,310a,310b) during the VR experience and transmitting a signal via the haptic computing device (150,350) in order to cause the gaming chair(s) (190,390a,and/or390b) to tilt synchronously with the VR gameplay. In some aspects, the method500includes extracting, via the processor of the Haptic CPU150and from the VR application data received from the gaming computing device140, first haptic data indicative of haptic effects being experienced by a virtual character within the VR gameplay and incorporating, via the processor of the Haptic CPU150, into the haptic output data, second haptic data that is proportional in magnitude to the haptic effects being experienced by the virtual character within the VR gameplay.

In some aspects, the method500may further include providing one or more hand-held VR gaming controllers (130a,130b,330a,330b,330c,330d) in communication with the gaming computing device (140,340a,340b) and controlling, via the gaming computing device (140,340a,340b), movement of a virtual character within the VR gameplay in response to manipulation of the VR gaming controller(s) (130a,130b,330a,330b,330c,330d) by the user(s) (110,310a,310b). As mentioned above, the VR gaming controllers (130a,130b,330a-d) are optional in certain embodiments. In addition, as mentioned above, in some multi-player implementations, the method500may include selectively activating any one of the haptic devices (370a-d) associated with any one of the users (310aor310b) via the processor of the haptic computing device350and based on an analysis by the processor of the haptic computing device350of multiple streams of the VR application data received from the gaming computing devices (140,340a,340b).

The systems and methods described herein provide for realistic generation of haptic effects palpable by the user in a synchronous and proportional fashion relative to the haptic effects affecting the virtual character/avatar being controlled by the user within the VR gameplay. The methods and systems described herein employ both a VR gaming computer that generates VR application data representative of the VR gameplay viewable by the user via a head-mounted VRD, and a dedicated haptic computer that processes the operations required to convert the VR application data generated by the VR gaming computer into haptic output data, which is then transmitted by the haptic computer to one or more selected haptic device controllers, which then activate, one or more selected devices to generate, the haptic computer-determined haptic effects synchronously to the VR gameplay and at levels of magnitude proportional to the action within the VR gameplay. Accordingly, the systems and methods described herein advantageously reproduce, synchronously, and in a proportion complementary to the VR gameplay, one or more haptic effects for VR users without significantly increasing the computing power required for the VR gaming computer to render such haptic effects.