Patent Description:
The present invention generally relates to presenting content and, in particular, to improving content presentation.

Some technologies require a large amount of data to be transmitted with very low latency. For example, virtual reality (VR) streaming typically includes streaming <NUM>-degree visual content and/or other multimedia content to a head-mounted display (HMD). To provide high data throughput and low latency for virtual reality, some virtual reality systems use a High-Definition Multimedia Interface (HDMI) cable physically connected to a user's head-mounted display. Unfortunately, while users are immersed in the virtual reality presented by such cable-based systems, there is a risk that the cable may contact the users, disrupting the users' immersive experience, or may trip the users, causing the users to fall and hurt themselves.

Millimeter wave (mmW) wireless communication -- which uses a wireless signal that has a wavelength (λ) is <NUM> to <NUM> in a range of frequency spectrum from <NUM> to <NUM> -- may advantageously provide high data throughput with low latency, making it a potential solution for virtual reality streaming and/or other technologies. Advantageously, by using a millimeter wave wireless signal, a virtual reality system does not require an obstructive HDMI cable, or the like, connected to a user's head-mounted display.

Unfortunately, when line of sight between a millimeter wave transmitter and a millimeter wave receiver is obstructed, the millimeter wave wireless signal's performance degrades. This performance degradation may cause the virtual reality content presented by a user's head-mounted display to pause and/or skip ahead, disrupting the user's immersion in the virtual reality and potentially ruining the overall user experience.

<CIT>relates to a system and method for delivering <NUM> degree immersive Video streaming to an electronic device and for allowing a user of the electronic device to change viewing directions when viewing 3D data/information. <CIT> relates to a selective adaptive bandwidth method to enable transmission of three dimensional <NUM> degree virtual reality content by slicing the content and utilizing different resolutions of the content. <CIT> relates to Quality of Experience within a context-aware computing environment. <CIT>, relates to predictive RF beamforming for transmission of data to head mounted displays.

A need therefore exists for systems and methods that eliminate or reduce the disadvantages and problems listed above and/or other disadvantages and problems.

In the following, each of the described methods, apparatuses, systems, examples and aspects which does not correspond to the invention as defined in the claims is present for illustration purposes or to highlight specific aspects or features of the claims. One aspect is a method for improving content presentation. The method comprises, by a processor, predicting a state of a presentation environment to be induced by one or more stimuli in content to be presented to one or more users in the presentation environment. The method also comprises predicting, based on the state of the presentation environment predicted to be induced by the one or more stimuli in the content to be presented, a state of at least one communication link of one or more communication links. The method also comprises performing an action in response to the state of the at least one communication link, which was predicted based on the state of the presentation environment predicted to be induced by the one or more stimuli in the content, meeting one or more criteria.

Another aspect is a system for improving content presentation. The system comprises a processor configured with processor-executable instructions to perform operations that comprise predicting a state of a presentation environment to be induced by one or more stimuli in content to be presented to one or more users in the presentation environment. The operations also comprise predicting, based on the state of the presentation environment predicted to be induced by the one or more stimuli in the content to be presented, a state of at least one communication link of one or more communication links.

The operations also comprise performing an action in response to the state of the at least one communication link, which was predicted based on the state of the presentation environment predicted to be induced by the one or more stimuli in the content, meeting one or more criteria.

Yet another aspect is an apparatus for improving content presentation. The apparatus may comprise means for predicting a state of a presentation environment to be induced by one or more stimuli in content to be presented to one or more users in the presentation environment. The apparatus may comprise means for predicting, based on the state of the presentation environment predicted to be induced by the one or more stimuli in the content to be presented, a state of at least one communication link of one or more communication links. The apparatus may comprise means for performing an action in response to the state of the at least one communication link, which was predicted based on the state of the presentation environment predicted to be induced by the one or more stimuli in the content, meeting one or more criteria.

Still another aspect is a non-transitory processor-readable storage medium having stored thereon processor-executable instructions configured to cause a processor of a server to perform operations that comprise predicting a state of a presentation environment to be induced by one or more stimuli in content to be presented to one or more users in the presentation environment. The operations also comprise predicting, based on the state of the presentation environment predicted to be induced by the one or more stimuli in the content to be presented, a state of at least one communication link of one or more communication links. The operations also comprise performing an action in response to the state of the at least one communication link, which was predicted based on the state of the presentation environment predicted to be induced by the one or more stimuli in the content, meeting one or more criteria.

For purposes of summarizing, some aspects, advantages and features of a few of the embodiments of the invention have been described in this summary. Some embodiments of the invention may include some or all of these summarized aspects, advantages and features. However, not necessarily all of (or any of) these summarized aspects, advantages or features will be embodied in any particular embodiment of the invention. Thus, none of these summarized aspects, advantages and features are essential. Some of these summarized aspects, advantages and features and other aspects, advantages and features may become more fully apparent from the following detailed description and the appended claims.

The appended drawings contain figures of preferred embodiments to further clarify the above and other aspects, advantages and features. It will be appreciated that these drawings depict only preferred embodiments of the invention and are not intended to limit its scope. These preferred embodiments will be described and explained with additional specificity and detail through the use of the accompanying drawings in which:.

The present disclosure is generally directed towards a virtual reality system. The principles of the present disclosure, however, are not limited to virtual reality systems. It will be understood that, in light of the present disclosure, the embodiments disclosed herein can be successfully used in connection with other types of content presentation systems, such as, augmented reality systems. A detailed description of the virtual reality system now follows.

<FIG> is a diagram of an embodiment of a content presentation system, such as, a virtual reality system <NUM>. The virtual reality system <NUM> may be configured to present content <NUM> to one or more users <NUM> in a presentation environment <NUM>. The content <NUM> may include visual content <NUM>, audio content <NUM>, haptic feedback content <NUM>, other content, or any combination thereof. The visual content <NUM> may include, for example, video, rendered graphics, other visual content, or any combination thereof.

The presentation environment <NUM> may include one or more users <NUM>, one or more content presentation devices <NUM>, one or more transmitters <NUM>, one or more access points <NUM>, one or more routers <NUM>, one or more repeaters <NUM>, one or more reflectors <NUM>, one or more other objects (e.g., a user's furniture, obstacles in a virtual reality facility, and/or other objects), or any combination thereof.

<FIG> is a diagram of an embodiment of a content presentation device, such as, a content presentation device <NUM>. The content presentation device <NUM> may be configured to present the content <NUM> to a user <NUM>.

As shown in <FIG>, the content presentation device <NUM> may include one or more output devices <NUM>, which may include one or displays <NUM> configured to present the visual content <NUM>, one or more speakers <NUM> configured to present the audio content <NUM>, one or more haptic devices <NUM> configured to present the haptic feedback content <NUM>, one or more other devices configured to present other content, or any combination thereof. Speakers <NUM> may include, or form part of, one or more earpieces.

The content presentation device <NUM> may include one or more receivers <NUM> configured to receive the content <NUM> through one or more communication links <NUM> in <FIG>. Receiver <NUM> may, for example, include one or more antennas <NUM> that may receive content <NUM> through a wireless communication link, such as a millimeter wave wireless communication link or other type of wireless communication link. It will be appreciated that the communication links <NUM> may comprise one or more wireless communication links, one or more wired communication links, or both, depending upon the configuration of the virtual reality system <NUM>. The content presentation device <NUM> may also include one or more processors <NUM>.

<FIG> is a diagram of an embodiment of a content presentation device. As shown in <FIG>, the content presentation device <NUM> may comprise a mobile phone <NUM>, which may include one or more displays <NUM>, one or more speakers <NUM>, one or more haptic devices <NUM>, one or more receivers <NUM>, one or more processors <NUM>, other devices, or any combination thereof.

<FIG> is a diagram of an embodiment of a content presentation device. As shown in <FIG>, the content presentation device <NUM> may comprise a head-mounted display <NUM>, which may include one or more displays <NUM>, one or more speakers <NUM>, one or more haptic devices <NUM>, one or more receivers <NUM>, one or more processors <NUM>, other devices, or any combination thereof.

In some embodiments, the head-mounted display <NUM> may comprise a mobile phone that may be selectively connected to and disconnected from the other components of the head-mounted display. For example, the head-mounted display <NUM> may include a receptacle or other type of receiving portion sized and configured to receive the mobile phone, which may allow the mobile phone to be mounted to the other components of the head-mounted display. With the mobile phone mounted to the other components of the head-mounted display <NUM>, the mobile phone, the other components of the head-mounted display, or both may provide one or more display devices <NUM>, one or more speakers <NUM>, one or more haptic devices <NUM>, or any combination thereof. The head-mounted display <NUM>, however, need not include a mobile phone and may comprise components that are configured to remain interconnected. In some embodiments, the head-mounted display <NUM> may comprise eyewear, a near-eye display, or any other suitable head-mounted display. For example, the head-mounted display <NUM> may comprise eyewear with see-through lenses that may be used to provide an augmented reality experience with which the user <NUM> can see both portions of the content <NUM> and portions of the user's physical environment.

As shown in <FIG>, a transmitter <NUM> may be configured to provide content <NUM> to a content presentation device <NUM> through a communication link <NUM>. In particular, the receiver <NUM> of the content presentation device <NUM> may receive the content <NUM> provided via the communication link <NUM>, and the content presentation device may present the received content to a user <NUM>. For example, in some embodiments, a communication link <NUM> may be a wireless communication link (such as a millimeter wave wireless communication link or other wireless communication link), and the transmitter <NUM> may provide (and the content presentation device <NUM> may receive) the content <NUM> via the wireless communication link.

A transmitter <NUM> may be embodied in any of a variety of suitable components, as shown in <FIG>. For example, an access point <NUM> may comprise a transmitter <NUM>, a router <NUM> may comprise a transmitter <NUM>, and a repeater <NUM> may comprise a transmitter <NUM>. A repeater <NUM> may, for example, receive a wireless signal providing a communication link <NUM> (e.g., from an access point <NUM> or from a router <NUM>), and the repeater may retransmit the received wireless signal.

A transmitter <NUM> may be configured to provide interactive data through a communication link <NUM>. In particular, as shown in <FIG>, a receiver <NUM> (e.g., of the access point <NUM>, the router <NUM>, the repeater <NUM> or any combination thereof) may receive interactive data <NUM> provided via the communication link <NUM>, which the virtual reality system <NUM> may use to generate the content <NUM>. For example, a content presentation device <NUM> may include a transmitter <NUM> (as shown in <FIG>), and the transmitter of the content presentation device may provide the interactive data <NUM> via the communication link <NUM> (as shown in <FIG>).

The interactive data <NUM> may include any of a variety of data, which the virtual reality system <NUM> may use to generate the content <NUM> and/or to perform other functions. In some embodiments, the interactive data <NUM> may indicate <NUM> degrees of freedom (6DoF) controller input. In some instances, the content presentation device <NUM> may generate the interactive data <NUM> indicating the 6DoF controller input, e.g., when the user <NUM> moves the content presentation device. In some instances, the content presentation device <NUM> may receive the interactive data <NUM> indicating the 6DoF controller input from other devices, e.g., from a controller held or worn by the user <NUM>, other controllers, a video camera and/or microphone held or worn by the user, or any other suitable device. In some embodiments, the interactive data <NUM> may include data indicating hand tracking, data indicating environmental depth mapping, recorded audio, recorded video, or any other suitable interactive data.

One or more components of the virtual reality system <NUM> may include a transceiver. For example, as shown in <FIG>, a content presentation device <NUM> may include a transceiver <NUM>, which may include a transmitter <NUM> and a receiver <NUM>. Also, for example, the access point <NUM>, the router <NUM>, and/or the repeater <NUM> may include a transceiver, which may include a transmitter <NUM> (as shown in <FIG>) and a receiver <NUM> (as shown in <FIG>).

As shown in <FIG>, the presentation environment <NUM> may include a reflector <NUM> that may be configured to reflect a wireless signal providing a communication link <NUM> (e.g., from an access point <NUM> or from a router <NUM> or from a content presentation device <NUM>). For example, the wireless signal providing a communication link <NUM> may be travelling in a first direction, and the reflector <NUM> may reflect the wireless signal in a different second direction.

<FIG> is a diagram of an embodiment of a content presentation system, such as, the virtual reality system <NUM>. As shown in <FIG>, in some embodiments, the virtual reality system <NUM> may be a peer-to-peer virtual reality system that comprises a plurality of content presentation devices <NUM>. For example, the peer-to-peer virtual reality system may include a first content presentation device 114a and a second content presentation device 114b. The first content presentation device 114a may include a first transmitter 116a, which may provide interactive data <NUM> and/or at least a portion of the content <NUM> to the second content presentation device 114b through a first communication link 126a. The second content presentation device 114b may include a second transmitter 116b, which may provide interactive data <NUM> and/or at least a portion of the content <NUM> to the first content presentation device 114a through a second communication link 126b. It will be appreciated that the peer-to-peer virtual reality system may include any number of content presentation devices <NUM> that may include one or more transmitters <NUM> configured to provide interactive data <NUM> and/or at least a portion of the content <NUM> to some or all of the other content presentation devices <NUM> via one or more communication links <NUM>.

To help provide a user <NUM> with an immersive virtual reality experience, a content presentation device <NUM> may present content <NUM> that may include one or more stimuli. As discussed below, the one or more stimuli in the presented content <NUM> may cause one or more users <NUM> to move, which may change the presentation environment <NUM>, and the change in the presentation environment may change a state of one or more communication links <NUM>.

The virtual reality system <NUM> and the content presentation devices <NUM> may be configured to allow one or more users <NUM> to move in the presentation environment <NUM>. For example, the users <NUM> may move in response to one or more stimuli in the content <NUM> presented by the content presentation devices <NUM>. The users <NUM> may, for instance, turn their heads, move their arms, move their legs, or make any other movement; and the users may make such movements while remaining at a location or while moving from a first location to a different second location. In some embodiments, the virtual reality system <NUM> and the content presentation devices <NUM> may be configured to provide a <NUM> degrees of freedom (6DoF) virtual reality experience, that is, in the virtual reality experience, the users <NUM> may move forward/backward, up/down, left/right (translation in three perpendicular axes) combined with rotation about three perpendicular axes, often termed pitch, yaw, and roll.

In addition to the users <NUM> themselves, one or more objects in the presentation environment (such as a head-mounted display <NUM> or other objects) may move, for example, in response to user movement. In one such embodiment, a head-mounted display <NUM> may move when the user <NUM> wearing the head-mounted display turns their head, moves their arm, moves their leg, or makes any other movement. In another such embodiment, the user <NUM> may unintentionally contact an object or may intentionally contact an object (e.g., push, pull, lift or a carry the object), which may cause the object to move. In yet another such embodiment, the object may be a robot that may move in response to user movement (e.g., a user gesture) per an algorithm designed to enhance the virtual reality experience provided by the virtual reality system <NUM>.

This movement of the users <NUM> and/or the objects changes the presentation environment <NUM>, and such changes in the presentation environment <NUM> may degrade the performance of the communication link <NUM>. For example, where the change in the presentation environment <NUM> creates an obstruction of a line of sight between a transmitter <NUM> and a receiver <NUM> of a content presentation device <NUM>, the performance of the communication link <NUM> may degrade, especially where the communication link is a millimeter wave wireless communication link. In some instances, such obstructions to the line of sight may include the users <NUM> themselves (e.g., a user <NUM> makes a gesture in which the user's arm obstructs the line of sight to a receiver <NUM> of the user's content presentation device <NUM>; a first user <NUM> moves to a location or makes a gesture that obstructs the line of sight to a receiver <NUM> of a content presentation device <NUM> of a second user <NUM>). In some instances, such obstructions to the line of sight may include objects in the presentation environment <NUM> (e.g., a user <NUM> moves to a location behind an object that obstructs the line of sight to a receiver <NUM> of the user's content presentation device <NUM>; a user wearing a head-mounted display <NUM> turns their head causing the head-mounted display itself to block the line of sight to a receiver <NUM> of the head-mounted display; walls, doors frames or other objects obstruct the line of sight to a receiver <NUM> of the user's content presentation device <NUM> as the user moves between rooms in the presentation environment).

This performance degradation may cause the content <NUM> presented by the content presentation devices <NUM> to pause and/or skip ahead, disrupting the user's immersion in the virtual reality and potentially ruining the overall user experience. This performance degradation may also prevent the virtual reality system <NUM> from promptly receiving the interactive data <NUM>, which may cause the virtual reality system to present content <NUM> that is out of sync with the user's expectations. Such out-of-sync content <NUM> may also disrupt the user's immersion in the virtual reality and potentially ruin the overall user experience. As discussed below, embodiments disclosed herein may advantageously help avoid or reduce this performance degradation, thus improving content presentation.

<FIG> is a flowchart illustrating an embodiment method <NUM> for improving content presentation, which may be performed by one or more processors. As shown in <FIG>, the method <NUM> may include one or more blocks, such as blocks <NUM>, <NUM>, <NUM>, and <NUM>.

At block <NUM>, the processor predicts a state of a presentation environment to be induced by one or more stimuli in content to be presented to one or more users in the presentation environment. For example, the processor may predict a state of the presentation environment <NUM> to be induced by one or more stimuli in the content <NUM> to be presented to one or more users <NUM> in the presentation environment <NUM>. The one or more stimuli in the content to be presented may include one or more audio stimuli in the content, one or more visual stimuli in the content, one or more haptic feedback stimuli in the content, other stimuli, or any combination thereof. The state of the presentation environment, which is predicted at block <NUM>, may comprise a location and/or orientation of one or more users <NUM> in the presentation environment <NUM>, a location and/or orientation of one or more body parts of the one or more users <NUM>, a location and/or orientation of one or more objects in the presentation environment <NUM> (e.g., an antenna of a transmitter <NUM>, an antenna <NUM> of a receiver <NUM>, a user's furniture, obstacles in a virtual reality facility or other objects), a condition of one or more objects in the presentation environment <NUM> (e.g., a power level and/or other attributes of a transmitter <NUM>), other information reflecting the state of the presentation environment <NUM>, or any combination thereof.

At block <NUM>, the processor predicts a state of at least one communication link of one or more communication links. The at least one communication link is configured to provide at least a portion of the content <NUM> after the one or more stimuli are presented. In some embodiments, the at least one communication link is configured to provide interactive data <NUM> after the one or more stimuli are presented.

In particular, at block <NUM>, the processor predicts, based on the state of the presentation environment predicted (at block <NUM>) to be induced by the one or more stimuli in the content to be presented, a state of at least one communication link of one or more communication links. For example, at block <NUM>, the processor may predict, based on the state of the presentation environment <NUM> predicted to be induced by the one or more stimuli in the content <NUM> to be presented, a state of at least one communication link <NUM> of one or more communication links <NUM>. As discussed above, one or more stimuli may cause one or more users <NUM> to move, which may change the presentation environment <NUM>, and the change in the presentation environment <NUM> may change a state of one or more communication links <NUM>. Thus, in some embodiments at block <NUM>, the processor may predict, based on the state of the presentation environment predicted to be induced by the one or more stimuli in the content to be presented, that the at least one communication link would move from a first state having a first level of performance to a second state having a second level of performance. As will be apparent from the discussion below, where the second level of performance is a performance degradation relative to the first level of performance, the processor may perform one or more actions that may advantageously help avoid or reduce this performance degradation, thus improving content presentation.

At block <NUM>, the processor performs an action in response to the state of the at least one communication link meeting one or more criteria. In particular, at block <NUM>, the processor performs an action in response to the state of the at least one communication link, which was predicted (at block <NUM>) based on the state of the presentation environment predicted (at block <NUM>) to be induced by the one or more stimuli in the content, meeting one or more criteria. The one or more criteria may comprise a threshold, a metric, one or more other suitable criteria, or any combination thereof. In some embodiments, the state of the at least one communication link meets the one or more criteria when the state of the at least one communication link would cause a failure to receive a subset of the content. In some embodiments, the state of the at least one communication link meets the one or more criteria when the state of the at least one communication link would cause a failure to receive a subset of the interactive data. In some embodiments, the one or more criteria comprises a threshold quality of service of the at least one communication link. In some embodiments, the one or more criteria comprises a latency of the at least one communication link.

The processor may perform the action at block <NUM> at any desired time. In some embodiments, performing the action at block <NUM> may occur before the predicted state of the presentation environment occurs, before the predicted state of the at least one communication link occurs, before the at least one communication link has a state that meets the one or more criteria, or any combination thereof. In instances in which performing the action occurs before the predicted state of the presentation environment occurs, before the predicted state of the at least one communication link occurs, before the at least one communication link has a state that meets the one or more criteria, or any combination thereof, the action may advantageously help avoid or reduce a performance degradation of the at least one communication link, thus improving content presentation. Some exemplary actions that may be performed at block <NUM> and that may help improve content presentation are discussed with reference to <FIG> below.

The processor may predict the state of the presentation environment at block <NUM> and predict the state of the at least one communication link at block <NUM> at any desired times. In some embodiments, predicting the state of the presentation environment at block <NUM> occurs before the one or more stimuli are presented to the one or more users. In some embodiments, predicting the state of the at least one communication link at block <NUM> occurs before the one or more stimuli are presented to the one or more users. In instances in which predicting the state of the presentation environment at block <NUM> and/or predicting the state of the at least one communication link at block <NUM> occur before the one or more stimuli are presented to the one or more users, performing the action at block <NUM> may occur before the one or more stimuli are presented to the one or more users, before the predicted state of the presentation environment occurs, before the predicted state of the at least one communication link occurs, before the at least one communication link has a state that meets the one or more criteria, or any combination thereof -- which may advantageously help avoid or reduce a performance degradation of the at least one communication link, thus improving content presentation, as discussed above.

<FIG> is a flowchart illustrating an embodiment method for improving content presentation, which may be performed by one or more processors. As shown in <FIG>, in some embodiments, block <NUM> (<FIG>) may include one or more blocks, such as blocks <NUM> and <NUM>. In particular, to predict (at block <NUM>) the state of the presentation environment to be induced by the one or more stimuli in the content to be presented to the one or more users in the presentation environment, the processor may identify the one or more stimuli in the content to be presented to the one or more users in the presentation environment (at block <NUM>) and predict that the identified one or more stimuli, when presented to the one or more users in the presentation environment, would induce the state of the presentation environment (at block <NUM>).

<FIG> is a flowchart illustrating an embodiment method for improving content presentation, which may be performed by one or more processors. As shown in <FIG>, in some embodiments, block <NUM> (<FIG>) may include one or more blocks, such as block <NUM>. In particular, to predict (at block <NUM>) the state of the presentation environment to be induced by the one or more stimuli in the content to be presented to the one or more users in the presentation environment, the processor predicts that the one or more stimuli, when presented to the one or more users in the presentation environment, would induce movement by the one or more users that would result in the state of the presentation environment (at block <NUM>). For example, it may be predicted that the one or more stimuli, when presented to the one or more users <NUM> in the presentation environment <NUM>, would induce the one or more users to turn their heads, move their arms, move their legs, or make any other movement, resulting in the state of the presentation environment. Also, for example, it may be predicted that the one or more stimuli, when presented to the one or more users <NUM> in the presentation environment <NUM>, would induce the one or more users to make a movement that causes one or more objects in the presentation environment to move, resulting in the state of the presentation environment. As discussed above, such objects may include a head-mounted display <NUM> worn by a user <NUM>, an object unintentionally contacted by a user, an object intentionally contacted by a user, a robot that may move in response to user movement (e.g., a user gesture), or other objects.

<FIG> is a flowchart illustrating an embodiment method for improving content presentation, which may be performed by one or more processors. As shown in <FIG>, in some embodiments, block <NUM> (<FIG>) may include one or more blocks, such as block <NUM>. In particular, to predict (at block <NUM>) the state of the presentation environment to be induced by the one or more stimuli in the content to be presented to the one or more users in the presentation environment, the processor predicts that the one or more stimuli, when presented to the one or more users in the presentation environment, would induce movement by one or more objects in the presentation environment that would result in the state of the presentation environment.

As shown in block <NUM> in <FIG>, block <NUM> in <FIG>, block <NUM> in <FIG>, and block <NUM> in <FIG>, one or more predictions related to states of the presentation environment may be made. In some embodiments, as will be apparent from the following discussion, these predictions may be made using historical data indicating how one or more users responded to the content, using heuristics data indicating expected responses to the content independent of any gathered historical data, metadata for the content, data indicating the location and/or orientation of one or more users and/or objects in the presentation environment, other data, or any combination thereof.

Some or all of the predictions made in blocks <NUM>, <NUM>, <NUM>, and <NUM> are predicted using historical data indicating how one or more users responded to the content. This historical data indicates how one or more current users in the presentation environment previously responded to the content, how other users previously responded to the content, or both. In further detail, the historical data may indicate how the one or more users, in response to the content, turned their heads, moved their arms, moved their legs, made any other movement, remained at a location, moved to a location, remained motionless, or responded in any other way to the content. For example, in instances where the content comprises a first-person-shooter game in which an enemy visibly approaches a user from the periphery of the user's field of view (FoV), the historical data may indicate that users tended to turn their head towards the enemy and/or move towards the enemy. Also, for example, in instances where the content includes a sound, the historical data may indicate that users tended to turn their head towards the sound and/or move towards the sound.

Some or all of the predictions made in blocks <NUM>, <NUM>, <NUM>, and <NUM> may be predicted using heuristics data indicating expected responses to the content, which may be independent of any gathered historical data. In further detail, the data indicating expected responses to the content may indicate how the one or more users, in response to the content, are expected to turn their heads, move their arms, move their legs, make any other movement, remain at a location, move to a location, remain motionless, or respond in any other way to the content. For example, in instances where the content comprises a first-person-shooter game in which an enemy visibly approaches a user from the periphery of the user's field of view, the data indicating expected responses to the content may indicate that users would turn and run towards the enemy or that users would turn and run away from the enemy, depending upon game dynamics and strategy. Also, for example, in instances where the content includes a sound, the data indicating expected responses to the content may indicate that users would turn and run towards the sound or that users would turn and run away from the sound, depending upon game dynamics and strategy.

Some or all of the predictions made in blocks <NUM>, <NUM>, <NUM>, and <NUM> may be predicted using metadata for the content. The metadata for the content may be provided, for example, by a developer of the content or by some other means. The metadata for the content may include metadata indicating one or more stimuli in the content (e.g., one or more audio stimuli in the content, one or more visual stimuli in the content, one or more haptic feedback stimuli in the content, one or more other stimuli in the content, or any combination thereof). In such embodiments where metadata for the content is provided, the historical data indicating how one or more users responded to the content may comprise data associating the metadata indicating one or more stimuli in the content with how one or more users turned their heads, moved their arms, moved their legs, made any other movement, remained at a location, moved to a location, remained motionless, or responded in any other way to the one or more stimuli. Also, in such embodiments where metadata for the content is provided, the heuristics data indicating expected responses to the content may comprise data associating the metadata indicating one or more stimuli in the content with how one or more users are expected to turn their heads, move their arms, move their legs, make any other movement, remain at a location, move to a location, remain motionless, or respond in any other way to the one or more stimuli.

Some or all of the predictions made in blocks <NUM>, <NUM>, <NUM>, and <NUM> may be predicted using data indicating the location and/or orientation of one or more users and/or objects in the presentation environment. For example, in such embodiments where the data indicates the location and/or orientation of one or more pieces of furniture and indicates the location of one or more walls, the movements of one or more users may be predicted in view of the walls and furniture, which may be used to predict the state of the presentation environment. For instance, because the walls are typically fixed and some furniture is difficult for users move, predictions of the state of the presentation environment may account for the likelihood that users will move around the furniture and will not move through the walls. The data indicating the location and/or orientation of one or more users and/or objects in the presentation environment may be gathered using simultaneous localization and mapping (SLAM) technology, computer vision, any other suitable means, or any combination thereof.

<FIG> is a flowchart illustrating an embodiment method for improving content presentation, which may be performed by one or more processors. As shown in <FIG>, in some embodiments, block <NUM> (<FIG>) may include one or more blocks, such as block <NUM>.

At block <NUM>, the processor may use a communication link profile to identify a communication link state that the communication link profile associates with the state of the presentation environment predicted to be induced, wherein the predicted state of the at least one communication link is the identified communication link state. For example, the processor may use a communication link profile, such as communication link profile <NUM> in <FIG>, to identify a communication link state that the communication link profile associates with the state of the presentation environment that was predicted (at block <NUM>) to be induced; and the state of the at least one communication link predicted at block <NUM> may be the communication link state identified at block <NUM>.

The communication link profile <NUM> may comprise data associating one or more communication link states with one or more states of the presentation environment. The communication link profile <NUM> may include, for example, data directly associating one or more communication link states with one or more states of the presentation environment, data indirectly associating one or more communication link states with one or more states of the presentation environment (e.g., data in an artificial intelligence (AI) system), other data, or any combination thereof.

<FIG> is a flowchart illustrating an embodiment method <NUM> for generating a communication link profile (e.g., the communication link profile <NUM>), which may be performed by one or more processors. As shown in <FIG>, the method <NUM> may include one or more blocks, such as block <NUM>; and in some embodiments, block <NUM> may include one or more blocks, such as blocks <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>.

At block <NUM>, the processor may generate a communication link profile in any of a variety of suitable ways. As shown in <FIG>, the method <NUM> may optionally include the block <NUM>.

In some embodiments, to generate the communication link profile at block <NUM>, a user, a receiver, and/or a transmitter may perform one or more routines. For example, as shown in <FIG>, at block <NUM>, a head-mounted display or other component of the virtual reality system may prompt a user wearing the head-mounted display to perform a routine of movements (e.g., turn head, move arms, move legs, make any other movement, remain at a location, move to a location, remain motionless). In another example, at block <NUM>, a transmitter (e.g., a transmitter <NUM>) may perform a routine of transmissions, for example, while the user performs the routine of movements at block <NUM>. The routine of transmissions may include, for example, a variety of different beam configurations. In yet another example, at block <NUM>, a receiver (e.g., a receiver <NUM>) may perform a routine of receptions, for example, while the user performs the routine of movements at block <NUM>. The routine of receptions may include, for example, a variety of different antenna configurations.

While the user performs the routine of movements at block <NUM>, the transmitter performs the routine of transmissions at block <NUM>, and/or the receiver performs the routine of receptions at block <NUM>, the processor may identify one or more communication link states at block <NUM> and may identify one or more states of the presentation environment at block <NUM>. The processor may then store data associating one or more communication link states with one or more states of the presentation environment at block <NUM>. For example, at block <NUM>, the processor may store -- as at least a portion of the communication link profile -- associations between one or more identified states of the communication link used in the transmissions/receptions and one or more identified states of the presentation environment. The identified states of the communication link used in the transmissions/receptions may include, for example, data throughput of the communication link, latency of the communication link, signal strength of the wireless signal providing the communication link, an amount of consistency in the communication link (e.g., the communication link has data throughput, latency and/or other attributes that are even or relatively consistent), an amount of reliability in the communication (e.g., the communication link has a data throughput, latency and/or other attributes that are reliably at a desired level) or other communication link states. The identified states of the presentation environment may include, for example, a location and/or orientation of one or more users in the presentation environment, a location and/or orientation of one or more body parts of the one or more users, a location and/or orientation of one or more objects in the presentation environment, a condition of one or more objects in the presentation environment, other information reflecting the state of the presentation environment, or any combination thereof.

It will be appreciated, however, that blocks <NUM> and <NUM> are not required and that users need not be prompted to perform routines and that routines of transmissions and/or receptions need not be performed. For example, in some embodiments, to generate the communication link profile, one or more communication states may be identified at block <NUM> and one or more states of the presentation environment may be identified at block <NUM> during one or more users' ordinary use of the virtual reality system <NUM>. Thus, in such embodiments, while the users use the virtual reality system, the processor may (at block <NUM>) store the data associating one or more communication link states with one or more states of the presentation environment.

In some embodiments, to generate the communication link profile, a computer vision scan may be performed to gather data identifying a location and/or orientation of one or more users in the presentation environment, a location and/or orientation of one or more body parts of the one or more users, a location and/or orientation of one or more objects in the presentation environment, or any combination thereof.

<FIG> is a flowchart illustrating an embodiment method for improving content presentation, which may be performed by one or more processors. As shown in <FIG>, in some embodiments, block <NUM> (<FIG>) may include one or more blocks, such as blocks <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> or any combination thereof.

As shown at block <NUM>, the action performed in response to the state of the at least one communication link meeting one or more criteria at block <NUM> may comprise beam forming. For example, a processor of a transmitter <NUM>, a processor of a receiver <NUM>, or both may perform beam forming at block <NUM>.

As shown at block <NUM>, the action performed in response to the state of the at least one communication link meeting one or more criteria at block <NUM> may comprise beam steering. For example, a processor of a transmitter <NUM>, a processor of a receiver <NUM>, or both may perform beam steering at block <NUM>.

The beam forming (block <NUM>) and the beam steering (block <NUM>) may include reconfiguring one or more antennas of one or more receivers <NUM>, reconfiguring one or more antennas of one or more transmitters <NUM>, or both. Reconfiguring one or more antennas of a receiver <NUM> may include, for example, switching from using one antenna of the receiver to another antenna of the receiver or altering the configuration of one or more currently used antennas of the receiver. Reconfiguring one or more antennas of a transmitter <NUM> may include, for example, switching from using one antenna of the transmitter to another antenna of the transmitter or altering the configuration of one or more currently used antennas of the transmitter.

The beam forming (block <NUM>) and the beam steering (block <NUM>) may be based on a communication link profile, such as the communication link profile <NUM>. For example, as part of block <NUM> or block <NUM>, the processor may use the communication link profile <NUM> to identify a state of the presentation environment <NUM> that the communication link profile associates with an improved communication link state, and the processor may then change the presentation environment to the identified state of the presentation environment by reconfiguring one or more antennas of the receiver <NUM> and/or reconfiguring one or more antennas of the transmitter <NUM>. In some instances, where the predicted state of the at least one communication link and/or the predicted state of the presentation environment involves a degradation of performance, the pre-beam forming and/or the beam steering may allow the content presentation device to present the content with fewer (or no) pauses or skips and in sync with the user's expectations, during the duration of that performance degradation.

As shown at block <NUM>, the action performed in response to the state of the at least one communication link meeting one or more criteria at block <NUM> may comprise switching from a first set of one or more transmitters providing the content to a second set of one or more transmitters providing the content. For example, this switching may include (<NUM>) switching from using a first access point to using a second access point that uses the same type of wireless communication link as the first access point or (<NUM>) switching from using a first access point to using a second access point that uses a different second type of wireless communication link than the first access point (e.g., from a millimeter wave wireless link to a wireless communication link other than millimeter wave, such as WiFi or Bluetooth).

The switching at block <NUM> may be based on a communication link profile, such as the communication link profile <NUM>. For example, as part of block <NUM>, the processor may use the communication link profile <NUM> to identify a state of the presentation environment <NUM> that the communication link profile associates with an improved communication link state, and the processor may then change the presentation environment to the identified state of the presentation environment by switching from a first set of one or more transmitters <NUM> providing the content to a second set of one or more transmitters <NUM> providing the content. In some instances, where the predicted state of the at least one communication link and/or the predicted state of the presentation environment involves a degradation of performance, the switching at block <NUM> may allow the content presentation device to present the content with fewer (or no) pauses or skips and in sync with the user's expectations, during the duration of that performance degradation.

As shown at block <NUM>, the action performed in response to the state of the at least one communication link meeting one or more criteria at block <NUM> may comprise buffering content. For example, in some embodiments, a processor <NUM> of a content presentation device <NUM> may pre-fetch (and buffer) additional content sufficient to present content during a predicted duration in which the predicted state of the at least one communication link meets the one or more criteria. As another example, a processor <NUM> of a content presentation device <NUM> may pre-fetch (and buffer) additional content sufficient to present content during a predicted duration in which the predicted state of the presentation environment occurs. In some instances, where the predicted state of the at least one communication link and/or the predicted state of the presentation environment involves a degradation of performance, the pre-fetching and buffering may allow the content presentation device to present the content with fewer (or no) pauses or skips, during the duration of that performance degradation.

As shown at block <NUM>, the action performed in response to the state of the at least one communication link meeting one or more criteria at block <NUM> may comprise prompting one or more users to move to a location and/or an orientation. For example, in some embodiments, a head-mounted display <NUM> or other component of the virtual reality system <NUM> may prompt a user to move to a selected location and/or an orientation. Where two or more users are using the virtual reality system <NUM> (e.g., in a peer-to-peer virtual reality system), some or all of the users may be prompted to move to one or more selected locations and/or orientations at block <NUM>.

The location and/or orientation at block <NUM> may be selected using the communication link profile <NUM>. The communication link profile <NUM> may be used to select a location and/or orientation predicted to avoid one or more states of the at least one communication link and/or the presentation environment that would result in a degradation of performance.

The users may be prompted at block <NUM> at any suitable time. For example, in some embodiments, a user may be prompted to move to a selected location and/or orientation to begin experiencing the content (e.g., a starting point). As another example, a user may be prompted to move to a selected location and/or orientation to while the user is experiencing the content (e.g., while the user is wearing a head-mounted display and is immersed in virtual reality). In some instances, the content <NUM> itself may be dynamically selected to cause to the user to move to the selected location and/or orientation. If desired, the dynamically selected content may be configured to be seamlessly integrated into the content such that the user is not aware of the prompt, thus preserving the user's immersion in virtual reality. Alternatively, the dynamically selected content may be configured to be distinct from the virtual reality (e.g., arrows or other directional indicators), which may be more effective in prompting the user's movement but may reduce the user's immersion in virtual reality.

As shown at block <NUM>, the action performed in response to the state of the at least one communication link meeting one or more criteria at block <NUM> may comprise using an intermediary. For example, in a peer-to-peer virtual reality system that includes a first content presentation device and a second first content presentation device, an access point, router, repeater, or a third content presentation device in the peer-to-peer virtual reality system may be used as an intermediary between the first and second content presentation devices at block <NUM>. In some instances, where the predicted state of the at least one communication link and/or the predicted state of the presentation environment involves a degradation of performance, the use of the intermediary may allow one or more content presentations devices to present the content with fewer (or no) pauses or skips and in sync with the user's expectations, during the duration of that performance degradation.

As shown at block <NUM>, the action performed in response to the state of the at least one communication link meeting one or more criteria at block <NUM> may comprise providing reduced quality content. In some embodiments, a transmitter <NUM> or other component of the virtual reality system <NUM> may provide reduced quality content. The reduced quality content may include content at a lower resolution, content at a lower frame rate, content with reduced colors, content with reduced or no background imagery, or other content with reduced quality.

In such embodiments, the transmitter <NUM> or other component of the virtual reality system <NUM> may configure the reduced quality content to include higher priority content and omit lower priority content. Including higher priority content and omitting lower priority content may be useful, for instance, where block <NUM> involves switching to a reliable but slower communication link type (e.g., from a millimeter wave wireless link to a wireless communication link other than millimeter wave, such as WiFi or Bluetooth). The prioritization of the content and decision to include or omit portions of the content may be performed using one or more algorithms based on defined rules and/or dynamically at runtime. For example, in an automobile driving virtual reality, an experienced user may not require the background image detail to enjoy the experience or perform well, and thus, the algorithm may deprioritize and/or omit the background imagery from the content. Also, for example, in a battlefield portion of a virtual reality, a user may require the content to be in full resolution; however, when walking through a path that has minimal activity, the frame rate or resolution could be reduced by the algorithm. In some instances, where the predicted state of the at least one communication link and/or the predicted state of the presentation environment involves a degradation of performance, providing reduced quality content may allow the content presentation device to present a satisfactory version of the content with fewer (or no) pauses or skips, during the duration of that performance degradation.

As shown at block <NUM>, the action performed in response to the state of the at least one communication link meeting one or more criteria at block <NUM> may comprise triggering one or more actions, for example, any of the actions at blocks <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> or any combination thereof. In particular, any component of the virtual reality system <NUM> may perform some or all the method <NUM> and, as part of that performance, may (at block <NUM>) trigger itself and/or any other component of the virtual reality system <NUM> to perform any of the actions at blocks <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM> or any combination thereof.

<FIG> is a flowchart illustrating an embodiment method for improving content presentation, which may be performed by one or more processors. As shown in <FIG>, in some embodiments, block <NUM> (<FIG>) may include one or more blocks, such as blocks <NUM>. In particular, to perform (at block <NUM>) an action in response to the state of the at least one communication link meeting one or more criteria, the processor may select one or more actions from a defined plurality of actions at block <NUM> and may perform the selected one or more actions at block <NUM>. In some embodiments, the defined plurality of actions may comprise any of the actions at blocks <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>; one or more other actions; or any combination thereof.

In some embodiments, the selection at block <NUM> may involve selecting what is the best action to take based on the predicted state of the at least one communication link and/or the predicted state of the presentation environment. In some embodiments, the selection at block <NUM> may be based on a probability that the predicted state of the at least one communication link and/or the predicted state of the presentation environment will or will not remain for a defined duration. For instance, certain actions may be more desirable if a user is likely to quickly return to a previous location and/or orientation, while other actions may be more desirable if a user is likely to remain at a predicted location and/or orientation for the defined duration. In some embodiments, the selection at block <NUM> may be based on at least one of one or more properties of the content to be presented to one or more users in the presentation environment, one or more properties of the state of the presentation environment predicted to be induced by the one or more stimuli in the content to be presented, one or more properties of the predicted state of at least one communication link, other factors, or any combination thereof.

In addition to performing the methods above, the virtual reality system <NUM> could also be configured to download as much relevant content <NUM> as possible while a strong and stable communication link <NUM> is available. A user <NUM> may be moving around quite a bit during a virtual reality session and frequently coming in and out of access to the communication link <NUM>. In this case, the virtual reality system <NUM> could provide as much content <NUM> as possible during the periods in which the communication link <NUM> is available, subject (of course) to the ability of a content presentation device's ability to store the content.

The functionality of the embodiments disclosed herein -- including but not limited to the functionality described with reference to the methods illustrated in <FIG>, <FIG>, <FIG>, <FIG>, <FIG>, <FIG> and <FIG> -- may be performed by a "processing system" that includes one or more processors (e.g., a processor <NUM>, a processor of a receiver <NUM>, a processor of transmitter <NUM>, a processor of an access point <NUM>, a processor of a router <NUM>, a processor of a repeater <NUM>, a processor one or more other components of the virtual reality system <NUM>, or any combination thereof). Examples of such processors include microprocessors, microcontrollers, graphics processing units (GPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems on a chip (SoC), baseband processors, field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. One or more processors in the processing system may execute software to perform the functionality of the embodiments disclosed herein.

In one or more exemplary aspects, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable medium or non-transitory processor-readable medium. The steps of a method or algorithm disclosed herein may be embodied in a processor-executable software module and/or processor-executable instructions, which may reside on a non-transitory computer-readable or non-transitory processor-readable storage medium. Non-transitory server-readable, computer-readable or processor-readable storage media may be any storage media that may be accessed by a computer or a processor. By way of example but not limitation, such non-transitory server-readable, computer-readable or processor-readable media may include RAM, ROM, EEPROM, FLASH memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. Combinations of the above are also included within the scope of non-transitory server-readable, computer-readable and processor-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a non-transitory server-readable, processor-readable medium and/or computer-readable medium, which may be incorporated into a computer program product.

The methods and systems described above require no particular component or function. Thus, any described component or function -- despite its advantages -- is optional. Also, some or all of the described components and functions described above may be used in connection with any number of other suitable components and functions.

One skilled in the art will also appreciate that although the exemplary embodiments discussed above have been described with respect to a virtual reality system, these aspects and features may also be used in connection with other types of systems that may present content.

Claim 1:
A method (<NUM>) for improving content presentation, the method comprising:
by a processor (<NUM>):
predicting (<NUM>) a state of a presentation environment (<NUM>) to be induced by one or more stimuli in content (<NUM>) to be presented to one or more users (<NUM>) in the presentation environment (<NUM>) including predicting that the one or more stimuli, when presented to the one or more users (<NUM>) in the presentation environment (<NUM>), would induce movement by the one or more users (<NUM>) that would result in the state of the presentation environment (<NUM>), wherein predicting (<NUM>) the state of the presentation environment is performed using historical data that indicates how one or more users previously moved in response to the content;
predicting (<NUM>), based on the state of the presentation environment (<NUM>) predicted to be induced by the one or more stimuli in the content (<NUM>) to be presented, a state of at least one communication link (<NUM>) of one or more communication links (<NUM>), the one or more communication links (<NUM>) used to deliver the content (<NUM>) to be presented and used to deliver at least a portion of the content (<NUM>) after the one or more stimuli are presented;
performing (<NUM>) an action in response to the predicted state of the at least one communication link (<NUM>), which was predicted based on the state of the presentation environment (<NUM>) predicted to be induced by the one or more stimuli in the content (<NUM>), meeting one or more criteria, wherein performing the action avoids or reduces a predicted performance degradation of the at least one communication link (<NUM>) as a result of the one or more stimuli and the action comprises one or more of: beam forming, beam steering of the communication link, triggering beam forming, or triggering beam steering; and
presenting the content (<NUM>) to be presented.