Patent Description:
In some instances, enhanced reality (ER) content is created without regard to spatial characteristics (e.g., volumetric space, shape, etc.) of a physical setting. For example, when the physical setting corresponds to a large-sized room with ample space, a user may be presented with ER content and interact with the ER content by walking around the physical setting. However, in another example, when the physical setting corresponds to a small room with limited space, the user is unable to navigate the ER content by traveling through the physical setting in the same way the user is able to travel through the large-sized room.

<CIT> describes a method of displaying a virtual reality simulation on a head-mounted display, wherein a user is instructed to perform a task and monitored for voluntary and involuntary responses to determine visual disability of the user.

<CIT> - and <NPL> - also relate to enhanced reality and the display of navigation paths.

In accordance with common practice the various features illustrated in the drawings may not be drawn to scale. Accordingly, the dimensions of the various features may be arbitrarily expanded or reduced for clarity. In addition, some of the drawings may not depict all of the components of a given system, method, or device. Finally, like reference numerals may be used to denote like features throughout the specification and figures.

The invention is directed to the method of claim <NUM>, the electronic device of claim <NUM> and the non-transitory memory of claim <NUM>.

Numerous details are described in order to provide a thorough understanding of the example implementations shown in the drawings. However, the drawings merely show some example aspects of the present disclosure and are therefore not to be considered limiting. Those of ordinary skill in the art will appreciate that other effective aspects and/or variants do not include all of the specific details described herein. Moreover, well-known systems, methods, components, devices, and circuits have not been described in exhaustive detail so as not to obscure more pertinent aspects of the example implementations described herein. Various examples of electronic systems and techniques for using such systems in relation to various enhanced reality technologies are described.

A physical setting refers to a world with which various persons can sense and/or interact without use of electronic systems. Physical settings, such as a physical park, include physical elements, such as, for example, physical wildlife, physical trees, and physical plants. Persons can directly sense and/or otherwise interact with the physical setting, for example, using one or more senses including sight, smell, touch, taste, and hearing.

An enhanced reality (ER) setting, in contrast to a physical setting, refers to an entirely (or partly) computer-produced setting that various persons, using an electronic system, can sense and/or otherwise interact with. In ER, a person's movements are in part monitored, and, responsive thereto, at least one attribute corresponding to at least one virtual object in the ER setting is changed in a manner that is consistent with one or more physical laws. For example, in response to an ER system detecting a person looking upward, the ER system may adjust various audio and graphics presented to the person in a manner consistent with how such sounds and appearances would change in a physical setting. Adjustments to attribute(s) of virtual object(s) in an ER setting also may be made, for example, in response to representations of movement (e.g., voice commands).

A person may sense and/or interact with an ER object using one or more senses, such as sight, smell, taste, touch, and sound. For example, a person may sense and/or interact with objects that create a multi-dimensional or spatial acoustic setting. Multi-dimensional or spatial acoustic settings provide a person with a perception of discrete acoustic sources in multi-dimensional space. Such objects may also enable acoustic transparency, which may selectively incorporate audio from a physical setting, either with or without computer-produced audio. In some ER settings, a person may sense and/or interact with only acoustic objects.

Virtual reality (VR) is one example of ER. A VR setting refers to an enhanced setting that is configured to only include computer-produced sensory inputs for one or more senses. A VR setting includes a plurality of virtual objects that a person may sense and/or interact with. A person may sense and/or interact with virtual objects in the VR setting through a simulation of at least some of the person's actions within the computer-produced setting, and/or through a simulation of the person or her presence within the computer-produced setting.

Mixed reality (MR) is another example of ER. An MR setting refers to an enhanced setting that is configured to integrate computer-produced sensory inputs (e.g., virtual objects) with sensory inputs from the physical setting, or a representation of sensory inputs from the physical setting. On a reality spectrum, an MR setting is between, but does not include, a completely physical setting at one end and a VR setting at the other end.

In some MR settings, computer-produced sensory inputs may be adjusted based on changes to sensory inputs from the physical setting. Moreover, some electronic systems for presenting MR settings may detect location and/or orientation with respect to the physical setting to enable interaction between real objects (i.e., physical elements from the physical setting or representations thereof) and virtual objects. For example, a system may detect movements and adjust computer-produced sensory inputs accordingly, so that, for example, a virtual tree appears fixed with respect to a physical structure.

Augmented reality (AR) is an example of MR. An AR setting refers to an enhanced setting where one or more virtual objects are superimposed over a physical setting (or representation thereof). As an example, an electronic system may include an opaque display and one or more imaging sensors for capturing video and/or images of a physical setting. Such video and/or images may be representations of the physical setting, for example. The video and/or images are combined with virtual objects, wherein the combination is then displayed on the opaque display. The physical setting may be viewed by a person, indirectly, via the images and/or video of the physical setting. The person may thus observe the virtual objects superimposed over the physical setting. When a system captures images of a physical setting, and displays an AR setting on an opaque display using the captured images, the displayed images are called a video pass-through. Alternatively, a transparent or semi-transparent display may be included in an electronic system for displaying an AR setting, such that an individual may view the physical setting directly through the transparent or semi-transparent displays. Virtual objects may be displayed on the semi-transparent or transparent display, such that an individual observes virtual objects superimposed over a physical setting. In yet another example, a projection system may be utilized in order to project virtual objects onto a physical setting. For example, virtual objects may be projected on a physical surface, or as a holograph, such that an individual observes the virtual objects superimposed over the physical setting.

An AR setting also may refer to an enhanced setting in which a representation of a physical setting is modified by computer-produced sensory data. As an example, at least a portion of a representation of a physical setting may be graphically modified (e.g., enlarged), so that the modified portion is still representative of (although not a fully reproduced version of) the originally captured image(s). Alternatively, in providing video pass-through, one or more sensor images may be modified in order to impose a specific viewpoint different than a viewpoint captured by the image sensor(s). As another example, portions of a representation of a physical setting may be altered by graphically obscuring or excluding the portions.

Augmented virtuality (AV) is another example of MR. An AV setting refers to an enhanced setting in which a virtual or computer-produced setting integrates one or more sensory inputs from a physical setting. Such sensory input(s) may include representations of one or more characteristics of a physical setting. A virtual object may, for example, incorporate a color associated with a physical element captured by imaging sensor(s). Alternatively, a virtual object may adopt characteristics consistent with, for example, current weather conditions corresponding to a physical setting, such as weather conditions identified via imaging, online weather information, and/or weather-related sensors. As another example, an AR park may include virtual structures, plants, and trees, although animals within the AR park setting may include features accurately reproduced from images of physical animals.

Various systems allow persons to sense and/or interact with ER settings. For example, a head mounted system may include one or more speakers and an opaque display. As another example, an external display (e.g., a smartphone) may be incorporated within a head mounted system. The head mounted system may include microphones for capturing audio of a physical setting, and/or image sensors for capturing images/video of the physical setting. A transparent or semi-transparent display may also be included in the head mounted system. The semi-transparent or transparent display may, for example, include a substrate through which light (representative of images) is directed to a person's eyes. The display may also incorporate LEDs, OLEDs, liquid crystal on silicon, a laser scanning light source, a digital light projector, or any combination thereof. The substrate through which light is transmitted may be an optical reflector, holographic substrate, light waveguide, optical combiner, or any combination thereof. The transparent or semi-transparent display may, for example, transition selectively between a transparent/semi-transparent state and an opaque state. As another example, the electronic system may be a projection-based system. In a projection-based system, retinal projection may be used to project images onto a person's retina. Alternatively, a projection-based system also may project virtual objects into a physical setting, for example, such as projecting virtual objects as a holograph or onto a physical surface. Other examples of ER systems include windows configured to display graphics, headphones, earphones, speaker arrangements, lenses configured to display graphics, heads up displays, automotive windshields configured to display graphics, input mechanisms (e.g., controllers with or without haptic functionality), desktop or laptop computers, tablets, or smartphones.

<FIG> is a block diagram of an example operating architecture <NUM> in accordance with some implementations. While pertinent features are shown, those of ordinary skill in the art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity and so as not to obscure more pertinent aspects of the example implementations disclosed herein. To that end, as a non-limiting example, the operating architecture <NUM> includes an optional controller <NUM> and an electronic device <NUM> (e.g., a tablet, mobile phone, laptop, wearable computing device, or the like).

In some implementations, the controller <NUM> is configured to manage and coordinate an ER experience for a user <NUM> (sometimes also referred to herein as a "virtual environment" or a "graphical environment") and zero or more other users. In some implementations, the controller <NUM> includes a suitable combination of software, firmware, and/or hardware. The controller <NUM> is described in greater detail below with respect to <FIG>. In some implementations, the controller <NUM> is a computing device that is local or remote relative to the physical setting <NUM>. For example, the controller <NUM> is a local server located within the physical setting <NUM>. In another example, the controller <NUM> is a remote server located outside of the physical setting <NUM> (e.g., a cloud server, central server, etc.). In some implementations, the controller <NUM> is communicatively coupled with the electronic device <NUM> via one or more wired or wireless communication channels <NUM> (e.g., BLUETOOTH, IEEE <NUM>1x, IEEE <NUM>. 16x, IEEE <NUM>. In some implementations, the functions of the controller <NUM> are provided by the electronic device <NUM>. As such, in some implementations, the components of the controller <NUM> are integrated into the electronic device <NUM>.

In some implementations, the electronic device <NUM> is configured to present audio and/or video content to the user <NUM>. In some implementations, the electronic device <NUM> is configured to present the ER experience to the user <NUM>. In some implementations, the electronic device <NUM> includes a suitable combination of software, firmware, and/or hardware. The electronic device <NUM> is described in greater detail below with respect to <FIG>.

According to some implementations, the electronic device <NUM> presents an enhanced reality (ER) experience to the user <NUM> while the user <NUM> is physically present within a physical setting <NUM> that includes a table <NUM> within the field-of-view <NUM> of the electronic device <NUM>. As such, in some implementations, the user <NUM> holds the electronic device <NUM> in his/her hand(s). In some implementations, while presenting the ER experience, the electronic device <NUM> is configured to present ER content (e.g., an ER cylinder <NUM>) and to enable video pass-through of the physical setting <NUM> (e.g., including the table <NUM>) on a display <NUM>. For example, the electronic device <NUM> corresponds to a mobile phone, tablet, laptop, wearable computing device, or the like.

In some implementations, the display <NUM> corresponds to an additive display that enables optical see-through of the physical setting <NUM> including the table <NUM>. For example, the display <NUM> correspond to a transparent lens, and the electronic device <NUM> corresponds to a pair of glasses worn by the user <NUM>. As such, in some implementations, the electronic device <NUM> presents a user interface by projecting the ER content (e.g., the ER cylinder <NUM>) onto the additive display, which is, in turn, overlaid on the physical setting <NUM> from the perspective of the user <NUM>. In some implementations, the electronic device <NUM> presents the user interface by displaying the ER content (e.g., the ER cylinder <NUM>) on the additive display, which is, in turn, overlaid on the physical setting <NUM> from the perspective of the user <NUM>.

In some implementations, the user <NUM> wears the electronic device <NUM> such as a near-eye system. As such, the electronic device <NUM> includes one or more displays provided to display the ER content (e.g., a single display or one for each eye). For example, the electronic device <NUM> encloses the field-of-view of the user <NUM>. In such implementations, the electronic device <NUM> presents the ER setting by displaying data corresponding to the ER setting on the one or more displays or by projecting data corresponding to the ER setting onto the retinas of the user <NUM>.

In some implementations, the electronic device <NUM> includes an integrated display (e.g., a built-in display) that displays the ER setting. In some implementations, the electronic device <NUM> includes a head-mountable enclosure. In various implementations, the head-mountable enclosure includes an attachment region to which another device with a display can be attached. For example, in some implementations, the electronic device <NUM> can be attached to the head-mountable enclosure. In various implementations, the head-mountable enclosure is shaped to form a receptacle for receiving another device that includes a display (e.g., the electronic device <NUM>). For example, in some implementations, the electronic device <NUM> slides/snaps into or otherwise attaches to the head-mountable enclosure. In some implementations, the display of the device attached to the head-mountable enclosure presents (e.g., displays) the ER setting. In some implementations, the electronic device <NUM> is replaced with an ER chamber, enclosure, or room configured to present ER content in which the user <NUM> does not wear the electronic device <NUM>.

In some implementations, the controller <NUM> and/or the electronic device <NUM> cause an ER representation of the user <NUM> to move within the ER setting based on movement information (e.g., body pose data, eye tracking data, hand tracking data, etc.) from the electronic device <NUM> and/or optional remote input devices within the physical setting <NUM>. In some implementations, the optional remote input devices correspond to fixed or movable sensory equipment within the physical setting <NUM> (e.g., image sensors, depth sensors, infrared (IR) sensors, event cameras, microphones, etc.). In some implementations, each of the remote input devices is configured to collect/capture input data and provide the input data to the controller <NUM> and/or the electronic device <NUM> while the user <NUM> is physically within the physical setting <NUM>. In some implementations, the remote input devices include microphones, and the input data includes audio data associated with the user <NUM> (e.g., speech samples). In some implementations, the remote input devices include image sensors (e.g., cameras), and the input data includes images of the user <NUM>. In some implementations, the input data characterizes body poses of the user <NUM> at different times. In some implementations, the input data characterizes head poses of the user <NUM> at different times. In some implementations, the input data characterizes hand tracking information associated with the hands of the user <NUM> at different times. In some implementations, the input data characterizes the velocity and/or acceleration of body parts of the user <NUM> such as his/her hands. In some implementations, the input data indicates joint positions and/or joint orientations of the user <NUM>. In some implementations, the remote input devices include feedback devices such as speakers, lights, or the like.

<FIG> illustrate a comparison between a first ER presentation scenario 200a sequence and a second ER presentation scenario 200b sequence in accordance with some implementations. While pertinent features are shown, those of ordinary skill in the art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity and so as not to obscure more pertinent aspects of the example implementations disclosed herein.

The ER presentation scenario sequences shown in <FIG> depict a comparison between the electronic device <NUM> and/or the controller (e.g., the controller <NUM> shown in <FIG> and <FIG>) adapting ER content in a first ER presentation scenario 200a and a second ER presentation scenario 200b. The comparison highlights the similarities and differences between generating adapted ER content based on the spatial characteristics of a first physical setting <NUM> corresponding to a large empty room and on the spatial characteristics of a second physical setting <NUM> corresponding to a small room filled with objects.

<FIG> illustrates a first state 201a (e.g., associated with T1 or a first time period) of a first ER presentation scenario 200a associated with the first physical setting <NUM> and a second ER presentation scenario 200b associated with the second physical setting <NUM>. With reference to the first ER presentation scenario 200a in <FIG>, the first physical setting <NUM> includes an empty room with a first x-dimension 203a, a first y-dimension 205a, and a first z-dimension 206a. As shown in the second ER presentation scenario 200b, the second physical setting <NUM> includes a room with a second x-dimension 203b, a second y-dimension 205b, and a second z-dimension 206b. For example, the second x-dimension 203b, the second y-dimension 205b, and the second z-dimension 206b of the second physical setting <NUM> are significantly smaller than the first x-dimension 203a, the first y-dimension 205a, and the first z-dimension 206a of the first physical setting <NUM>. Thus, compared to the first physical setting <NUM>, the second physical setting <NUM> corresponds to a much smaller volumetric size than the first physical setting <NUM>. Furthermore, in contrast to the empty room in the first physical setting <NUM>, the second ER presentation scenario 200b includes physical objects such as chairs 220a, 220c, a credenza 220b, coffee tables 220d, 220e, and a sofa <NUM> within the second physical setting <NUM>.

In some implementations, where the field-of-view of the user is enclosed, the electronic device <NUM> is configured to enable video pass-through of the first physical setting <NUM> on a display <NUM>. In some implementations, the electronic device <NUM> is also configured to present the first ER presentation scenario 200a on the display <NUM>. In some implementations, the display <NUM> corresponds to an additive display that enables optical-see through of the first physical setting <NUM>. For example, the display <NUM> corresponds to a transparent lens and the electronic device <NUM> corresponds to a pair of glasses worn by the user. In some implementations, the electronic device <NUM> presents the first ER presentation scenario 200a by projecting adapted ER content on the additive display, which is, in turn overlaid on the first physical setting <NUM> from the perspective of the user. In some implementations, the electronic device <NUM> presents the first ER presentation scenario 200a by rendering the adapted ER on the additive display, which is also, in turn overlaid on the first physical setting <NUM> from the perspective of the user.

In some implementations, where the field-of-view of the user is enclosed, the electronic device <NUM> is configured to enable video pass-through of the second physical setting <NUM> including the physical objects on the display <NUM>. In some implementations, the electronic device <NUM> is also configured to present the second ER presentation scenario 200b on the display <NUM>. In some implementations, the display <NUM> corresponds to an additive display that enables optical-see through of the second physical setting <NUM> including the physical objects. For example, the display <NUM> corresponds to a transparent lens and the electronic device <NUM> corresponds to a pair of glasses worn by the user. In some implementations, the electronic device <NUM> presents the second ER presentation scenario 200b by projecting adapted ER content on the additive display, which is, in turn overlaid on the second physical setting <NUM> from the perspective of the user. In some implementations, the electronic device <NUM> presents the second ER presentation scenario 200b by rendering the adapted ER on the additive display, which is also, in turn overlaid on the second physical setting <NUM> from the perspective of the user.

<FIG> illustrates a second state 201b (e.g., associated with T2 or a second time period) of the first ER presentation scenario 200a and the second ER presentation scenario 200b. In some implementations, the electronic device <NUM> and/or the controller identifies a plurality of subsets associated with the physical setting. In some implementations, the electronic device <NUM> obtains, via an exterior-facing image sensor of the electronic device, image data that corresponds to the physical setting. In turn, the electronic device <NUM> identifies the plurality of subsets within the physical setting based at least in part on the image data and determines the sets of spatial characteristics for the plurality of subsets based at least in part on the image data and/or depth information. In some implementations, the electronic device <NUM> obtains depth information, image data, and/or the like from the one or more optional remote input devices.

In some implementations, the electronic device <NUM> identifies a plurality of subsets of the first physical setting <NUM> including a first subset 230a and a second subset 232a. In some implementations, the electronic device <NUM> determines the plurality of subsets based on a metric that is not associated with physical divisions. For example, if a physical setting does not include physical divisions, then the electronic device <NUM> identifies the plurality of subsets by dividing the physical setting in half and identifying a first half of the physical setting as a first subset and the second half of the physical setting as a second subset. Similarly, as shown in the second ER presentation scenario 200b, the electronic device <NUM> also identifies a plurality of subsets of the second physical setting <NUM> as a first subset 230b and a second subset 232b. In some implementations, the electronic device <NUM> determines the plurality of subsets based on physical division. For example, with reference to <FIG>, if the physical setting corresponds to a house <NUM>, then the electronic device <NUM> identifies the plurality of subsets based on each individual room (e.g., the basement <NUM>, the kitchen <NUM>, the bedroom <NUM>, and the living room <NUM>). As another example, if the physical setting corresponds to a single room, then the electronic device <NUM> identifies the plurality of subsets based on the different walls of the single room. As yet another example, if the physical setting corresponds to a single room, then the electronic device <NUM> identifies the plurality of subsets based on the corners of a room.

In some implementations, the electronic device <NUM> determines a set of spatial characteristics for each of the plurality of subsets within the physical setting. As shown in the first ER presentation scenario 200a, the electronic device <NUM> determines a first set of spatial characteristics for the first subset 230a including, for example: a volumetric size of the first subset 230a based on an x-dimension 207a, a y-dimension 209a, and a z-dimension 216a; an indication of no physical objects within the first subset 230a; and/or the like. Additionally, the electronic device <NUM> also determines a second set of spatial characteristics for the second subset 232a including, for example: a volumetric size of the second subset 232a based on an x-dimension 211a, a y-dimension 213a, and a z-dimension 215a; an indication of no physical objects within the first physical setting <NUM>; and/or the like.

Similarly, as shown in the second ER presentation scenario 200b, the electronic device <NUM> also determines a first set of spatial characteristics for the first subset 230b including, for example: a volumetric size of the first subset 230b based on an x-dimension 207b, a y-dimension 209b, and a z-dimension 216b; an indication of physical objects (e.g., the chair 220c, the credenza 220b, the coffee table 220e, and the sofa <NUM>) within the second physical setting <NUM>; and/or the like. Additionally, the electronic device <NUM> also determines a second set of spatial characteristics for the second subset 232b including, for example, a volumetric size of the second subset 232b based on an x-dimension 211b, a y-dimension 213b, and a z-dimension 215b; an indication of physical objects (e.g., the chair 220a, the credenza 220b, the coffee tables 220d, 220e, and the sofa <NUM>) within the second physical setting <NUM> and/or the like. As shown in <FIG>, the dimensions of the first subset 230b and the second subset 232b of the second physical setting <NUM> are much smaller than dimensions of the first subset 230a and second subset 232a of the first physical setting <NUM>.

<FIG> illustrates a third state 201c (e.g., associated with T3 or a third time period) of the first ER presentation scenario 200a and the second ER presentation scenario 200b. In some implementations, after logically mapping a first ER portion to the first subset 230a, the electronic device <NUM> and/or the controller <NUM> generates a first ER content portion adapted from predetermined ER content based at least in part on the first set of spatial characteristics. With reference to the first ER presentation scenario 200a in <FIG>, the electronic device <NUM> generates the adapted first ER content 240a-<NUM>, 240a-<NUM>, and 240a-<NUM> based on the first set of spatial characteristics for the first subset 230a of the first physical setting <NUM>. As shown in <FIG>, the electronic device <NUM> presents the adapted first ER content 240a-<NUM>, 240a-<NUM>, and 240a-<NUM> within the first subset 230a of the first physical setting <NUM>.

Similarly, in some implementations, after logically mapping a first ER portion to the first subset 230b, the electronic device <NUM> and/or the controller <NUM> generates an adapted first ER content portion by adapting predetermined ER content based on the first set of spatial characteristics. With reference to the second ER presentation scenario 200b in <FIG>, the electronic device <NUM> generates the adapted first ER content 240b-<NUM>, 240b-<NUM>, and 240b-<NUM> based on the first set of spatial characteristics for the first subset 230b of the second physical setting <NUM>. As shown in <FIG>, the electronic device <NUM> presents the adapted first ER content 240b-<NUM>, 240b-<NUM>, and 240b-<NUM> within the first subset 230b of the second physical setting <NUM>. However, compared to the first ER presentation scenario 200a, the adapted ER content 240b-<NUM>, 240b-<NUM>, and 240b-<NUM> in the first subset 230b is scaled down to a smaller size than the adapted ER content 240a-<NUM>, 240a-<NUM>, 240a-<NUM> in the first subset 230a due to the difference in dimensions between the first subset 230a of the first physical setting <NUM> and the first subset 230b of the second physical setting <NUM>.

In some implementations, after logically mapping a second ER portion to the second subset 232a, the electronic device <NUM> and/or the controller <NUM> generates a second ER content portion by adapting predetermined ER content based on the second set of spatial characteristics. With reference to the first ER presentation scenario 200a in <FIG>, the electronic device <NUM> generates adapted second ER content 242a-<NUM>, 242a-<NUM>, 242a-<NUM>, and 242a-<NUM> based on the second set of spatial characteristics for the second subset 232a of the first physical setting <NUM>. As shown in <FIG>, the electronic device <NUM> presents the adapted second ER content 242a-<NUM>, 242a-<NUM>, 242a-<NUM>, and 242a-<NUM> within the second subset 232ba of the first physical setting <NUM>.

Similarly, in some implementations, after logically mapping a second ER portion to the second subset 232b, the electronic device <NUM> and/or the controller <NUM> generates a second ER content portion by adapting the predetermined ER content based on the second set of spatial characteristics. With reference to the second ER presentation scenario 200b in <FIG>, the electronic device <NUM> generates the adapted second ER content 242b-<NUM> and 242b-<NUM> based on the second set of spatial characteristics for the second subset 232b of the second physical setting <NUM>. As shown in <FIG>, the electronic device <NUM> presents the adapted 242b-<NUM> and 242b-<NUM> within the second subset 232b of the second physical setting <NUM>. However, compared to the first ER presentation scenario 200a, the electronic device <NUM> removes corresponding adapted ER content 242a-<NUM> and 242a-<NUM> (e.g., shown in the second subset 232b of the first physical setting <NUM>) when generating the adapted second ER content in the second subset 232b due to the volumetric size of the second subset 232b and the indication of objects (e.g., the sofa <NUM>, the chair 220a, and the coffee table 220d) within the second subset 232b of the second physical setting <NUM>. Furthermore, compared to the first ER presentation scenario 200a, the adapted ER content 242b-<NUM> and 242b-<NUM> in the second subset 232b is also scaled down to a smaller size than the adapted ER content 242a-<NUM> and 242a-<NUM> in the second subset 232a due to the difference in dimensions between the second subset 232a of the first physical setting <NUM> and the second subset 232b of the second physical setting <NUM>. Accordingly, in the second ER presentation scenario 200b, the electronic device <NUM> presents both a smaller scaled-down version of second ER content 242b-<NUM> and 242b-<NUM> in the second subset 232b and foregoes presenting a version of the adapted ER content 242a-<NUM> and 242a-<NUM> in the second subset 232b.

<FIG> illustrates a fourth state 201d (e.g., associated with T4 or a fourth time period) of the first example ER presentation scenario 200a and the second ER presentation scenario 200b. In some implementations, the electronic device <NUM> and/or the controller generates one or more navigation options that allow a user to traverse between the first and the second subsets based on the first and second sets of spatial characteristics.

With reference to the first ER presentation scenario 200a in <FIG>, the electronic device <NUM> generates a navigation option that includes a first navigation path 217a in order to allow the user <NUM> to traverse between the first subset 230a (not shown) and the second subset 232a (not shown) of the first physical setting <NUM>. In some implementations, the user <NUM> navigates between the first and second subsets by physically walking from the first subset 230a to the second subset 232a along the first navigation path 217a. The electronic device <NUM> determines that the first navigation path 217a corresponds to a long-winding path due to the ample space and lack of obj ects in the first physical setting <NUM>. As shown in <FIG>, the first navigation path 217a weaves between first adapted ER content 240a-<NUM>, 240a-<NUM>, and 240a-<NUM> in the first subset 230a and between second adapted ER content 242a-<NUM>, 242a-<NUM>, 242a-<NUM>, and 242a-<NUM> in the second subset 232a.

With reference to the second ER presentation scenario 200b in <FIG>, the electronic device <NUM> generates a navigation option that includes a second navigation path 217b in order to allow the user <NUM> to traverse between the first subset 230b (not shown) and the second subset 232b (not shown) of the second physical setting <NUM>. However, in contrast to the first ER presentation scenario 200a, the electronic device <NUM> determines that the second navigation path 217b corresponds to a shorter circular navigation path rather than the long-winding path due to the smaller dimensions of the second physical setting <NUM> and the indication of objects (e.g., chairs 220a, 220c, credenza 220b, coffee tables 220d, 220e, and a sofa <NUM>) within the second physical setting <NUM>. As shown in <FIG>, the second navigation path 217b circles around the adapted first ER content 240b-<NUM>, 240b-<NUM>, and 240b-<NUM> in the first subset 230b and the adapted second ER content 242b-<NUM>, 242b-<NUM> in the second subset 232b.

<FIG> illustrate an example ER presentation scenario <NUM> sequence for generating adapted ER content based on spatial characteristics of a plurality of subsets within a physical setting in accordance with some implementations. While pertinent features are shown, those of ordinary skill in art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity and so as not to obscure more pertinent aspects of the example implementations disclosed herein. The sequence shown in <FIG> depicts the electronic device <NUM> generating adapted ER content associated with a temporally linear plot based on a set of spatial characteristics for each room within a house <NUM>.

<FIG> illustrates a first state 301a (e.g., associated with T1 or a first time period) of the ER presentation scenario <NUM>. However, before the first state 301a, the electronic device <NUM> obtains, via the exterior-facing image sensor of the electronic device <NUM>, image data that corresponds to the house <NUM> in order to identify a plurality of subsets (e.g., rooms) within the house <NUM> and determine the sets of spatial characteristics for the plurality of subsets based at least in part on the image data. In some implementations, the electronic device <NUM> uses sensor information from one or more remote input devices (e.g., the optional remote input devices) within each room in order to collect/capture input data with informed user consent and provide the input data to the electronic device <NUM> and/or the controller <NUM>. In some implementations, the electronic device <NUM> may extract sets of spatial characteristics for a plurality of subsets based at least in part on obtaining data from depth sensors associated with the electronic device <NUM>. In some implementations, the electronic device <NUM> and/or the controller <NUM> obtains a set of known spatial characteristics for the plurality of subsets from an external database or library. Those of ordinary skill in the art will appreciate that there are many methods of obtaining image data that corresponds to the physical setting or obtaining a set of known spatial characteristics from an external database. For the sake of brevity, an exhaustive listing of all such methods is not provided therein.

In some implementations, a first ER content portion corresponds to a first thematic scene within predetermined content, and a second ER content portion corresponds to a second thematic scene within the predetermined content. In some implementations, the predetermined content corresponds to ER content associated with a real-life event, story, movie, TV episode, or the like. For example, as shown in <FIG>, the ER content portions are associated with thematic scenes within the predetermined content corresponding to a story with five parts. Therefore, the predetermined content is divided into five ER content portions that are linked together and presented to the user <NUM> in a temporally ordered sequence in order to deliver a linear story to the user <NUM>. As such, in <FIG>, the predetermined content corresponds to a temporally linear plot with a beginning, a rising action, a climax, a falling action, and a resolution. To that end, the electronic device <NUM> presents an adapted first ER content (e.g., the beginning) before presenting an adapted second ER content portion (e.g., the rising action) before presenting an adapted third ER content portion (e.g., the climax) and so forth.

As shown in <FIG>, the house <NUM> includes at least a basement <NUM>, a kitchen <NUM>, a bedroom <NUM>, and a living room <NUM>. In the first state 301a, the user <NUM> wearing the electronic device <NUM> is located in the basement <NUM>. As mentioned earlier, prior to time T1 in <FIG>, the electronic device <NUM> identifies each room as a subset of the house <NUM>. In addition, the electronic device <NUM> also obtains ER content portions from the predetermined content to present to the user <NUM> within a user interface of the electronic device <NUM>. In some implementations, the electronic device <NUM> obtains the predetermined content from an ER library or database.

In some implementations, the electronic device <NUM> may determine that a particular ER content portion should be presented in a particular room of the house <NUM> based on the set of spatial characteristics characterizing one or more dimensions of the particular room. As an example, the first ER content portion may be better suited for large rooms because the first ER content portion includes the most ER content as compared to the other ER content portions. In turn, the electronic device <NUM> logically maps the first ER content portion to the basement <NUM> rather than the other rooms in the house <NUM> because the basement <NUM> has the largest volumetric size among the rooms within the house <NUM>. As another example, the second ER content portion may be better suited for the smallest room in the house <NUM> because the second ER content portion includes less ER content as compared to the other ER content portions. As a result, the electronic device <NUM> logically maps the second ER content portion to the kitchen <NUM> because the kitchen <NUM> is the smallest room in the house <NUM>.

In some implementations, after the electronic device <NUM> logically maps the ER content portions to the plurality of subsets, the electronic device <NUM> generates one or more navigation options that allow the user <NUM> to traverse between the subsets (e.g., the basement <NUM>, the kitchen <NUM>, the bedroom <NUM>, and the living room <NUM>) of the house <NUM> in an ordered sequence. In some implementations, the user <NUM> navigates between the subsets by physically walking between rooms of the house <NUM>. To that end, the electronic device <NUM> determines a navigation path <NUM> that links the rooms with ER content portions to be presented to the user <NUM> in a temporally-ordered sequence (e.g., the adapted first ER content portion <NUM> is presented before the adapted second ER content portion <NUM> and so on). In the example of <FIG>, the electronic device <NUM> determines that the five ER content portions should be presented in a manner such that the user traverses from the bottom of the house <NUM> to the second story of the house <NUM> along the navigation path <NUM>. As will be explained in more detail below in <FIG> and <FIG>, since there are five ER content portions from the predetermined content to present to the user, but only four subsets associated with the house <NUM>, the electronic device <NUM> presents the adapted fourth ER content portion <NUM> and the adapted fifth ER content portion <NUM> in the living room <NUM>. Thus, the electronic device <NUM> begins the story by presenting the adapted first ER content portion (e.g., the beginning) to the user <NUM> in the basement <NUM> and completes the story by presenting the adapted fifth ER content portion (e.g., the resolution) to the user in the living room <NUM>.

<FIG> illustrates a second state 301b (e.g., associated with T2 or a second time period) of the ER presentation scenario <NUM>. In the second state 301b, at least a portion of the basement <NUM> is within a field-of-view 311a of the electronic device <NUM>. As shown in <FIG>, the basement <NUM> includes a set of stairs. In other words, while holding or wearing the electronic device <NUM>, the user is looking at the set of stairs <NUM> from a side or perspective orientation through the electronic device <NUM>. In some implementations, after logically mapping the first ER content portion to the basement <NUM>, the electronic device <NUM> generates an adapted first ER content portion <NUM> based at least in part on the first set of spatial characteristics of the basement <NUM>. As shown in <FIG>, in response to determining that the first presentation criterion is satisfied, the electronic device <NUM> presents, on the display <NUM>, a user interface <NUM> including the set of stairs <NUM>, the navigation path <NUM> (optional), and the adapted first ER content portion <NUM>. Here, the first presentation criterion may correspond to the location of the electronic device <NUM>. At time T3 shown in <FIG>, the electronic device <NUM> moves to the second subset (e.g., the kitchen <NUM>) by following the navigation path <NUM> in order to satisfy a second presentation criterion (e.g., a location of the electronic device <NUM>) for displaying the adapted second ER content portion <NUM>.

<FIG> illustrates a third state 301c (e.g., associated with T3 or a third time period) of the ER presentation scenario <NUM>. In comparison to <FIG>, the field-of-view of the electronic device <NUM> changes due to translational movement of the electronic device <NUM> from the basement <NUM> (e.g., the first subset of the house <NUM>) to the kitchen <NUM> (e.g., second subset of the house <NUM>). As shown in <FIG>, in the third state 301c, a field-of-view 311b of the electronic device <NUM> of the kitchen <NUM> includes a partial view of the dining table <NUM> and a refrigerator <NUM>. In some implementations, after logically mapping the second ER content portion to the kitchen <NUM>, the electronic device <NUM> generates an adapted second ER content portion <NUM> based at least in part on the second set of spatial characteristics of the kitchen <NUM>. As shown in <FIG>, in response to determining that the second presentation criterion is satisfied, the electronic device <NUM> presents, on the display <NUM>, the user interface <NUM> including a partial view of the dining table <NUM>, the refrigerator <NUM>, the navigation path <NUM> (optional), and the adapted second ER content portion <NUM>. At time T4 shown in <FIG>, the electronic device <NUM> moves to the third subset (e.g., the bedroom <NUM>) by following the navigation path <NUM> in order to satisfy a third presentation criterion for displaying the adapted third ER content portion <NUM>.

<FIG> illustrates a fourth state 301d (e.g., associated with T4 or a fourth time period) of the ER presentation scenario <NUM>. In comparison to <FIG>, the field-of-view of the electronic device <NUM> changes due to translational movement of the electronic device <NUM> from the kitchen <NUM> (e.g., the second subset of the house <NUM>) to the bedroom <NUM> (e.g., the third subset of the house <NUM>). As shown in <FIG>, in the fourth state 301d, a field-of-view 311c of the electronic device <NUM> of the bedroom <NUM> includes a partial view of a bed <NUM>. In some implementations, after logically mapping the third ER content portion to the bedroom <NUM>, the electronic device <NUM> generates an adapted third ER content portion <NUM> based at least in part on the third set of spatial characteristics of the bedroom <NUM>. As shown in <FIG>, in response to the determining that the third presentation criterion is satisfied, the electronic device <NUM> presents, on the display <NUM>, the user interface <NUM> including the partial view of the bed <NUM>, the navigation path <NUM> (optional), and the adapted third ER content portion <NUM>. At time T5 shown in <FIG>, the electronic device <NUM> moves to the fourth subset (e.g., the living room <NUM>) by following the navigation path <NUM> in order to satisfy the fourth presentation criterion for displaying the adapted fourth ER content portion <NUM>.

<FIG> illustrates a fifth state 301e (e.g., associated with T5 or a fifth time period) of the ER presentation scenario <NUM>. In comparison to <FIG>, the field-of-view of the electronic device <NUM> changes due to translational movement of the electronic device <NUM> from the bedroom <NUM> (e.g., the third subset of the house <NUM>) to the living room <NUM> (e.g., the fourth subset of the house <NUM>) associated with the house <NUM>. As shown in <FIG>, in the fifth state 301e, a field-of-view 311d of the electronic device <NUM> includes a sofa <NUM>. In some implementations, after logically mapping the fourth ER content portion to the living room <NUM>, the electronic device <NUM> generates an adapted fourth ER content portion <NUM> based at least in part on the fourth set of spatial characteristics of the living room <NUM>. As shown in <FIG>, in response to determining that the fourth presentation criterion is satisfied, the electronic device <NUM> presents, on the display <NUM>, the user interface <NUM> including the sofa <NUM>, the navigation path <NUM> (optional), and the adapted fourth ER content portion <NUM> for a temporal parameter.

In contrast to the first, second, and third presentation criteria in <FIG>, the fourth presentation criteria and the fifth presentation criterion correspond to a temporal criterion associated with predetermined content and a coordinate-based or location-based criterion. The temporal criterion is associated with the adapted fourth and fifth ER content portions <NUM>, <NUM> because the electronic device <NUM> presents both the adapted fourth ER content portion <NUM> and the adapted fifth ER content portion <NUM> in the fourth subset (e.g., the living room <NUM>). As such, the electronic device <NUM> presents the adapted fourth ER content portion <NUM> based at least in part on a temporal criterion (e.g., a playback time "<NUM>:<NUM>" to "<NUM>:<NUM>") before presenting the adapted fifth ER content portion <NUM>. In this example, provided for reference, the ER presentation scenario <NUM> depicts a playback time <NUM> of "<NUM>:<NUM>" and the electronic device <NUM> presents the adapted fourth ER content portion <NUM> until "<NUM>:<NUM>. " In some implementations, the user <NUM> may navigate between different ER content portions in the same subset based at least in part on a physical motion such as turning his or her head toward a specific area within the physical setting.

<FIG> illustrates a sixth state 301f (e.g., associated with T6 or a sixth time period) of the ER presentation scenario <NUM>. In comparison to <FIG>, the field-of-view 311d of the electronic device <NUM> shown in <FIG> is the same as in <FIG> due to the electronic device <NUM> being stationary. However, in contrast to <FIG>, at time T6 in <FIG>, time elapses such that the playback time <NUM> is now "<NUM>:<NUM>. " In some implementations, after logically mapping the fifth ER content portion to the living room <NUM>, the electronic device <NUM> generates an adapted fifth ER content portion <NUM> based at least in part on the fifth set of spatial characteristics of the living room <NUM>. Accordingly, at time T6 in <FIG>, the fifth presentation criterion is satisfied because the adapted fourth ER content portion <NUM> surpasses its temporal time limit. As shown at time T6 in <FIG>, in response to determining that the fifth presentation criterion (e.g., the temporal criterion) is satisfied, the electronic device <NUM> presents, on the display <NUM>, the user interface <NUM> including the sofa <NUM> and the adapted fifth ER content portion <NUM>.

In some implementations, the predetermined content may be non-linear such that it does not matter what order the electronic device <NUM> presents the ER content portions from the predetermined content. In some implementations, the electronic device <NUM> obtains a first set of environmental characteristics associated with the first subset and a second set of environmental characteristics associated with the second subset, wherein generating the adapted first ER content portion includes logically mapping the first ER content portion to the first subset based at least in part on the first set of spatial characteristics of the first subset and the first set of environmental characteristics associated with the first subset and generating the adapted second ER content portion includes logically mapping the second ER content portion to the second subset based at least in part on the second set of spatial characteristics of the second subset and the second set of environmental characteristics associated with the second subset. In some implementations, adapting the ER content portions may be based on environmental characteristics such as a type of room, temperature information, lighting information, objects within the physical setting, a time of day, background color of the physical setting, and/or the like. Those of ordinary skill in the art will appreciate that there are many different types of environmental characteristics. For the sake of brevity, an exhaustive listing of all such types is not provided herein.

As a non-limiting example, the electronic device <NUM> may be configured to present portions of predetermined content corresponding to different ecosystems based on the environmental characteristics of the subset associated with the physical setting. Continuing with the previous non-limiting example, a first portion of predetermined content may correspond to ER content associated with a subterranean ecosystem, a second portion of predetermined content may correspond to ER content associated with a grassland ecosystem, a third portion of predetermined content may correspond to ER content associated with a forest ecosystem, and a fourth portion of predetermined content may correspond to ER content associated with a desert ecosystem.

With reference to <FIG>, for example, the electronic device <NUM> obtains environmental characteristics associated with the different rooms within the house <NUM> in order to determine which rooms satisfy the mapping criteria for the portions of predetermined content. For example, the first portion of the predetermined content associated with the subterranean ecosystem includes a set of mapping criteria for a subset that is located underground, having the coolest temperature conditions in the plurality of subsets, and the darkest lighting conditions in the plurality of subsets and first location-based presentation criterion corresponding to a location of the electronic device <NUM>. As such, the electronic device <NUM> logically maps the first portion of predetermined content associated with the subterranean ecosystem to the basement <NUM> because the basement <NUM> satisfies the set of mapping criteria of being located underground, and having the coolest temperature and the darkest lighting conditions compared to the rest of the rooms in the house <NUM>.

As another example, the second portion of the predetermined content associated with the grassland ecosystem includes a set of mapping criteria for a subset that has hot temperature conditions and has the brightest lighting conditions in the plurality of subsets and a second location-based presentation criterion corresponding to the location of the electronic device <NUM>. As such, the electronic device <NUM> logically maps the second portion of the predetermined content associated with the grassland ecosystem to the kitchen <NUM> because the kitchen <NUM> satisfies the set of mapping criteria of having the brightest lighting condition compared to the rest of the rooms in the house <NUM>.

As yet another example, the third portion of the predetermined content associated with the forest ecosystem includes a set of mapping criteria for a subset that is located on a second floor and a third location-based presentation criterion corresponding to the location of the electronic device <NUM>. As such, the electronic device <NUM> logically maps the third portion of the predetermined content associated with the forest ecosystem to the bedroom <NUM> because the bedroom <NUM> satisfies the mapping criterion of being located on the second story of the house <NUM>.

As another example, the fourth portion of the predetermined content associated with the desert ecosystem includes a set of mapping criteria for a subset that has the hottest temperature in the plurality of subsets and a fourth location-based presentation criterion corresponding to the location of the electronic device <NUM>. As such, the electronic device <NUM> logically maps the fourth portion of the predetermined content associated with the desert ecosystem to the living room <NUM> because the living room <NUM> has the hottest temperature compared to the rest of the rooms in the house <NUM>.

Continuing with the above example, at time T2 shown in <FIG>, the electronic device <NUM> is located in the basement. As such, in response to determining that the electronic device <NUM> satisfies the first location-based presentation criterion, the electronic device <NUM> displays the adapted first portion of the predetermined content associated with the subterranean ecosystem overlaid on the first field-of-view 311a of the electronic device <NUM> corresponding to the basement <NUM>. At time T3 shown in <FIG>, the electronic device <NUM> is located in the kitchen <NUM>. As such, in response to determining that the electronic device <NUM> satisfies the second location-based presentation criterion, the electronic device <NUM> displays the adapted second portion of the predetermined content associated with the grassland ecosystem overlaid on the second field-of-view 311b of the electronic device <NUM> corresponding to the kitchen <NUM>. At time T4 shown in <FIG>, the electronic device <NUM> is located in the bedroom <NUM>. As such, in response to determining that the electronic device <NUM> satisfies the third location-based presentation criterion, the electronic device <NUM> displays the adapted third portion of the predetermined content associated with the forest ecosystem overlaid on the third field-of-view 311c of the electronic device <NUM> corresponding to the bedroom <NUM>. At time T5 shown in <FIG>, the electronic device <NUM> is located in the living room <NUM>. As such, in response to determining that the electronic device <NUM> satisfies the fourth location-based presentation criterion, the electronic device <NUM> displays the adapted fourth portion of the predetermined content associated with the desert ecosystem overlaid on the fourth field-of-view 311d of the electronic device <NUM> corresponding to the living room <NUM>.

<FIG> illustrate another ER presentation scenario sequence <NUM> for generating adapted ER content based on spatial characteristics of a plurality of subsets within a physical setting in accordance with some implementations. While pertinent features are shown, those of ordinary skill in art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity and so as not to obscure more pertinent aspects of the example implementations disclosed herein.

<FIG> illustrates a first state 401a (e.g., associated with T1 or a first time period) of an example ER presentation scenario <NUM>. In the first state 401a, at least a portion of the physical setting is within a field-of-view 411a of the electronic device <NUM>. In some implementations, the electronic device <NUM> determines a connectivity matrix between a plurality of subsets within the physical setting and generates additional ER content in order to present the additional ER content based at least in part on the connectivity matrix between the plurality of subsets within the physical setting. In this example, the hallway <NUM> is not part of the plurality of subsets within the physical setting, but rather corresponds to a space between the plurality of subsets.

As shown in <FIG>, the hallway <NUM> includes a door that leads to a room (e.g., the first subset of the physical setting). Therefore, the electronic device <NUM> generates additional ER content in order to bridge the plurality of subsets within the physical setting. In some implementations, the additional ER content corresponds to emergent ER content. In some implementations, the additional ER content corresponds to ER content selected from predetermined content. In some implementations, the additional ER content corresponds to emergent ER content that is generated based ER content portions logically mapped to the previous and subsequent subsets. In some implementations, the one or more navigation options are generated based at least in part on the connectivity matrix. Here, the electronic device <NUM> generates a navigation option that includes a navigation path <NUM> based at least in part on the hallway <NUM>. As shown in <FIG>, the electronic device <NUM> presents, on the display <NUM>, a user interface <NUM> including the hallway <NUM>, the navigation path <NUM> (optional), and the additional ER content 410a, 410b. At time T2 shown in <FIG>, the electronic device <NUM> moves to the first subset (e.g., the room <NUM>) by following the navigation path <NUM> in order to satisfy a presentation criterion for displaying adapted ER content.

<FIG> illustrates a second state 401b (e.g., associated with T2 or a second time period) of an example ER presentation scenario <NUM>. In comparison to <FIG>, the field-of-view of the electronic device <NUM> changes due to translation movement of the electronic device <NUM> from the hallway <NUM> to inside the room <NUM>. As shown in <FIG>, in the second state 401b, the field-of-view 411b of the electronic device <NUM> includes a painting <NUM> and the door <NUM>. In some implementations, after logically mapping the first ER content portion to the room <NUM>, the electronic device <NUM> generates adapted ER content (e.g., the virtual agent <NUM>) based at least in part on the first set of spatial characteristics of the room <NUM>. In this example, the first presentation criterion associated with the adapted ER content is location-based. As such, in response to determining that the first presentation criterion is satisfied, the electronic device <NUM> presents, on the display <NUM>, the user interface <NUM> including the painting <NUM>, the door <NUM>, and the virtual agent <NUM> appearing to stand in the room <NUM>.

According to the invention, the electronic device <NUM> presents one or more ER content items associated with the ER content moving from the first subset to the second subset. The sequence shown in <FIG> and <FIG> depicts a virtual agent <NUM> associated with the ER content moving from the room <NUM> to another subset within the physical setting while the field-of-view 411b of the electronic device <NUM> is stationary.

<FIG> illustrates a third state 401c (e.g., associated with T3 or a third time period) of an example ER presentation scenario <NUM>. In comparison to <FIG>, the field-of-view 411b of the electronic device is the same, but, at time T3 in <FIG>, the virtual agent <NUM> appears to move from the room <NUM> (e.g., the first subset) toward another room beyond the door <NUM> within the physical setting. Accordingly, the electronic device <NUM> generates a path indicator <NUM> that allows the user to follow the virtual agent <NUM> to the other room beyond the door <NUM> within the physical setting. In some implementations, following the path indicator <NUM> enables the user to traverse between the room <NUM> (e.g., the first subset) and the other room beyond the door <NUM> (e.g., second subset within the physical setting). As such, in <FIG>, the electronic device <NUM> presents, on the display <NUM>, the user interface <NUM> including the painting <NUM>, the door <NUM>, the virtual agent <NUM> moving from the room <NUM> to a second subset, and the path indicator <NUM> (optional).

<FIG> is a flowchart representation of a method <NUM> of generating adapted ER content based on spatial characteristics of a plurality of subsets within a physical setting in accordance with some implementations. In various implementations, the method <NUM> is performed at an electronic device (e.g., the electronic device <NUM> shown in <FIG> and <FIG>, the controller <NUM> shown in <FIG> and <FIG>, or a suitable combination thereof) with an image sensor, one or more processors, a non-transitory memory, and a display. In some implementations, the method <NUM> is performed by processing logic, including hardware, firmware, software, or a combination thereof. In some implementations, the method <NUM> is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory).

As represented by block <NUM>, the method <NUM> includes identifying a plurality of subsets associated with a physical setting (e.g., the physical setting <NUM> shown in <FIG> or the first and second physical settings <NUM>, <NUM> shown in <FIG>). In some implementations, the plurality of subsets may correspond to portions of a room associated with the physical setting. For example, as shown in <FIG>, the plurality of subsets corresponds to portions of a single room within a first physical setting <NUM> and a second physical setting <NUM>. In some implementations, the plurality of subsets may correspond to rooms within the physical setting. For example, as shown in <FIG>, the plurality of subsets corresponds to individual rooms (e.g., the basement <NUM>, the kitchen <NUM>, the bedroom <NUM>, and the living room <NUM>) associated with the house <NUM>.

In some implementations, the electronic device determines the plurality of subsets based on physical divisions. For example, with reference to <FIG>, if the physical setting corresponds to a house <NUM>, then the electronic device <NUM> identifies the plurality of subsets based on each individual room (e.g., the basement <NUM>, the kitchen <NUM>, the bedroom <NUM>, and the living room <NUM>). As another example, if the physical setting corresponds to a single room, then the electronic device <NUM> identifies the plurality of subsets based on the different walls of the single room. As yet another example, if the physical setting corresponds to a single room, then the electronic device <NUM> identifies the plurality of subsets based on the corners of a room.

In some implementations, the electronic device <NUM> determines the plurality of subsets based on a metric that is not associated with physical divisions. In some implementations, the electronic device determines the plurality of subsets based on a metric that is not associated with physical divisions. For example, if a physical setting does not include physical divisions, then the electronic device <NUM> identifies the plurality of subsets by dividing the physical setting in half and identifying a first half of the physical setting as a first subset and the second half of the physical setting as a second subset. As an example, with reference to <FIG>, in the first ER presentation scenario 200a, the electronic device <NUM> divides the first physical setting <NUM> in half and identifies the first half of the first physical setting <NUM> as a first subset 230a and a second subset 232a. As another example, also with reference to <FIG>, in the second ER presentation scenario 200b, the electronic device <NUM> divides the second physical setting <NUM> in half and identifies the first half of the second physical setting <NUM> as a first subset 230b and the second half of the second physical setting <NUM> as a second subset 232b.

As represented by block <NUM>, the method <NUM> includes determining a set of spatial characteristics for each of the plurality of subsets within the physical setting, wherein a first set of spatial characteristics characterizes one or more dimensions of a first subset of the plurality of subsets and a second set of spatial characteristics characterizes one or more dimensions of a second subset of the plurality of subset. In some implementations, the first set of spatial characteristics include at least one of a volumetric size of the first subset of the physical setting, an indication of physical objects within the first subset of the physical setting, a shape of the first subset of the physical setting, or the like. In some implementations, the one or more dimensions may correspond to a width, length, height, or the like of a subset of the plurality of subsets.

For example, as shown in the first ER presentation scenario 200a in <FIG>, a first set of spatial characteristics for the first subset 230a include a volumetric size of the first subset 230a based on the x-dimension 207a, the y-dimension 209a, and the z-dimension 216a within the first physical setting <NUM> and the indication of no physical objects within the first physical setting <NUM>; and a second set of spatial characteristics for the second subset 232a include the volumetric size of the second subset 232a based on the x-dimension 211a, the y-dimension 213a, the z-dimension 215a, and the indication of no physical objects within the first physical setting <NUM>.

In another example, as shown in the second ER presentation scenario 200b in <FIG>, a first set of spatial characteristics for the first subset 230b include a volumetric size of the first subset 230b based on the x-dimension 207b, the y-dimension 209b, the z-dimension 216b within the second physical setting <NUM> and the indication of physical objects (e.g., the chair 220c, the credenza 220b, the coffee table 220e, and the sofa <NUM>) within the second physical setting <NUM>; and a second set of spatial characteristics for the second subset 232b include the volumetric size of the second subset 232b based on the x-dimension 211b, the y-dimension 213b, the z-dimension (215b), and the indication of physical objects (e.g., the chair 220a, the credenza 220b, the coffee tables 220d, 220e, and the sofa <NUM>) within the second physical setting <NUM>. In some implementations, the spatial characteristics may include a point cloud for a physical setting that is labeled with obj ects. In some implementations, the electronic device determines the set of spatial characteristics by performing semantic segmentation or instance segmentation. In some implementations, semantic segmentation corresponds to detecting and labeling objects that appear within image data. In some implementations, instance segmentation corresponds to detecting and delineating distinct objects that appear within the image data.

As represented by block <NUM>, the method <NUM> includes generating an adapted first enhanced reality (ER) content portion for the first subset of the plurality of subsets based at least in part on the first set of spatial characteristics. Specifically, in some implementations, the electronic device <NUM> logically maps a first portion of ER content to a first subset that satisfies a first mapping criterion associated with the first portion of ER content. Next, in some implementations, the electronic device <NUM> generates the adapted first ER content portion by adapting reference ER content (e.g., the first portion of ER content) based on the spatial characteristics of the first subset. Finally, in some implementations, in response to determining that a presentation criterion associated with the adapted first ER content portion is satisfied, the electronic device <NUM> presents the adapted first ER content portion within the first subset.

As an example, in the first ER presentation scenario 200a shown in <FIG> and <FIG>, after logically mapping the first ER content portion to the first subset 230a, the electronic device <NUM> generates the adapted first ER content portions 240a-<NUM>, 240a-<NUM>, 240a-<NUM> based at least in part on the x-dimension 207a, the y-dimension 209a, and the z-dimension 216a of the first subset 230a within the first physical setting <NUM>. Similarly, also shown in the second ER presentation scenario 200b shown in <FIG> and <FIG>, after logically mapping the first ER content portion to the first subset 230b, the electronic device <NUM> generates the adapted first ER content portions 240b-<NUM>, 240b-<NUM>, 240b-<NUM> based at least in part on the x-dimension 207b, the y-dimension 209b, and the z-dimension 216b of the first subset 230b within the second physical setting <NUM>.

In some implementations, the ER content corresponds to emergent ER content or predetermined ER content. In some implementations, the method <NUM> further includes generating emergent ER content including the first ER content portion and the second ER content portion. In some implementations, the ER content corresponds to stereoscopic models of objects, characters, or scenery that is associated with emergent content. For example, emergent content may correspond to one or more objective effectuators carrying out actions in order to achieve a particular objective. According to the invention, as shown in <FIG>, the emergent content corresponds to the virtual agent <NUM> (e.g., the objective effectuator) carrying out an action of searching for paintings within the physical setting. As another example, as shown in <FIG>, the electronic device <NUM> generates emergent ER content in order to bridge the plurality of subsets within the physical setting.

In some implementations, the method <NUM> further includes obtaining predetermined content that includes the first ER content portion and the second ER content portion, wherein the first ER content portion and the second ER content portion are obtained from a database or library. In some implementations, the ER content corresponds to stereoscopic models of objects, characters or scenery that is associated with predetermined content. In some implementations, the predetermined content corresponds to content associated with a real-life event, story, movie, television episode, or the like. In some implementations, the first ER content portion is associated with a first thematic scene within predetermined content and the second ER content portion is associated with a second thematic scene within the predetermined content. In some implementations, the predetermined content corresponds to a temporally linear plot, and the adapted first ER content portion is presented before presenting the adapted second ER content portion. For example, as shown in <FIG>, the adapted first ER content portion <NUM> corresponds to a beginning, the adapted second ER content portion <NUM> corresponds to a rising action, the adapted third ER content portion <NUM> corresponds to a climax, the adapted fourth ER content portion <NUM> corresponds to a falling action, and the adapted fifth ER content portion <NUM> corresponds to a resolution are associated with a linear plot such that the adapted ER content portions are linked in an ordered manner. As such, continuing with the example shown in <FIG>, the electronic device <NUM> presents the adapted first ER content portion <NUM>, the adapted second ER content portion <NUM>, the adapted third ER content portion <NUM>, the adapted fourth ER content portion <NUM>, and the adapted fifth ER content portion <NUM> in an ordered sequence in order for the story or plot to make sense.

In some implementations, the method <NUM> further includes generating an adapted first ER content portion based at least in part on determining whether the first set of spatial characteristics associated with the first subset satisfies a mapping criterion for the first ER content portion of the predetermined content. In some implementations, the mapping criterion corresponds to spatial criteria for finding a best fit for the adapted first ER content portion. In some implementations, the method <NUM> further includes in response to determining that the first set of spatial characteristics associated with the first subset satisfies the first mapping criterion, determining a placement for the adapted first ER content portion within the first subset based at least in part on the first set of spatial characteristics of the first subset, wherein presenting an adapted first ER content portion composited with a first pass-through image data includes presenting the adapted ER content in accordance with the placement. For example, as shown in <FIG>, the electronic device <NUM> generates an adapted first ER content portion <NUM> based at least in part on determining that the dimensions associated with the basement <NUM> satisfies the first mapping criterion of being the largest room in the house <NUM> for the first ER content portion of predetermined content. As such, continuing with the example in <FIG>, in response to determining that the first set of spatial characteristics associated with the basement <NUM> satisfies the first mapping criterion of being the largest room in the house <NUM>, the electronic device <NUM> determines a placement for the adapted first ER content portion <NUM> within the basement <NUM> by placing the adapted first ER content portion <NUM> next to the set of stairs <NUM>. Finally, continuing with the example in <FIG>, the electronic device <NUM> presents, on the display <NUM>, the user interface <NUM> including the set of stairs <NUM> and the adapted first ER content portion <NUM> composited with a first pass-through image data in accordance with the placement.

As represented by block <NUM>, the method <NUM> includes generating an adapted second ER content portion for the second subset of the plurality of subsets based at least in part on the second set of spatial characteristics. Specifically, in some implementations, the electronic device <NUM> logically maps a second portion of ER content to a second subset that satisfies a second mapping criterion associated with the second portion of ER content. Next, in some implementations, the electronic device <NUM> generates the adapted second ER content portion by adapting reference ER content (e.g., the second portion of ER content) based on the spatial characteristics of the second subset. Finally, in some implementations, in response to determining that a presentation criterion associated with the adapted second ER content portion is satisfied, the electronic device <NUM> presents the adapted second ER content portion within the second subset.

As an example, in the first ER presentation scenario 200a shown in <FIG> and <FIG>, after logically mapping the adapted second ER content portion to the second subset 232a, the electronic device <NUM> generates the adapted second ER content portions 242a-<NUM>, 242a-<NUM>, 242a-<NUM>, and <NUM>-<NUM> based at least in part on the x-dimension 211a, the y-dimension 213a, and the z-dimension 215a of the second subset 232a within the first physical setting <NUM>. Similarly, also shown in the second ER presentation scenario 200b in <FIG> and <FIG>, after logically mapping the second ER content portion to the second subset 232b, the electronic device <NUM> generates the adapted second ER content portions 242b-<NUM>, 242b-<NUM> based at least in part on the x-dimension <NUM>1b, the y-dimension 213b, and the z-dimension 215b of the second subset 232b within the second physical setting <NUM>.

As represented by block <NUM>, the method <NUM> includes generating one or more navigation options that allow a user to traverse between the first and the second subsets based on the first and second sets of spatial characteristics. In some implementations, the navigation options include at least one of a path indicator between the first and the second subsets, a circular path between the first and the second subsets, and a determination of a navigation path between the first and the second subsets. For example, as shown in the first ER presentation scenario 200a in <FIG>, the electronic device <NUM> generates a first navigation path 217a in the first physical setting <NUM> in order to traverse between the first subset 230a to the second subset 232a within the first physical setting <NUM>. As another example, as shown in the second ER presentation scenario 200b in <FIG>, the electronic device <NUM> generates a second navigation path 217b corresponding to a circular path in the second physical setting <NUM> in order to traverse between the first subset 230b to the second subset 232b within the second physical setting <NUM>. As yet another example, as shown in <FIG>, the electronic device <NUM> generates a navigation path <NUM> in order to allow the user to traverse between a plurality of subsets (e.g., a basement <NUM>, a kitchen <NUM>, a bedroom <NUM>, and a living room <NUM>) within the house <NUM>. Finally, as another example, as shown in <FIG>, the electronic device <NUM> generates a path indicator <NUM> in order to allow the user to traverse to a subsequent subset within the physical setting. In some implementations, the navigation option includes detecting a user input such as a physical movement from the user or a head rotation of the user. In some implementations, the navigation option includes a user navigating between subsets by turning his head or physically walking to a subsequent subset.

In some implementations, the method <NUM> further includes in response to determining that first presentation criterion is satisfied, presenting, on the display, the adapted first ER content portion overlaid on a first field-of-view of the device that corresponds to the first subset of the physical setting; and in response to determining that second presentation criterion is satisfied, presenting, on the display, the adapted second ER content portion overlaid on a second field-of-view of the device that corresponds to the second subset within the physical setting. In some implementations, the presentation criterion may correspond to at least one of a temporal criterion associated with the adapted first ER content portion, coordinates of the first subset, coordinates of the second subset, adjacency between the first ER content portion and the second ER content portion, and a location of the electronic device. For example, the presentation criterion may correspond to an adjacency between the first ER content portion and the second ER content portion if the first subset mapped to the first ER content portion and the second subset mapped to the second ER content portion are located in close proximity to each other. Continuing with the example, the adjacency presentation criterion enables the electronic device <NUM> to display a transition between presenting the first ER content portion and the second ER content portion that is practically immediate and does not include a prolonged delay between presenting the first ER content portion and the second ER content portion. As another non-limiting example, the presentation criterion may correspond to a temporal time limit if there is more than one subset within a single room, or if there are not enough rooms to present all ER content. For example, as shown in <FIG> and <FIG>, the electronic device <NUM> determines whether a presentation criterion (e.g., temporal parameter associated with the predetermined content) is satisfied by checking the current time before transitioning between presenting the adapted fourth ER content portion <NUM> and the adapted fifth ER content portion <NUM>. Continuing with the example shown in <FIG> and <FIG>, after the time period associated with the adapted fourth ER content portion <NUM> elapses, the electronic device <NUM> presents the adapted fifth ER content portion <NUM> in the living room <NUM>.

As a non-limiting example, the first presentation criterion may be satisfied based at least in part on when the electronic device or user is proximate to the first subset. As an example, in <FIG>, the presentation criterion is satisfied when the electronic device <NUM> is proximate to the first subset (e.g., the basement <NUM>). As another example, in <FIG>, the presentation criterion is satisfied when the electronic device <NUM> is located within a first subset (e.g., the room <NUM>). In some implementations, the method <NUM> further includes presenting, on the display, one or more ER content items associated with the first ER content portion moving from the first subset to the second subset. For example, as shown in <FIG> and <FIG>, the virtual agent <NUM> is initially presented in a first subset (e.g., the room <NUM>) at time T2 in <FIG> and then moves from the first subset to a second subset at time T3 in <FIG>.

In some implementations, a first ER content portion includes a default appearance from which the adaption takes place. In some implementations, generating adapted ER content includes at least one of adding, removing, scaling, or modifying a set of available interactions associated with the ER content to the spatial characteristics of the plurality of subsets.

In some implementations, the method <NUM> further includes adding one or more ER content items to the first ER content portion based at least in part on the first set of spatial characteristics of the first subset. For example, after adapting the first ER content portion to a first subset, if there is ample room left in the first subset, the electronic device may add one or more ER content items to the first ER content portion in order to fill out the first subset.

In some implementations, the method <NUM> further includes removing one or more ER content items from the first ER content portion based at least in part on the first set of spatial characteristics of the first subset. In some implementations, the ER content portion may be critical or non-critical such that the electronic device may decide to remove the non-critical ER content portions rather than the critical ER portions. In some implementations, the owner or the developer of the ER content determines what ER content portions are critical or non-critical. As an example, as shown in the first ER presentation scenario 200a in <FIG>, the electronic device <NUM> generates the adapted second ER content 242a-<NUM>, 242a-<NUM>, 242a-<NUM>, 242a-<NUM> in the second subset 232a of the first physical setting <NUM>. Similarly, as shown in the second ER presentation scenario 200b in <FIG>, the electronic device <NUM> also generates the adapted second ER content 242b-<NUM>, 242b-<NUM> in the second subset 232b of the second physical setting <NUM>, but the electronic device <NUM> removes the adapted ER content 242a-<NUM>, 242a-<NUM> in the second physical setting <NUM> due to the smaller dimensions of the second subset 232b and objects (e.g., chair 220a, credenza 220b, coffee tables 220d, 220e, and sofa <NUM>) within the second physical setting <NUM>.

In some implementations, the method <NUM> further includes scaling one or more ER content items associated with the first ER content portion based at least in part on the first set of spatial characteristics of the first subset. For example, as shown in the first ER presentation scenario 200a in <FIG>, the electronic device <NUM> generates adapted first ER content 240a-<NUM>, 240a-<NUM>, 240a-<NUM> within the first subset 230a in the first physical setting <NUM>. Similarly, as shown in the second ER presentation scenario 200b in <FIG>, the electronic device <NUM> also generates adapted first ER content 240b-<NUM>, 240b-<NUM>, 240b-<NUM> within the first subset 230b in the second physical setting <NUM>. However, compared to the first ER presentation scenario 200a, the adapted first ER content <NUM>-b1, 240b-<NUM>, 240b-<NUM> in the second ER presentation scenario 200b is scaled down to a smaller size than the adapted first ER content 240a-<NUM>, 240a-<NUM>, 240a-<NUM> in the first ER presentation scenario 200a due to the difference in dimensions between the first physical setting <NUM> and the second physical setting <NUM>.

In some implementations, the method <NUM> further includes modifying a set of available interactions associated with the first ER content portion based at least in part on the first set of spatial characteristics of the first subset. For example, as shown in the first ER presentation scenario 200a, the user <NUM> interacts with the adapted first ER content portions 240a-<NUM>, 240a-<NUM> by walking <NUM> degrees around the adapted first ER content portion 240a-<NUM>, 240a-<NUM> in order to see every angle of the adapted first ER content portions 240a-<NUM>, 240a-<NUM>. However, as shown in the second ER presentation scenario 200b, the user <NUM> may not be able to interact with the adapted first ER content portions 240b-<NUM>, 240b-<NUM> by walking <NUM> degrees around the adapted first ER content portions 240b-<NUM>, 240b-<NUM> in the same manner as the first ER presentation scenario 200a. Instead, in the second ER presentation scenario 200b, the user <NUM> is limited to walking around certain portions of the adapted first ER content portions 240b-<NUM>, 240b-<NUM> due to the smaller size of the second physical setting <NUM> and the plurality of objects within the second physical setting <NUM>.

In some implementations, the method <NUM> further includes identifying an object within the first subset that satisfies an object presentation criterion; and in response to determining that the object within the first subset satisfies the object presentation criterion, placing the adapted first ER content portion within the object identified in the first subset. In some implementations, the object presentation criterion corresponds to a volumetric or an access threshold. For example, the electronic device may identify a trunk within a living room that satisfies an object presentation criteria of being a specific volumetric size and an access threshold of being able to open at the top; and in response to determining that the trunk within the living room satisfies both the object presentation criteria of being the specific volumetric size and the access threshold of being able to open at the top, placing the adapted first ER content portion within the trunk identified in the living room.

In some implementations, the method <NUM> further includes determining a connectivity matrix between the plurality of subsets within the physical setting; and generating additional ER content in order to present the additional ER content based at least in part on the connectivity matrix between the plurality of subsets within the physical setting. In some implementations, the one or more navigation options are generated based at least in part on the connectivity matrix. For example, as shown in <FIG>, the electronic device determines a connectivity matrix between rooms within the physical setting and generates a navigation path <NUM> based at least in part on the connectivity matrix. Continuing with the example in <FIG>, the electronic device generates additional ER content 410a, 410b by presenting the additional ER content in the hallway <NUM> in order to bridge the plurality of subsets within the physical setting.

In some implementations, the method <NUM> further includes obtaining a first set of environmental characteristics associated with the first subset and a second set of environmental characteristics associated with the second subset, wherein: generating the adapted first ER content portion is based at least in part on the first set of spatial characteristics of the first subset and the first set of environmental characteristics associated with the first subset, and generating the adapted second ER content portion is based at least in part on the second set of spatial characteristics of the second subset and the second set of environmental characteristics associated with the second subset. In some implementations, adapting the ER content portions may be based on environmental characteristics such as a type of room, temperature information, lighting information, objects within the physical setting, a time of day, or background color of the physical setting.

As a non-limiting example and with reference to <FIG>, the electronic device <NUM> may be configured to present portions of predetermined content corresponding to different ecosystems based on the environmental characteristics of the subset within the physical setting. Continuing with the non-limiting example, a first portion of predetermined content corresponds to a subterranean ecosystem, a second portion of predetermined content corresponds to a grassland ecosystem, a third portion of predetermined content corresponds to a forest ecosystem, and a fourth portion of predetermined content corresponds to a desert ecosystem. Referring to <FIG>, the electronic device <NUM> may obtain environmental characteristics associated with the different rooms within the house <NUM>. In this example, the environmental characteristics include a type of room, temperature information, or lighting information.

Accordingly, the electronic device <NUM> presents the adapted first ER content portion <NUM> corresponding to the subterranean ecosystem in the user interface <NUM> of the field-of-view 311a of the basement <NUM> based at least in part on environmental characteristics such as the basement <NUM> being located underground, the cool temperature of the basement <NUM>, or the dark lighting conditions of the basement <NUM>. As another example, the electronic device <NUM> presents the adapted second ER content portion <NUM> corresponding to the grassland ecosystem in the user interface <NUM> of the field-of-view 311b of the kitchen <NUM> based at least in part on environmental characteristics such as the lighting characteristics of the kitchen <NUM>, the warm temperature of the kitchen <NUM>, or the fact that the room is a kitchen <NUM>. As a further example, the electronic device <NUM> presents the adapted third ER content portion <NUM> corresponding to the forest ecosystem in the user interface <NUM> of the field-of-view 311c of the bedroom <NUM> based at least in part on environmental characteristics such as the bedroom being located on the second story of the house <NUM> or the lighting conditions of the bedroom <NUM>. As yet another example, the electronic device <NUM> may present the adapted fourth ER content portion <NUM> corresponding to the desert ecosystem in the user interface <NUM> of the field-of-view 311d of the living room <NUM> based at least in part on environmental characteristics such as the hot temperature of the living room <NUM>.

In some implementations, the method <NUM> further includes, obtaining, via an exterior-facing image sensor of the electronic device, image data that correspond to the physical setting, wherein identifying the plurality of subsets within the physical setting is based at least in part on the image data and determining the sets of spatial characteristics for the plurality of subsets within the physical setting is based at least in part on the image data frames. In some implementations, the image data corresponds to video pass-through of other optical information. In some implementations, the image data correspond to first video pass-through image data or second video pass-through image data. In some implementations, if the image data correspond to video pass-through, then the adaptive ER content portion may be composited into the video pass-through content of the live scene. In some implementations, if the display <NUM> corresponds to an additive display that enables optical see-through of the physical setting, then electronic device <NUM> presents ER content by projecting or displaying the adaptive ER content portion on the additive display, which is, in turn, overlaid on the physical setting from the perspective of the user.

<FIG> is a block diagram of an example controller (e.g., the controller <NUM> shown in <FIG>) in accordance with some implementations. While certain specific features are illustrated, those of ordinary skill in the art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity, and so as not to obscure more pertinent aspects of the implementations disclosed herein. To that end, as a non-limiting example, in some implementations the controller <NUM> includes one or more processing units <NUM> (e.g., microprocessors, application-specific integrated-circuits (ASICs), field-programmable gate arrays (FPGAs), graphics processing unit (GPUs), central processing units (CPUs), processing cores, and/or the like), one or more input/output (I/O) devices and sensors <NUM>, one or more communications interface <NUM> (e.g., universal serial bus (USB), IEEE <NUM>. 3x, IEEE <NUM>. 11x, IEEE <NUM>. 16x, global system for mobile communications (GSM), code division multiple access (CDMA), time division multiple access (TDMA), global positioning systems (GPS), infrared (IR), BLUETOOTH, ZIGBEE, and/or the like type interfaces), one or more programming (e.g., I/O) interfaces <NUM>, a memory <NUM>, and one or more communication buses <NUM> for interconnecting these and various other components.

In some implementations, the one or more communication buses <NUM> include circuitry that interconnects and controls communications between system components. In some implementations, the one or more I/O devices and sensors include at least one of a keyboard, a mouse, a touchpad, a joystick, one or more microphones, one or more speakers, one or more image sensors, one or more displays, and/or the like.

The memory <NUM> includes high-speed random-access memory, such as DRAM, SRAM, DDR, RAM, or other random-access solid-state memory devices, and may include non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory <NUM> optionally includes one or more storage devices remotely located from the one or more processing units <NUM>. The memory <NUM> comprises a non-transitory computer readable storage medium. In some implementations, the memory <NUM> or the non-transitory computer readable storage medium of the memory <NUM> stores the following programs, modules, and data structures, or a subset thereof including an operating system <NUM>, a management module <NUM>, an identification module <NUM>, a content adapter module <NUM>, and a navigation module <NUM>. In some implementations, one or more instructions are included in a combination of logic and non-transitory memory.

The operating system <NUM> includes procedures for handling various basic system services and for performing hardware-dependent tasks.

In some implementations, the management module <NUM> is configured to render, manage, and/or coordinate one or more user interfaces (e.g., the user interface <NUM> shown in <FIG>, the user interface <NUM> shown in <FIG>, or the user interface <NUM> shown in <FIG>) for one or more devices associated with different users. To that end, in various implementations, the management module <NUM> includes a data obtaining unit <NUM>, a content manager unit <NUM>, and a data transmitting unit <NUM>.

In some implementations, the data obtaining unit <NUM> is configured to obtain data (e.g., presentation data, user interaction data, sensor data, location data, etc.) from at least the electronic device <NUM> shown in <FIG> and <FIG>. To that end, in various implementations, the data obtaining unit <NUM> includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the content manager unit <NUM> is configured to manage and coordinate the user interface presented to the user by the electronic device <NUM> shown in <FIG> and <FIG>. To that end, in various implementations, the content manager unit <NUM> includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the data transmitting unit <NUM> is configured to transmit data (e.g., presentation data, location data, etc.) to at least the electronic device <NUM> shown in <FIG> and <FIG>. To that end, in various implementations, the data transmitting unit <NUM> includes instruction and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the identification module <NUM> is configured to identify a plurality of subsets associated with a physical setting and determine a set of spatial characteristics for each of the plurality of subsets within the physical setting. To that end in various implementations, the identification module <NUM> includes instruction and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the content adapter module <NUM> is configured to logically map a portion of ER content to a subset that satisfies mapping criterion by identifying the subset that is best suited for each content portion and to adapt the portion of the ER content from reference ER content to the spatial characteristics of the identified subset. To that end in various implementations, the content adapter module <NUM> includes instruction and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the navigation module <NUM> is configured to generate one or more navigation options that allow a user to traverse between the first subset and second subset. To that end in various implementations, the navigation module <NUM> includes instructions and/or logic therefor, and heuristics and metadata therefor.

Although the management module <NUM>, the identification module <NUM>, the content adapter module <NUM> and the navigation module <NUM> are shown as residing on a single device (e.g., the controller <NUM>), it should be understood that in some implementations, any combinations of the management module <NUM>, the identification module <NUM>, the content adapter module <NUM> and the navigation module <NUM> may be located in separate computing devices.

In some implementations, the functionalities of the controller <NUM> are provided by and/or combined with the electronic device <NUM> shown below in <FIG>. Moreover, <FIG> is intended more as a functional description of the various features that could be present in a particular implementation as opposed to a structural schematic of the implementations described herein. As recognized by those of ordinary skill in the art, items shown separately could be combined and some items could be separated. For example, some functional modules shown separately in <FIG> could be implemented in a single module and the various functions of single functional blocks could be implemented by one or more functional blocks in various implementations. The actual number of modules and the division of particular functions and how features are allocated among them will vary from one implementation to another and, in some implementations, depends in part on the particular combination of hardware, software, and/or firmware chosen for a particular implementation.

<FIG> is a block diagram of an example electronic device <NUM> (e.g., a mobile phone, tablet, laptop, near-eye system, etc.) in accordance with some implementations. While certain specific features are illustrated, those of ordinary skill in the art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity, and so as not to obscure more pertinent aspects of the implementations disclosed herein. To that end, as a non-limiting example, in some implementations the electronic device <NUM> includes one or more processing units <NUM> (e.g., microprocessors, ASICs, FPGAs, GPUs, CPUs, processing cores, and/or the like), one or more I/O devices and sensors <NUM>, one or more communications interfaces <NUM> (e.g., USB, IEEE <NUM>. 3x, IEEE <NUM>. 11x, IEEE <NUM>. 16x, GSM, CDMA, TDMA, GPS, IR, BLUETOOTH, ZIGBEE, and/or the like type interfaces), one or more programming interfaces <NUM>, one or more displays <NUM>, one or more image sensors <NUM>, a memory <NUM>, and one or more communication buses <NUM> for interconnecting these and various other components.

In some implementations, the one or more communication buses <NUM> include circuitry that interconnects and controls communications between system components. In some implementations, the one or more I/O devices and sensors <NUM> include at least one of an inertial measurement unit (IMU), an accelerometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., blood pressure monitor, heart rate monitor, blood oxygen sensor, blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptics engine, a heating and/or cooling unit, a skin shear engine, and/or the like.

In some implementations, the one or more displays <NUM> are capable of presenting a user interface (e.g., the user interface <NUM> shown in <FIG>, the user interface <NUM> shown in <FIG>, or the user interface <NUM> shown in <FIG>) or ER content. In some implementations, the one or more displays <NUM> are also configured to present flat video content to the user (e.g., a <NUM>-dimensional or "flat" audio video interleave (AVI), flash video (FLV), Windows Media Video (WMV), or the like file associated with a TV episode or a movie, or live video pass-through of the operating environments. In some implementations, the one or more displays <NUM> correspond to an additive display, holographic, digital light processing (DLP), liquid-crystal display (LCD), liquid-crystal on silicon (LCoS), organic light-emitting field-effect transitory (OLET), organic light-emitting diode (OLED), surface-conduction electron-emitter display (SED), field-emission display (FED), quantum-dot light-emitting diode (QD-LED), micro-electro mechanical systems (MEMS), and/or the like display types. In some implementations, the one or more displays <NUM> correspond to diffractive, reflective, polarized, holographic, etc. waveguide displays. For example, the electronic device <NUM> includes a single display. In another example, the electronic device <NUM> includes a display for each eye of the user.

In some implementations, the one or more image sensors <NUM> are configured to obtain image data frames. For example, the one or more image sensors <NUM> correspond to one or more RGB cameras (e.g., with a CMOS image sensor, or a CCD image sensor), infrared (IR) image sensors, event-based cameras, and/or the like.

The memory <NUM> includes high-speed random-access memory, such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices. In some implementations, the memory <NUM> includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory <NUM> optionally includes one or more storage devices remotely located from the one or more processing units <NUM>. The memory <NUM> comprises a non-transitory computer readable storage medium. In some implementations, the memory <NUM> or the non-transitory computer readable storage medium of the memory <NUM> stores the following programs, modules and data structures, or a subset thereof including an optional operating system <NUM> and a presentation module <NUM>.

The optional operating system <NUM> includes procedures for handling various basic system services and for performing hardware dependent tasks. In some implementations, the presentation module <NUM> is configured to present user interfaces or ER content to the user via the one or more displays <NUM>. To that end, in various implementations, the presentation module <NUM> includes a data obtaining unit <NUM>, a presentation unit <NUM>, and a data transmitting unit <NUM>.

In some implementations, the data obtaining unit <NUM> is configured to obtain data (e.g., presentation data, interaction data, location data, etc.) from at least one of the one or more I/O devices and sensors <NUM> associated with the electronic device <NUM> or the controller <NUM> shown in <FIG> and <FIG>. To that end, in various implementations, the data obtaining unit <NUM> includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the presentation unit <NUM> is configured to present a user interface (e.g., the user interface <NUM> shown in <FIG>, the user interface <NUM> shown in <FIG>, or the user interface <NUM> shown in <FIG>) or an ER experience via the one or more displays <NUM>. To that end, in various implementations, the presentation unit <NUM> includes instructions and/or logic therefor, and heuristics and metadata therefor.

In some implementations, the data transmitting unit <NUM> is configured to transmit data (e.g., presentation data, location data, etc.) to the controller <NUM> shown in <FIG> and <FIG>. To that end, in various implementations, the data transmitting unit <NUM> includes instructions and/or logic therefor, and heuristics and metadata therefor.

Although the data obtaining unit <NUM>, the presentation unit <NUM>, and the data transmitting unit <NUM> are shown as residing on a single device (e.g., the electronic device <NUM> shown in <FIG> or <FIG>), it should be understood that in some implementations, any combination of the data obtaining unit <NUM>, the presentation unit <NUM>, and the data transmitting unit <NUM> may be located in separate computing devices. In some implementations, the functions and/or components of the controller <NUM> are combined with or provided by the electronic device <NUM>.

Moreover, <FIG> is intended more as a functional description of the various features that could be present in a particular implementation as opposed to a structural schematic of the implementations described herein. As recognized by those of ordinary skill in the art, items shown separately could be combined and some items could be separated. For example, some functional modules shown separately in <FIG> could be implemented in a single module and the various functions of single functional blocks could be implemented by one or more functional blocks in various implementations. The actual number of modules and the division of particular functions and how features are allocated among them will vary from one implementation to another and, in some implementations, depends in part on the particular combination of hardware, software, and/or firmware chosen for a particular implementation.

It will also be understood that, although the terms "first", "second", etc. may be used herein to describe various elements, these elements should not be limited by these terms. For example, a first subset could be termed a second subset, and, similarly, a second subset could be termed a first subset, which changing the meaning of the description, so long as the occurrences of the "first subset" are renamed consistently and the occurrences of the "second subset" are renamed consistently. The first subset and the second subset are both subsets, but they are not the same subset.

The terminology used herein is for the purpose of describing particular implementations only and is not intended to be limiting of the claims. As used in the description of the implementations and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

Claim 1:
A method comprising:
at an electronic device (<NUM>) including an image sensor (<NUM>), one or more processors (<NUM>), a non-transitory memory (<NUM>), and a display (<NUM>):
identifying a plurality of subsets (230a, 232a) associated with a physical setting (<NUM>);
determining a set of spatial characteristics for each of the plurality of subsets (230a, 232a) within the physical setting (<NUM>), wherein a first set of spatial characteristics characterizes one or more dimensions of a first subset (230a) associated with the physical setting (<NUM>) among the plurality of subsets associated with the physical setting (<NUM>), and a second set of spatial characteristics characterizes one or more dimensions of a second subset (232a) associated with the physical setting (<NUM>) among the plurality of subsets associated with the physical setting (<NUM>);
generating an adapted first enhanced reality, ER, content portion (240a) for the first subset (232a) associated with the physical setting (<NUM>) based at least in part on the first set of spatial characteristics;
generating an adapted second ER content portion (242a) for the second subset (232a) associated with the physical setting (<NUM>) based at least in part on the second set of spatial characteristics;
generating a navigation option (217a) that allows a user to traverse between the first and the second subsets (230a, 232a) associated with the physical setting based on the first and second sets of spatial characteristics; and
while displaying the adapted first ER content portion (240a) associated with the first subset (230a) associated with the physical setting (<NUM>), indicating the navigation option by displaying a movement of a virtual agent associated with the adapted first ER content portion from the first subset to the second subset to indicate the navigation option that allows the user to traverse between the first and the second subsets (230a, 232a) associated with the physical setting (<NUM>) by following the movement of the virtual agent while the virtual agent is carrying out an action within the physical setting.