Patent Description:
In first-person perspective-mediated reality a user's real point of view (location and orientation) determines the point of view (location and orientation) within a virtual space of a virtual user. The point of view of the virtual user determines a representation of a virtual space rendered to the user.

<CIT> discloses systems and methods for interacting with virtual reality space using a head-mounted display. It discloses detecting a real world object in a real world space in which a user is interacting. An image of the real world object is presented in the VR space. The image of the real world object is used to identify presence of the real world object while interacting with the VR space. Interaction by the user with the real world object is detected. The VR space is configured to generate a simulator view of the user interacting with the VR object that is mapped to the interaction with the real world object. This simulator view is presented in the head-mounted display.

According to various embodiments there is provided an apparatus as defined by claim <NUM>, a method as defined by claim <NUM>, a computer program as defined by claim <NUM>, and a system as defined by claim <NUM>.

In some but not necessarily all examples, the apparatus comprises means for determining a boundary of the available real space and determining a boundary of the constrained virtual space that corresponds to the available real space.

In some but not necessarily all examples, the apparatus comprises means for determining a shape of a boundary of the available real space and determining a shape of a boundary of the constrained virtual space that corresponds to the available real space and means for determining a location of the boundary of the constrained virtual space within the virtual space.

In some but not necessarily all examples, the apparatus comprises means for determining the location of the boundary of the constrained virtual space within the virtual space and/or the scale factor for a mapping from the available real space to the constrained virtual space.

In some but not necessarily all examples, the apparatus comprises means for providing a map to the user than illustrates a boundary of the constrained virtual space within the virtual space and provides an option for the user to re-locate the constrained virtual space within the virtual space and/or change a scale, but not a shape, of the constrained virtual space within the virtual space.

In some but not necessarily all examples, the determined sub-set of available mediated reality content is limited to what can be seen by the user moving, without interacting, within the determined available real space or the determined sub-set of available mediated reality content is limited to what can be seen by the user moving, with interacting, within the determined available real space.

In some but not necessarily all examples, the multiple representations of the virtual space include virtual visual scenes that are representations of a virtual visual space viewed from a location within the virtual space and/or
sound scenes that are representations of a virtual sound space listened to from a position within the virtual space.

In some but not necessarily all examples, automatically determining a sub-set of the available mediated reality content without requiring the user to change location in the real space, generates a content for a panning preview and wherein
causing rendering of at least some of the determined sub-set of the available virtual reality content to the user, causes rendering of the panning preview without requiring the user to change location in the real space.

In some but not necessarily all examples, the sub-set of available mediated reality content comprises a sub-set of the multiple representations of the virtual space when observed from a sub-set of virtual locations within the constrained virtual space, wherein the sub-set of virtual locations are selected by the user during rendering of a preview.

In some but not necessarily all examples, the apparatus comprises means for selecting the sub-set of virtual locations based on a gaze direction and gaze duration during a preview.

According to various, but not necessarily all, embodiments there is provided a method comprising:.

In some but not necessarily all examples, the method occurs in response to switching to first-person perspective-mediated reality from third person perspective-mediated reality.

According to various, but not necessarily all, embodiments there is provided a computer program comprising program instructions for causing an apparatus to perform at least the following:.

In some but not necessarily all examples, the computer program comprises program instructions for causing an apparatus to perform at least the methods.

According to various, but not necessarily all, embodiments there is provided a system comprising:.

"artificial environment" may be something that has been recorded or generated.

"virtual visual space" refers to a fully or partially artificial environment that may be viewed, which may be three dimensional.

"virtual visual scene" refers to a representation of the virtual visual space viewed from a particular point of view (position) within the virtual visual space.

'virtual visual object' is a visible virtual object within a virtual visual scene.

"sound space" (or "virtual sound space") refers to an arrangement of sound sources in a three-dimensional space. A sound space may be defined in relation to recording sounds (a recorded sound space) and in relation to rendering sounds (a rendered sound space).

"sound scene" (or "virtual sound scene") refers to a representation of the sound space listened to from a particular point of view (position) within the sound space.

"sound object" refers to sound source that may be located within the sound space. A source sound object represents a sound source within the sound space, in contrast to a sound source associated with an object in the virtual visual space. A recorded sound object represents sounds recorded at a particular microphone or location. A rendered sound object represents sounds rendered from a particular location.

"virtual space" may mean a virtual visual space, mean a sound space or mean a combination of a virtual visual space and corresponding sound space. In some examples, the virtual space may extend horizontally up to <NUM>° and may extend vertically up to <NUM>°.

"virtual scene" may mean a virtual visual scene, mean a sound scene or mean a combination of a virtual visual scene and corresponding sound scene.

'virtual object' is an object within a virtual scene, it may be an artificial virtual object (e.g. a computer-generated virtual object) or it may be an image of a real object in a real space that is live or recorded. It may be a sound object and/or a virtual visual object.

"Virtual position" is a position within a virtual space. It may be defined using a virtual location and/or a virtual orientation. It may be considered to be a movable 'point of view'.

"Correspondence" or "corresponding" when used in relation to a sound space and a virtual visual space means that the sound space and virtual visual space are time and space aligned, that is they are the same space at the same time.

"Correspondence" or "corresponding" when used in relation to a sound scene and a virtual visual scene (or visual scene) means that the sound space and virtual visual space (or visual scene) are corresponding and a notional (virtual) listener whose point of view defines the sound scene and a notional (virtual) viewer whose point of view defines the virtual visual scene (or visual scene) are at the same location and orientation, that is they have the same point of view (same virtual position).

"real space" (or "physical space") refers to a real environment, which may be three dimensional.

"real scene" refers to a representation of the real space from a particular point of view (position) within the real space.

"real visual scene" refers to a visual representation of the real space viewed from a particular real point of view (position) within the real space.

"mediated reality" in this document refers to a user experiencing, for example visually, a fully or partially artificial environment (a virtual space) as a virtual scene at least partially rendered by an apparatus to a user. The virtual scene is determined by a point of view (virtual position) within the virtual space. Displaying the virtual scene means providing a virtual visual scene in a form that can be perceived by the user.

"augmented reality content" is a form of mediated reality content which enables a user to experience, for example visually, a partially artificial environment (a virtual space) as a virtual scene. Augmented reality content could include interactive content such as a video game or non-interactive content such as motion video.

"virtual reality content" is a form of mediated reality content which enables a user to experience, for example visually, a fully artificial environment (a virtual space) as a virtual scene. Virtual reality content could include interactive content such as a video game or non-interactive content such as motion video.

"virtual user" defines the point of view (virtual position- location and/or orientation) in virtual space used to generate a perspective-mediated sound scene and/or visual scene. A virtual user may be a notional listener and/or a notional viewer.

"notional listener" defines the point of view (virtual position- location and/or orientation) in virtual space used to generate a perspective-mediated sound scene, irrespective of whether or not a user is actually listening.

"notional viewer" defines the point of view (virtual position- location and/or orientation) in virtual space used to generate a perspective-mediated visual scene, irrespective of whether or not a user is actually viewing.

Three degrees of freedom (3DoF) describes mediated reality where the virtual position is determined by orientation only (e.g. the three degrees of three-dimensional orientation). In relation to first-person perspective-mediated reality, only the user's orientation determines the virtual position.

Six degrees of freedom (6DoF) describes mediated reality where the virtual position is determined by both orientation (e.g. the three degrees of three-dimensional orientation) and location (e.g. the three degrees of three-dimensional location). In relation to first-person perspective-mediated reality, both the user's orientation and the user's location in the real space determine the virtual position.

<FIG> illustrate the application of a method to a situation where there is rendering of mediated reality using virtual content. In this context, mediated reality means the rendering of mediated reality for the purposes of achieving mediated reality, for example, augmented reality or virtual reality. In these examples, the mediated reality is first-person perspective-mediated reality. It may or may not be user interactive. It may be 3DoF or 6DoF.

<FIG> illustrate at a first time a real space <NUM>, a sound space <NUM> and a visual space <NUM> respectively. There is correspondence between the sound space <NUM> and the virtual visual space <NUM>. As illustrated in <FIG>, a user <NUM> in the real space <NUM> has a position (point of view) defined by a location <NUM> and an orientation <NUM>. The location is a three-dimensional location and the orientation is a three-dimensional orientation. As illustrated in <FIG> a virtual user <NUM> in the virtual space (sound space <NUM> and/or virtual visual space <NUM>) has a virtual position (virtual point of view) defined by a virtual location <NUM> and a virtual orientation <NUM>. The virtual location <NUM> is a three-dimensional location and the virtual orientation <NUM> is a three-dimensional orientation.

In 3DoF first-person perspective-mediated reality, an orientation <NUM> of the user <NUM> controls a virtual orientation <NUM> of a virtual user <NUM>. There is a correspondence between the orientation <NUM> and the virtual orientation <NUM> such that a change in the orientation <NUM> produces the same change in the virtual orientation <NUM>. The virtual orientation <NUM> of the virtual user <NUM> in combination with a virtual field of view <NUM> defines a virtual visual scene <NUM> within the virtual visual space <NUM>. In some examples, it may also define a virtual sound scene <NUM>. A virtual visual scene <NUM> is that part of the virtual visual space <NUM> that is displayed to a user <NUM>. A virtual sound scene <NUM> is that part of the virtual sound space <NUM> that is rendered to a user <NUM>. The virtual sound space <NUM> and the virtual visual space <NUM> correspond in that a position within the virtual sound space <NUM> has an equivalent position within the virtual visual space <NUM>. In 3DoF mediated reality, a change in the location <NUM> of the user <NUM> does not change the virtual position <NUM> or virtual orientation <NUM> of the virtual user <NUM>.

In the example of 6DoF first-person perspective-mediated reality, the situation is as described for 3DoF and in addition it is possible to change the rendered virtual sound scene <NUM> and the displayed virtual visual scene <NUM> by movement of a location <NUM> of the user <NUM>. For example, there may be a mapping M between the location <NUM> of the user <NUM> and the virtual location <NUM> of the virtual user <NUM>. A change in the location <NUM> of the user <NUM> produces a corresponding change in the virtual location <NUM> of the virtual user <NUM>. A change in the virtual location <NUM> of the virtual user <NUM> changes the rendered sound scene <NUM> and also changes the rendered visual scene <NUM>.

This may be appreciated from <FIG> which illustrate the consequences of a change in location <NUM> and orientation <NUM> of the user <NUM> on respectively the rendered sound scene <NUM> (<FIG>) and the rendered visual scene <NUM> (<FIG>).

In the example illustrated in <FIG>, the real space <NUM> is constrained to an available real space <NUM>. The available real space <NUM> is that part of the real space <NUM> that the user <NUM> can have a location <NUM> within. The user <NUM> cannot have a location <NUM> outside the available real space <NUM>. This therefore constrains 6DoF virtual reality. As illustrated in <FIG>, the available real space <NUM> has a corresponding constrained virtual space <NUM>. The constrained virtual space <NUM> is that part of the virtual space (sound space <NUM> and/or virtual visual space <NUM>) that a virtual user <NUM> can have a virtual location <NUM> within.

It is possible to change the rendered virtual sound scene <NUM> and/or the displayed virtual visual scene <NUM> by movement of a location <NUM> of the user <NUM> within the available real space <NUM>. For example, there is a mapping M between the location <NUM> of the user <NUM> in the available real space <NUM> and the virtual location <NUM> of the virtual user <NUM> in the constrained virtual space <NUM>. This mapping creates a correspondence between the available real space <NUM> and the constrained virtual space <NUM>. A change in the location <NUM> of the user <NUM> within the available real space <NUM> produces a corresponding change in the virtual location <NUM> of the virtual user <NUM> within the constrained virtual space <NUM>. A change in the virtual location <NUM> of the virtual user <NUM> within the constrained virtual space <NUM> changes the rendered sound scene <NUM> and also changes the rendered visual scene <NUM>. The virtual user <NUM> cannot have a virtual location <NUM> outside the constrained virtual space <NUM> without a change in the mapping M that creates correspondence between the real space <NUM> and the virtual space.

The available real space <NUM> comprises a boundary <NUM>, enclosing a volume defining the available real space <NUM>. The constrained virtual space <NUM> that corresponds to the available real space <NUM> via the mapping M comprises a boundary <NUM>, enclosing a volume defining the constrained virtual space <NUM>.

It is only within the available real space <NUM>, that a location <NUM> and orientation <NUM> of the user <NUM> determines a real point of view (position) of the user <NUM> that is mapped to a virtual point of view (virtual position) of the virtual user <NUM> within the constrained virtual space <NUM>. The virtual point of view of the virtual user <NUM> determines the rendered virtual content. The location <NUM> and orientation <NUM> are typically three-dimensional but in some examples may be two-dimensional.

The boundary <NUM> of the available real space <NUM> corresponds to the boundary <NUM> of the constrained virtual space <NUM> via the mapping M.

<FIG> illustrates a top-perspective view of an example of an available real space <NUM>. In this example the real space <NUM> includes a room <NUM> that includes furniture <NUM>. At least some of the floor space <NUM> is not available to the user <NUM> because it is blocked by furniture <NUM>. The available floor space <NUM> is the available real space <NUM> in two-dimensions.

<FIG> illustrates a top-perspective view of an example of a constrained virtual space <NUM> that corresponds to the available real space <NUM> illustrated in <FIG>. In this example the virtual space extends beyond the boundary <NUM> of the constrained virtual space <NUM>.

<FIG> illustrate that different mappings M can be used to define different correspondences between the available real space <NUM> and the constrained virtual space <NUM>.

Each different mapping uses an orientation reference mapping to map an origin orientation of the user <NUM> to an origin virtual orientation of the virtual user <NUM>. Changes in orientation <NUM> of the user <NUM> are measured relative to the origin orientation. Changes in virtual orientation <NUM> of the virtual user <NUM> are effected relative to the origin virtual orientation. The orientation reference mapping orients the constrained virtual space <NUM> corresponding to the available real space <NUM> within the virtual space.

Each different mapping uses an orientation mapping that maps changes in the orientation <NUM> of the user <NUM> to changes in the orientation <NUM> of the virtual user <NUM>. The orientation mapping is exact and maps changes in the orientation <NUM> of the user <NUM> to the same changes in the orientation <NUM> of the virtual user <NUM>. If an orientation <NUM> of the user <NUM> is defined by a polar angle (θ) and an azimuthal angle (ϕ) (from the origin orientation) and a virtual orientation <NUM> of the virtual user <NUM> can be defined by a polar angle (θ') and an azimuthal angle (ϕ')(from the virtual origin orientation), then a change in the orientation <NUM> of the user <NUM> is defined by a change in the polar angle (Δθ) and a change in the azimuthal angle (Δϕ) and a change in the virtual orientation <NUM> of the virtual user <NUM> can be defined by a change in the polar angle (Δθ') and a change in the azimuthal angle (Δϕ'). The mapping M in this example, is such that Δθ = Δθ' & Δϕ = Δϕ'.

Each different mapping uses a location reference mapping to map an origin location of the user <NUM> to an origin virtual location of the virtual user <NUM>. Changes in location <NUM> of the user <NUM> are measured relative to the origin location. Changes in virtual location <NUM> of the virtual user <NUM> are effected relative to the origin virtual location. The location reference mapping re-locates the constrained virtual space <NUM> corresponding to the available real space <NUM> within the virtual space.

Each different mapping uses a change in location mapping (a scale mapping) to map changes in the location <NUM> of the user <NUM> to changes in the virtual position <NUM> of the virtual user <NUM>. The scale mapping re-sizes the constrained virtual space <NUM> corresponding to the available real space <NUM> within the virtual space. If a location <NUM> of the user <NUM> is defined by a Cartesian co-ordinate (x, y, z) then a virtual location <NUM> of the virtual user <NUM> can be defined by corresponding Cartesian coordinates (x', y', z'). A change in the location <NUM> of the user <NUM> is defined by a change in one or more of the coordinates (Δx, Δy, Δz) and a change in the virtual position <NUM> of the virtual user <NUM> can be defined by a change in one or more of the coordinates (Δx', Δy', Δz'). The mapping M in this example, is such that Δx = kx. Δx', Δy= ky. Δy' , Δz= kz. Δz', where kx, ky, kz, are independent, and possibly different, constants defined by the mapping M. In some but not necessarily all examples the mapping M is isomorphic and kx= ky= kz. In some but not necessarily all example the scaling is exact (<NUM> to <NUM>) i.e. kx= ky= kz =<NUM>.

The illustrated mappings M differ in that they have different orientation reference mappings and/or location reference mappings and/or different scale mappings.

<FIG> illustrate a first mapping M that has a first orientation reference mapping, a first location mapping and a first scale mapping.

<FIG> illustrate a second mapping M that has a second orientation reference mapping (different to the first orientation reference mapping), a second location mapping (different to the first location mapping) and the first scale mapping. The constrained virtual space <NUM> has been re-oriented and re-located but not re-sized compared to <FIG>. In other examples it could only be re-oriented or only re-located.

<FIG> illustrate a third mapping M that has the first orientation reference mapping, a third location mapping (different to the first location mapping and the second location mapping) and a second scale mapping (different to the first scale mapping). The constrained virtual space <NUM> has been re-located and re-sized compared to <FIG> but not re-oriented. In other examples it could only be re-located or only re-sized.

<FIG> illustrates an example of a method <NUM>.

The method <NUM> comprises, at block <NUM>, determining an available real space <NUM> comprising a portion of a real space <NUM> that comprises locations <NUM> available to a user <NUM> to control a corresponding virtual location <NUM> of a virtual user <NUM>, via first-person perspective-mediated reality.

As previously explained, first-person perspective-mediated reality creates a correspondence between a location <NUM> and orientation <NUM> of a user <NUM> in real space <NUM> to a virtual location <NUM> and virtual orientation <NUM> of a virtual user <NUM>.

The method <NUM> comprises, at block <NUM>, determining a constrained virtual space <NUM> comprising a portion of a virtual space <NUM>, <NUM> that corresponds to the available real space <NUM>.

The method <NUM> comprises, at block <NUM>, determining available mediated reality content which comprises multiple representations of the virtual space <NUM>, <NUM> when observed from multiple virtual locations <NUM> within the constrained virtual space <NUM>. The multiple representations may be virtual visual scenes, sound scenes or virtual visual scenes and corresponding sound scenes.

An example of available mediated reality content <NUM> is described later with respect to <FIG>.

The method <NUM> comprises, at block <NUM>, automatically determining a sub-set <NUM> of the available mediated reality content <NUM> without requiring the user <NUM> to change location <NUM> in the real space <NUM>.

The method <NUM> comprises, at block <NUM>, causing rendering of at least some of the determined sub-set <NUM> of the available virtual reality content to the user <NUM>.

<FIG> illustrates, in more detail, an example of blocks <NUM> and <NUM> of the method <NUM>.

In this example, at block <NUM>, the method <NUM> determines the available real space <NUM> by determining a boundary <NUM> of the available real space. At block <NUM>, the method <NUM> determines the constrained virtual space <NUM> by determining a boundary <NUM> of the constrained virtual space <NUM> that corresponds to the boundary <NUM> of the available real space <NUM>.

Determining the boundary <NUM> of the constrained virtual space <NUM> comprises, at block <NUM>, determining a shape of the boundary <NUM> of the available real space <NUM> and at block <NUM> determining a shape of the boundary <NUM> of the constrained virtual space <NUM> that corresponds to the shape of the boundary <NUM> of the available real space <NUM>. The method also comprises at block <NUM>, determining an orientation of the boundary <NUM> of the constrained virtual space <NUM> within the virtual space. The method also comprises at block <NUM>, determining a location of the boundary <NUM> of the constrained virtual space <NUM> within the virtual space. The method <NUM> may also comprise, at block <NUM>, determining a scale factor for a mapping from the available real space <NUM> to the constrained virtual space <NUM>.

The shape of the boundary <NUM> of the available real space <NUM> can, for example, be determined by sensing the presence of obstructions using transmitted signals, for example ultrasound signals, or by a user <NUM> tracing out the boundary <NUM> via user movement. This may be achieved by using a pointer device that the user points to the boundary <NUM> at floor level, the location and orientation of the pointing device being converted to the boundary <NUM> at floor level. This may be alternatively be achieved by using a tracking device that the user carries over the boundary <NUM> at floor level, the location of the tracking device being converted to a trace of the boundary <NUM> at floor level.

It will be appreciated from the foregoing description that block <NUM> may be based upon an isomorphic mapping M from the available real space <NUM> to the constrained virtual space <NUM>. Thus the shape of the boundary in the real space is also the same shape of the boundary <NUM> in the virtual space. The block <NUM> is based upon the orientation reference mapping of the mapping M between the available real space <NUM> and the constrained virtual space <NUM>. The block <NUM> is based upon the location mapping of the mapping M between the available real space <NUM> and the constrained virtual space <NUM>. The block <NUM> is based upon the scale mapping of the mapping M between the available real space <NUM> and the constrained virtual space <NUM>.

The mapping M may be automatically optimized for one or more different criteria. For example, the mapping M may be optimized in dependence upon the available mediated reality content defined by the constrained virtual space <NUM>. The constrained virtual space <NUM> may, for example, be defined so that the available mediated reality content <NUM> is content that satisfies one or more criteria such as duration, interactive, rating, etc..

The mapping M can therefore be determined automatically as part of the method <NUM>.

In some examples it may be desirable to enable the user <NUM> to define or to adjust the mapping M between the available real space <NUM> and the constrained virtual space <NUM>, so that the user <NUM> can, in effect, control the constrained virtual space <NUM> and, as a consequence, the available mediated reality content <NUM>.

<FIG> illustrates an example of a user interface <NUM> that is configured to enable a user <NUM> to control, by defining or adjusting, the mapping M between the available real space <NUM> and the constrained virtual space <NUM>. The user interface <NUM> comprises a map <NUM> that illustrates the virtual space. The user interface includes, as an overlay on the map <NUM>, a representation of the constrained virtual space <NUM>. In this example, the boundary <NUM> of the constrained virtual space <NUM> is illustrated within the map <NUM>.

The user interface <NUM> provides user input controls <NUM>.

In this example but not necessarily all examples, the user input controls <NUM> comprise a user input control <NUM> for controlling the orientation reference mapping. This enables the user <NUM> to change an orientation of the constrained virtual space <NUM> within the map <NUM> of the virtual space.

In this example but not necessarily all examples, the user input controls <NUM> comprises a user input control <NUM> for controlling the location reference mapping. This enables the user <NUM> to change a location of the constrained virtual space <NUM> within the map <NUM> of the virtual space.

In this example but not necessarily all examples, the user input controls <NUM> comprises a user input control <NUM> for controlling a scale mapping. This enables the user <NUM> to resize the constrained virtual space <NUM> within the map <NUM> of the virtual space.

The user input controls may be any suitable detector that enables a user <NUM> to provide a control input such as a touch screen that displays the map <NUM>. A simultaneous two-point contact on the touch screen over the constrained virtual space <NUM> within the map <NUM> may be used to re-size the constrained virtual space <NUM> (e.g. by increasing the separation distance between the two points of simultaneous contact) and/or reorient the constrained virtual space <NUM> (e.g. by rotating the two-points of simultaneous contact), and/or re-locate the constrained virtual space <NUM> (e.g. by sliding movement of the two points of simultaneous contact).

The user interface <NUM> therefore provides a map <NUM> to the user <NUM> that illustrates a boundary <NUM> of the constrained virtual space <NUM> within the virtual space and provides one or more options for the user <NUM> to relocate and/or resize and/or reorient the constrained virtual space <NUM> within the virtual space. It will be appreciated that the options maintain the isomorphism between the real space and the virtual space. Thus although the values of kx, ky, and kz may, in some examples, be varied they remain equal to each other.

In some examples, it may be possible for the user <NUM> to provide a user input that voluntarily changes the available real space <NUM> and therefore cause a consequential variation in the constrained virtual space <NUM>.

In some, but not necessarily all, examples, the user interface <NUM> may additionally or alternatively be used to enable a user <NUM> to control the sub-set <NUM> of the available mediated reality content <NUM>. For example, if the available mediated reality content <NUM> comprises representations of the virtual space observable from virtual locations, the user interface <NUM> may allow the user <NUM> to determine preferred representations of the virtual space observable from preferred virtual locations. In some examples, the user may be able to define the sub-set <NUM> of the available mediated reality content <NUM> by indicating a first set of the preferred locations that are within the constrained virtual space <NUM> and indicating a second set of the preferred locations that are outside the constrained virtual space <NUM>.

<FIG> illustrates a relationship between the constrained virtual space <NUM> and its boundary <NUM> and the available mediated reality content <NUM>.

In some but not necessarily all examples, the available mediated reality content <NUM> comprises content that can be seen by the user <NUM> moving, without interacting, within the determined available real space <NUM>. This content is seen by the user <NUM> moving within the determined available real space <NUM>. In some examples, the available mediated reality content <NUM> is limited to only content that can be seen by the user <NUM> moving, without interacting, within the determined available real space <NUM>.

In some but not necessarily all examples, the available mediated reality content <NUM> comprises content 170A that is both visible to and accessible by the virtual user <NUM> moving within the constrained virtual space <NUM>. This content can be seen by the user <NUM> moving within the determined available real space <NUM>. In some examples, the available mediated reality content <NUM> is limited to only content that is both visible to and accessible 170A by the virtual user <NUM>.

In some but not necessarily all examples, the available mediated reality content <NUM> comprises what is visible to the virtual user <NUM> moving within the constrained virtual space <NUM>. This corresponds to the content 170A, 170B that can be seen by the user moving within the determined available real space <NUM>. It includes the content 170A that is visible and accessible and the content 170B that is visible but not accessible. The user <NUM> (or virtual user <NUM>) cannot interact with content that is not accessible.

In the illustrated example the determined available mediated reality content <NUM> includes content that is visible content, renderable as a consequence of action by the user <NUM> within the determined available real space <NUM> and accessible content, renderable for interaction as a consequence of action by the user <NUM> within the determined available space <NUM>. The determined available mediated reality content <NUM> comprises multiple representations of the virtual space when observed from multiple locations within the constrained virtual space <NUM>. The multiple representations of the virtual space <NUM> may, for example, include virtual visual scenes that are representations of a virtual visual space viewed from a location within the virtual space and/or sound scenes that are representations of a virtual sound space listened to from a position within the virtual space. In some, but not necessarily all, examples, the multiple representations of the virtual space include virtual visual scenes that are video representations of a virtual visual space.

In the illustrated example the determined available mediated reality content <NUM> does not include content <NUM> that is not visible or not accessible.

<FIG> and <FIG> illustrate an example of the operation of block <NUM> of the method <NUM>. In this example the method comprises, at block <NUM>, automatically determining a sub-set <NUM> of the available mediated reality content <NUM> without requiring the user <NUM> to change location <NUM> in the real space <NUM>. In this example, the sub-set <NUM> of the available mediated reality content <NUM> is a preview of the available mediated reality content <NUM>. The block <NUM> therefore automatically creates a preview.

As the available mediated reality content only comprises representations of the virtual space observable from multiple virtual locations within the constrained virtual space <NUM>, the sub-set <NUM> of the available mediated reality content <NUM> only comprises representations of the virtual space observable from virtual locations within the constrained virtual space <NUM>. Therefore all the content promised by the preview is viewable by the user <NUM> by moving only within the available real space <NUM>, after the preview is exited and the previewed content is selected either manually or automatically. The preview contains only those parts of the content that are visible from different locations within the available real space <NUM>. In some example, the preview may be generated from the content by analysis of the content and in other examples the preview may be generated by editing an existing preview of the content to remove unviewable content by analysis of the existing preview.

In the example illustrated in <FIG>, the preview is a panning preview <NUM>. The block <NUM> of method <NUM> (not illustrated), generates content for a panning preview <NUM> and block <NUM> of method <NUM> (not illustrated) causes rendering of the panning preview <NUM> without requiring the user <NUM> to change location <NUM> in the real space <NUM>. The sub-set <NUM> of the available mediated reality content <NUM>, that controls the panning preview, comprises multiple representations of the virtual space that are observable from some or all orientations <NUM> along a trail <NUM> of virtual locations <NUM>.

In some examples, the trail <NUM> is wholly within the constrained virtual space <NUM>. In other examples, the trail <NUM> is partially or wholly outside the constrained virtual space <NUM>. Where the trail <NUM> is outside the constrained virtual space <NUM>, the sub-set <NUM> of the available mediated reality content <NUM>, that control the panning preview, comprises multiple representations of the virtual space that are observable from orientations <NUM>, along the trail <NUM> of virtual locations <NUM>, that are directed towards the constrained virtual space <NUM>.

In the example illustrated in <FIG>, the panning preview <NUM> is created as a fly-through trail <NUM>. In this fly-through trail, the virtual location <NUM> of the virtual user <NUM> that determines a rendered representation of the virtual space, moves automatically along the trail <NUM>. In some examples during rendering of the panning preview as a fly-through trail <NUM>, changes in the virtual orientation <NUM> of the virtual user <NUM> that determine the rendered representation of the virtual space (e.g. virtual visual scene) correspond to changes in the orientation <NUM> of the user <NUM> as the virtual user <NUM> moves automatically along the trail <NUM>. In other examples of a fly-through trail <NUM>, the virtual orientation <NUM> of the virtual user <NUM> changes automatically as the virtual user <NUM> moves automatically along the trail <NUM> and is not dependent upon changes in the orientation <NUM> of the user <NUM>.

<FIG> illustrates a different example, in which the sub-set <NUM> of the available mediated reality content <NUM>, that controls the preview, comprises multiple representations of the virtual space when observed from specific discrete virtual locations <NUM> within the constrained virtual space <NUM>. In this example the specific discrete virtual locations are a set of preferred locations <NUM> within the constrained virtual space <NUM>.

In some but not necessarily all examples, the sub-set <NUM> of the available mediated reality content <NUM>, that controls the preview, comprises multiple representations of the virtual space when observed from specific discrete virtual locations <NUM> within the constrained virtual space <NUM> at specific orientations.

In the example illustrated in <FIG>, in the preview, the virtual location <NUM> of the virtual user <NUM> that determines a rendered representation of the virtual space, moves automatically from location <NUM> to location <NUM>. In some examples during rendering of the preview, changes in the virtual orientation <NUM> of the virtual user <NUM> that determine the rendered representation of the virtual space (e.g. virtual visual scene) correspond to changes in the orientation <NUM> of the user <NUM> as the virtual user <NUM> moves automatically from location <NUM> to location <NUM>. In other examples, the virtual orientation <NUM> of the virtual user <NUM> changes automatically as the virtual user <NUM> moves automatically from location <NUM> to location <NUM> and is not dependent upon changes in the orientation <NUM> of the user <NUM>.

The examples illustrated in <FIG> describe examples in which the sub-set <NUM> of the available mediated reality content <NUM> is automatically determined during rendering of a preview. In these examples, the sub-set <NUM> of available mediated reality content <NUM> comprises a sub-set of the multiple representations of the virtual space when observed from a sub-set of virtual locations within the constrained virtual space <NUM>. The sub-set of virtual locations <NUM> are selected by the user during rendering of a preview. The selected virtual locations <NUM> and the available real space <NUM> determines the constrained virtual space <NUM>.

<FIG> illustrates an example of the method <NUM> in which a preview is used to determine the sub-set of the available mediated reality content <NUM>.

The method <NUM>, at block <NUM>, determines the available real space <NUM>. Next, at blocks <NUM>, <NUM>, the method determines a constrained virtual space <NUM> that corresponds to the available real space <NUM> and determines available mediated reality content <NUM>. The available mediated reality content <NUM> comprises multiple representations of the virtual space when observed from multiple virtual locations within the constrained virtual space <NUM>. The representations can include virtual visual scenes and/or sound scenes.

Next, at block <NUM>, the method automatically determines a sub-set <NUM> of the available mediated reality content <NUM> without requiring the user <NUM> to change location <NUM> in the real space <NUM>. In this example the block <NUM> comprises, at block <NUM>, generation of a preview based on the available mediated reality content <NUM>. This preview is rendered to the user <NUM>. The sub-set <NUM> of the available mediated reality content <NUM> is determined, at block <NUM>, during rendering of the preview. The determination is based upon which portions of the available mediated reality content <NUM> attract the user's attention. This can for example be determined by monitoring where and for how long a user looks at particular available mediated reality content <NUM>. The content that is most popular based on this monitoring, is then included in the sub-set <NUM> of the available mediated reality content <NUM>. Finally, at block <NUM>, the method <NUM> causes rendering of at least some of the determined sub-set <NUM> of the available virtual reality content to the user <NUM>. This can for example be achieved by redetermining the constrained virtual space <NUM> that corresponds to the available real space <NUM> such that the new available mediated reality content observable from the virtual locations within the new constrained virtual space <NUM> are determined by the sub-set <NUM> of the available mediated reality content. This corresponds to a re-location of the constrained virtual space <NUM>. The constrained virtual space <NUM> may, in addition, be reoriented and/or resized.

The operation of the method of <FIG> may be further understood by reference to <FIG> and <FIG> and <FIG>. <FIG> illustrates an initial placement of a constrained virtual space <NUM>, having a boundary <NUM>, relative to particular content <NUM> in the virtual space. <FIG> illustrates a trajectory <NUM> of a fly-through that may be used to generate a preview of the content <NUM>. <FIG> illustrates a direction <NUM> in which the user looks, relative to the trajectory <NUM>, during the preview. It can be seen that the directions <NUM> point towards a particular portion 190A of the content <NUM>. In this example, the user looks towards particular interactive objects <NUM>. <FIG> illustrates a reorientation and/or relocation and/or re-sizing of the constrained virtual space <NUM> as a consequence of the user's attention towards the portion 190A of the content <NUM>. In the example illustrated, there is a reorientation and relocation of the constrained virtual space <NUM> as a consequence of the user's attention towards the portion 190A. It can be seen that the constrained virtual space <NUM> has been positioned and oriented such that it overlaps with the portion of the virtual space that corresponds to the portion of the content 190A.

Consequently, the sub-set <NUM> of available mediated reality content <NUM> corresponds to the portion 190A and comprises a sub-set of the multiple representations of the virtual space when observed from a sub-set of virtual locations within the constrained virtual space <NUM>, wherein the sub-set of virtual locations are selected by the user during rendering of the preview. In this example the selecting of the sub-set of virtual locations is based upon a gaze direction (the direction in which the user <NUM> is looking) and gaze duration (time period during which user <NUM> is looking in a particular direction) during the preview.

<FIG> illustrates an example of a controller <NUM>. Implementation of a controller <NUM> may be as controller circuitry. The controller <NUM> may be implemented in hardware alone, have certain aspects in software including firmware alone or can be a combination of hardware and software (including firmware).

The memory <NUM> stores a computer program <NUM> comprising computer program instructions (computer program code) that controls the operation of the apparatus <NUM> when loaded into the processor <NUM>. The computer program instructions, of the computer program <NUM>, provide the logic and routines that enables the apparatus to perform the methods illustrated in <FIG> and <FIG>. The processor <NUM> by reading the memory <NUM> is able to load and execute the computer program <NUM>.

This operation may be configured to occur in response to switching to first-person perspective-mediated reality from third person perspective-mediated reality. That is it is a calibration step in setting up first-person perspective-mediated reality.

As illustrated in <FIG>, the computer program <NUM> may arrive at the apparatus <NUM> via any suitable delivery mechanism <NUM>. The delivery mechanism <NUM> may be, for example, a machine readable medium, a computer-readable medium, a non-transitory computer-readable storage medium, a computer program product, a memory device, a record medium such as a Compact Disc Read-Only Memory (CD-ROM) or a Digital Versatile Disc (DVD) or a solid state memory, an article of manufacture that comprises or tangibly embodies the computer program <NUM>. The delivery mechanism may be a signal configured to reliably transfer the computer program <NUM>. The apparatus <NUM> may propagate or transmit the computer program <NUM> as a computer data signal.

Computer program instructions for causing an apparatus to perform at least the following or for performing at least the following:.

The blocks illustrated in the <FIG> and <FIG> may represent steps in a method and/or sections of code in the computer program <NUM>. The illustration of a particular order to the blocks does not necessarily imply that there is a required or preferred order for the blocks and the order and arrangement of the block may be varied. Furthermore, it may be possible for some blocks to be omitted.

<FIG> illustrates an example of a system <NUM> comprising the apparatus <NUM> and a head mounted display apparatus <NUM>.

The head mounted display apparatus <NUM> comprises a display <NUM> that is not a see-through display. The display is configured to display virtual reality content, for example, a virtual visual scene. The head mounted display apparatus <NUM> can, in some examples, comprise audio output device(s) <NUM>. The audio output device(s) <NUM> is/are configured to render virtual reality content, for example, a sound scene.

The head mounted display apparatus <NUM> is configured to enable tracking of an orientation <NUM> of a user <NUM> wearing the head mounted display apparatus <NUM>. In this example, the head mounted display apparatus <NUM> comprises positioning circuitry <NUM> that enables tracking of a location <NUM> and enables tracking of a head orientation <NUM> of the user <NUM> wearing the head mounted apparatus <NUM>.

The apparatus <NUM> is, for example as previously described. In some examples, it comprises means for:.

As used here 'module' refers to a unit or apparatus that excludes certain parts/components that would be added by an end manufacturer or a user. The controller <NUM> my be a module, for example.

Although embodiments have been described in the preceding paragraphs with reference to various examples, it should be appreciated that modifications to the examples given can be made without departing from the scope of the claims.

Claim 1:
An apparatus comprising means for:
determining an available real space (<NUM>), comprising a portion of a real space (<NUM>), by determining a boundary of the available real space (<NUM>), wherein the available real space (<NUM>) comprises locations (<NUM>) available to a user (<NUM>) to control a corresponding virtual location (<NUM>) of a virtual user (<NUM>), via first-person perspective-mediated reality, wherein first-person perspective-mediated reality creates a correspondence between a location and orientation (<NUM>) of a user in real space to a virtual location and virtual orientation (<NUM>) of a virtual user;
determining a constrained virtual space (<NUM>), comprising a portion of a virtual space (<NUM>, <NUM>), by determining a boundary of the constrained virtual space (<NUM>), wherein the constrained virtual space corresponds to the available real space;
determining available mediated reality content (<NUM>) which comprises multiple representations of the virtual space when observed from multiple virtual locations within the constrained virtual space;
automatically determining a sub-set (<NUM>) of the available mediated reality content which comprises multiple representations of the virtual space when observed from multiple virtual locations within the constrained virtual space, without requiring the user to change location in the real space; and
causing rendering of at least some of the determined sub-set of the available virtual reality content which comprises multiple representations of the virtual space when observed from multiple virtual locations within the constrained virtual space to the user.